Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add AGORA invariants #497

Open
wants to merge 129 commits into
base: master
Choose a base branch
from

Conversation

JuanCarlosAlonsoValenzuela

Add 22 new invariants designed for the context of REST API testing. These invariants are based on an analysis of a systematically collected dataset of 40 realistic APIs from the RapidAPI platform and were used in the paper entitled “AGORA: Automated Generation of test oracles for REST APIs”, accepted for publication at ISSTA 2023. All the new invariants are disabled by default. A complete description of these invariants can be found in the header of each invariant class and here: https://github.com/isa-group/Beet/wiki/Modified-version-of-Daikon#added-invariants

Modify computeConfidence function of certain invariants that always returned CONFIDENCE_JUSTIFIED (Range.java.jpp, Member.java.jpp, Reverse.java.jpp, SubSequence.java.jpp, SubSet.java.jpp), now these functions return CONFIDENCE_JUSTIFIED if there is at least one occurrence of the invariant during the program execution.

Add 22 new invariants designed for the context of REST API testing. These invariants are based on an analysis of a systematically collected dataset of 40 realistic APIs from the RapidAPI platform. All the new invariants are disabled by default.

Modify computeConfidence function of certain invariants that always returned CONFIDENCE_JUSTIFIED (Range.java.jpp, Member.java.jpp, Reverse.java.jpp, SubSequence.java.jpp, SubSet.java.jpp), now these functions return CONFIDENCE_JUSTIFIED if there is at least one occurrence of the invariant during the program execution.
java/daikon/Daikon.java Outdated Show resolved Hide resolved
java/typequals/vindex/VIndexAnnotatedTypeFactory.java Outdated Show resolved Hide resolved
@JuanCarlosAlonsoValenzuela JuanCarlosAlonsoValenzuela changed the title Add AGORA invariants and modifications in computeConfidence functions Add AGORA invariants Jul 14, 2023
@mernst
Copy link
Member

mernst commented Oct 7, 2023

@JuanCarlosAlonsoValenzuela
In several Agora invariants, the merge(List, PptTopLevel) method is not defined. This is leading to incorrect results, because they inherit an implementation that does not work correctly. This was in turn leading to test failures.

I have enhanced Daikon to warn about missing overrides of that method. You can see errors like this in the misc job:

Exception in thread "main" java.lang.Error: daikon.inv.unary.string.FixedLengthString defines no merge method, but these fields might store state: length

Could you please correct the invariants so that their merge(List, PptTopLevel) methods behave correctly?

I realize that the tests still won't pass at that point; right now, I am only asking you to eliminate all the errors about merge not being overridden.

@JuanCarlosAlonsoValenzuela
Copy link
Author

@mernst

I have added all the merge methods for the AGORA invariants causing the new exception that you mentioned in your previous comment (missing merge method).
I also implemented the clone() method for all these invariants.
I still have one doubt, in your comment and in the exception message, the method is called merge(List, PptTopLevel), but I think that it should be merge(List, PptSlice).
On a side note, I am still having issues to install Daikon using JDK 17 and I had to use Java 8 to test the implementation of these changes (see comment below).

@JuanCarlosAlonsoValenzuela
Copy link
Author

@mernst
As I mentioned in my previous comment, I am still facing some issues to install Daikon using Java 17. The most likely scenario is that I made some mistake when following the steps, but just in case I will share the steps that I followed in case you find that I did something wrong (I have also created a VM with the results of these steps to ease replication). In the worst case, maybe this could help to improve the documentation.

It is worth mentioning that I did not run into these issues when installing a Daikon release (available at: https://plse.cs.washington.edu/daikon/download/).

The steps that I followed in a Ubuntu VM with Java 17are the following:

  1. Clone Daikon from the branch containing this PR.
  2. Create $DAIKONDIR variable in /home/jcav/Documents/GitHub/daikon by adding the following lines to the ~/.bashrc file:
    export DAIKONDIR=/home/jcav/Documents/GitHub/daikon
    source $DAIKONDIR/scripts/daikon.bashrc
  3. Install all the requirements for compiling Daikon (specified here: https://plse.cs.washington.edu/daikon/download/doc/developer.html#Requirements-for-compiling-Daikon), more specifically, the ones in this Dockerfile: https://github.com/codespecs/daikon/blob/master/scripts/Dockerfile-ubuntu-jdk17-plus
  4. Compile Daikon and build other tools by running: make -C $DAIKONDIR rebuild-everything
  5. Open the project using IntelliJ idea, set directory /daikon/java as Sources Root and add all jars inside the lib directory. I did not add the tools.jar (as specified here https://plse.cs.washington.edu/daikon/download/doc/developer.html#Using-Eclipse) jar because it is not present since JDK 9 (https://stackoverflow.com/questions/69353634/tools-jar-does-not-come-with-jdk-installation).
  6. After these steps (and introducing the changes of my latest commits), I ran Daikon.java and found the following exceptions:
  • File: java/daikon/dcomp-transfer/DCRuntime.java.
    • Error: java: package exists in another module: java.base.
  • File: java/typequals/vindex/VIndexAnnotatedTypeFactory.java
    • Error 1: Line 25, Cannot find symbol class VIndexTop. This can be solved simply by importing it: import typequals.vindex.qual.VIndexTop, but this import is not added by default.
    • Error 2: Line 52, cannot find method leastUpperBound.
    • It is worth mentioning that, for some reason, this file is not part of the Daikon releases that can be downloaded here: https://plse.cs.washington.edu/daikon/download/ When implementing AGORA, I used the latest release, hence why I did not find this issue until now.

Of course, I did not commit these changes without testing them first. I tested them with my computer using Java 8 and the declaration and data traces generated by Beet, but I still had to remove the /java/typequals/vindex/VIndexAnnotatedTypeFactory.java file.

I have sent you an email containing an Ubuntu VM with the results of the aforementioned steps, the subject of the email is “Daikon VM”. Let me know if you need more details about this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants