🚧 Third party repository detection probe #323

Jagrutiti · 2023-06-01T16:58:40Z

Description

This probe will check:

Whether third party repositories is being used. If yes, it may list the repositories .
If no third party repositories are used, it will succeed.

Closes #299

Submitter checklist

If the issue exists, it is well described and linked in the description
The description of this pull request is detailed and explain why this pull request is needed
The changeset is on a specific branch
- feature/ for new feature, or improvements
- fix/ for bug fixes
- docs/ for any documentation changes
If required, the documentation has been updated
There is automated tests to cover the code change / addition
- If there is no test, include an explanation why in the description
Run mvn verify locally and all tests are passing successfully
- It is OK to create a pull request which has failing tests if it is created as a draft, is to fix a bug and the first commit is the test to prove the existence of the bug.
There is no new warnings (checkstyle nor spotbugs) on the code

alecharp

I think it would be good to have multiple pom.xml files (in different folders) to have

one with one repository with the correct url
one with one repository with url pointing outside of the Jenkins environement
one with two repositories, one ok and one not.

core/src/test/resources/fake-test-directory/pom.xml

Co-authored-by: Adrien Lecharpentier <adrien.lecharpentier@gmail.com>

...test/java/io/jenkins/pluginhealth/scoring/probes/ThirdPartyRepositoryDetectionProbeTest.java

alecharp

Would it be possible to have no repository configured in the pom.xml?

core/pom.xml

.gitignore

...src/main/java/io/jenkins/pluginhealth/scoring/probes/ThirdPartyRepositoryDetectionProbe.java

...test/java/io/jenkins/pluginhealth/scoring/probes/ThirdPartyRepositoryDetectionProbeTest.java

Jagrutiti · 2023-06-03T15:13:54Z

Would it be possible to have no repository configured in the pom.xml?

Maybe

...src/main/java/io/jenkins/pluginhealth/scoring/probes/ThirdPartyRepositoryDetectionProbe.java

.gitignore

Co-authored-by: Adrien Lecharpentier <adrien.lecharpentier@gmail.com>

…/Jagrutiti/plugin-health-scoring into third-party-repository-detection

...src/main/java/io/jenkins/pluginhealth/scoring/probes/ThirdPartyRepositoryDetectionProbe.java

Jagrutiti · 2023-06-10T12:16:12Z

...src/main/java/io/jenkins/pluginhealth/scoring/probes/ThirdPartyRepositoryDetectionProbe.java

+    public static final String KEY = "third-party-repository-detection-probe";
+    final String hostName = "https://repo.jenkins-ci.org";
+//    final String parentPom = "https://raw.githubusercontent.com/jenkinsci/plugin-pom/master/pom.xml";
+    final String parentPom = "https://github.com/jenkinsci/plugin-pom/blob/master/pom.xml";


I wanted to use this in the beginning but now I am not.

For test cases, I do not need to. But I am not sure about real-world scenarios.

I was unable to read the pom from the URL in MavenXpp3Reader

I don't think this is what we want here.

…/Jagrutiti/plugin-health-scoring into third-party-repository-detection

core/src/main/java/io/jenkins/pluginhealth/scoring/probes/Probe.java

...src/main/java/io/jenkins/pluginhealth/scoring/probes/ThirdPartyRepositoryDetectionProbe.java

alecharp · 2023-06-14T09:04:38Z

...src/main/java/io/jenkins/pluginhealth/scoring/probes/ThirdPartyRepositoryDetectionProbe.java

+        return new String[] { SCMLinkValidationProbe.KEY};
+    }
+
+    public Model parsePomFromUrl(String pomUrl) {


I don't like this because what it happening if someone puts an url pointing to a fake pom.xml which run random code?

You mean what if there is a fake parent path in the child pom.xml ?

How do I validate the path? What is the correct way to resolve this? I am clueless here.

One thing we can do is check the extension of the file. Whether it ends with .xml or the file name is pom.xml.

I don't think we should rely on the relativePath tag of the parent.
We should look if the plugin has a folder configured (module) and if so, we should check that the parent pom of the repository has a parent to what we expect, that the module has a parent to the root pom file and that none of them have a third party repository.

core/src/main/java/io/jenkins/pluginhealth/scoring/probes/Probe.java

alecharp · 2023-06-14T09:07:08Z

...src/main/java/io/jenkins/pluginhealth/scoring/probes/ThirdPartyRepositoryDetectionProbe.java

+    public static final String KEY = "third-party-repository-detection-probe";
+    final String hostName = "https://repo.jenkins-ci.org";
+//    final String parentPom = "https://raw.githubusercontent.com/jenkinsci/plugin-pom/master/pom.xml";
+    final String parentPom = "https://github.com/jenkinsci/plugin-pom/blob/master/pom.xml";


I don't think this is what we want here.

...src/main/java/io/jenkins/pluginhealth/scoring/probes/ThirdPartyRepositoryDetectionProbe.java

alecharp · 2023-06-14T09:22:46Z

...src/main/java/io/jenkins/pluginhealth/scoring/probes/ThirdPartyRepositoryDetectionProbe.java

+            allRepositories.addAll(model.getRepositories());
+            allRepositories.addAll(model.getPluginRepositories());
+
+            if (!model.getParent().getRelativePath().isBlank()) {
+                Model parentPomModel = parsePomFromUrl(model.getParent().getRelativePath());
+                allRepositories.addAll(parentPomModel.getRepositories());
+                allRepositories.addAll(parentPomModel.getPluginRepositories());
+            }
+            for (Repository repository : allRepositories) {
+                if (!repository.getUrl().startsWith(hostName)) {
+                    return ProbeResult.failure(KEY, "Third party repositories detected in the plugin");
+                }
+            }


I'm sure you can simplify that by filtering the model repositories before adding them to the set.

On what parameters you want me to filter them?

Co-authored-by: Adrien Lecharpentier <adrien.lecharpentier@gmail.com>

alecharp · 2023-06-16T10:53:25Z

To complete this pull request, we need to be able to generate the effective-pom of a project. To do so, the project is missing a complex system which is larger than the current pull request and need to be address separately.

We are not throwing away the current content but delaying it until we have the extra service.

Jagrutiti · 2023-09-03T12:17:22Z

To complete this pull request, we need to be able to generate the effective-pom of a project. To do so, the project is missing a complex system which is larger than the current pull request and need to be address separately.

I have added a prototype for the effective-pom service.

As per my understanding, the test cases that are failing are correct. They cannot pass in the case when effective-pom is generated because third-party repository exists in the super pom.

Started with writing test cases for the probe

8d74f5c

alecharp reviewed Jun 2, 2023

View reviewed changes

Jagrutiti and others added 7 commits June 2, 2023 20:19

Removing unwanted details from test pom file.

7d62076

Co-authored-by: Adrien Lecharpentier <adrien.lecharpentier@gmail.com>

Adding dummy to pom test files

4286ff4

Adding dummy values to pom test files

f412657

Restoring valuable test data

537d68a

Updating the code and test cases

491d16d

Removing unused imports

3f593a4

Fixing code

365db47

Jagrutiti commented Jun 3, 2023

View reviewed changes

...test/java/io/jenkins/pluginhealth/scoring/probes/ThirdPartyRepositoryDetectionProbeTest.java Outdated Show resolved Hide resolved

alecharp reviewed Jun 3, 2023

View reviewed changes

Jagrutiti changed the title ~~Third party repository detection probe~~ feature/third party repository detection probe Jun 3, 2023

alecharp reviewed Jun 3, 2023

View reviewed changes

...src/main/java/io/jenkins/pluginhealth/scoring/probes/ThirdPartyRepositoryDetectionProbe.java Outdated Show resolved Hide resolved

Undid irrelevant change

3bb27f5

alecharp reviewed Jun 3, 2023

View reviewed changes

.gitignore Outdated Show resolved Hide resolved

Jagrutiti changed the title ~~feature/third party repository detection probe~~ Third party repository detection probe Jun 3, 2023

Apply suggestions from code review

99a1379

Co-authored-by: Adrien Lecharpentier <adrien.lecharpentier@gmail.com>

Jagrutiti closed this Jun 3, 2023

Jagrutiti deleted the third-party-repository-detection branch June 3, 2023 15:40

Jagrutiti restored the third-party-repository-detection branch June 3, 2023 15:43

Jagrutiti reopened this Jun 3, 2023

Jagrutiti added 4 commits June 3, 2023 21:46

Restoring .gitignore

e050c24

Merge branch 'third-party-repository-detection' of https://github.com…

d830b63

…/Jagrutiti/plugin-health-scoring into third-party-repository-detection

Updated the code and the test case

fda43ae

Updating the test case

c5c3530

Jagrutiti marked this pull request as ready for review June 3, 2023 17:07

Jagrutiti requested a review from alecharp June 3, 2023 17:07

alecharp reviewed Jun 3, 2023

View reviewed changes

...src/main/java/io/jenkins/pluginhealth/scoring/probes/ThirdPartyRepositoryDetectionProbe.java Show resolved Hide resolved

updating the code and revamping the test cases

5dfd1ed

Jagrutiti requested a review from alecharp June 10, 2023 12:07

Jagrutiti self-assigned this Jun 10, 2023

Jagrutiti commented Jun 10, 2023

View reviewed changes

Jagrutiti added 10 commits June 10, 2023 17:52

Undoing editorconfig change

039362e

Adding additional test files

2f1638e

Fixed code and checkstyle

5560353

Removing the effective-pom file

ce97eb0

Added spotbugs fixes

148d4dd

Added spotbugs fixes

dbe70d5

Merge branch 'main' into third-party-repository-detection

276968c

Added spotbugs fixes

0be6114

Merge branch 'third-party-repository-detection' of https://github.com…

9343f56

…/Jagrutiti/plugin-health-scoring into third-party-repository-detection

Removing unused imports

9d43283

alecharp requested changes Jun 14, 2023

View reviewed changes

Jagrutiti and others added 4 commits June 14, 2023 20:45

Apply suggestions from code review

6d42558

Co-authored-by: Adrien Lecharpentier <adrien.lecharpentier@gmail.com>

Fixed missing symbols error

3659137

Fixing checkstyle issues

58ba0b1

Merge branch 'main' into third-party-repository-detection

da55c4d

alecharp marked this pull request as draft June 16, 2023 10:53

alecharp changed the title ~~Third party repository detection probe~~ 🚧 Third party repository detection probe Jun 16, 2023

Jagrutiti added 2 commits September 3, 2023 17:30

Adding code for effective-pom generation and updated the test cases:

fc8ec3b

Adding comments

d77baf1

Jagrutiti requested review from aneveux, alecharp and lemeurherve September 3, 2023 12:12

Merge branch 'main' into third-party-repository-detection

9b23f07

Jagrutiti added 2 commits September 7, 2023 22:32

Merge branch 'main' into third-party-repository-detection

85c10f3

Fixed merge issues

1fa7c2b

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🚧 Third party repository detection probe #323

🚧 Third party repository detection probe #323

Jagrutiti commented Jun 1, 2023 •

edited

alecharp left a comment

alecharp left a comment

Jagrutiti commented Jun 3, 2023

Jagrutiti Jun 10, 2023

alecharp Jun 14, 2023

alecharp Jun 14, 2023

Jagrutiti Jun 14, 2023 •

edited

Jagrutiti Jun 14, 2023 •

edited

Jagrutiti Jun 14, 2023

alecharp Jun 15, 2023

alecharp Jun 14, 2023

alecharp Jun 14, 2023

Jagrutiti Jun 14, 2023

alecharp commented Jun 16, 2023

Jagrutiti commented Sep 3, 2023

🚧 Third party repository detection probe #323

Are you sure you want to change the base?

🚧 Third party repository detection probe #323

Conversation

Jagrutiti commented Jun 1, 2023 • edited

Description

Submitter checklist

alecharp left a comment

Choose a reason for hiding this comment

alecharp left a comment

Choose a reason for hiding this comment

Jagrutiti commented Jun 3, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Jagrutiti Jun 14, 2023 • edited

Choose a reason for hiding this comment

Jagrutiti Jun 14, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alecharp commented Jun 16, 2023

Jagrutiti commented Sep 3, 2023

Jagrutiti commented Jun 1, 2023 •

edited

Jagrutiti Jun 14, 2023 •

edited

Jagrutiti Jun 14, 2023 •

edited