[#2898] Build CI on more realistic combinations (and faster)

[undermyumbrella1]

Fixes #2898

• builds once
• uploads build artifacts
• other jobs/other java version will download this build artifact
• then it will run compiled tests to check that they pass

Considerations:

1. originally matrix jobs are used

• but this mean other jobs will have to depend on all jobs in the matrix to complete
• instead of a single job in the matrix to complete = slower
• Therefore, reusable workflows is used instead
• so each run will be a separate gh job

2. Android job depends on build job's build artifacts

• specifically Task :androidTest:bundleDebugClassesToRuntimeJar
• is this expected behaviour or should android job be able to run successfully, independent of build job?

[TimvdLippe]

Thanks for the PR! I can't trigger the CI pipeline on this, so I think the ci.yml no longer works as expected. Can you update that file to the expected syntax so that we can trigger a build?

[TWiStErRob]

This looks risky, if we miss a task (e.g. by project evolution), it might run a mixed build and there might be no visible indication of it. Also as far as I remember this might fail in Gradle 8, because tasks depending on other tasks got stricter.

I think instead of uploading the build directory, it might be a more stable approach to run publishToAllPublicationsMavenRepository (see https://github.com/mockito/mockito/blob/main/gradle/java-publication.gradle#L100) and only save that folder with the "production-like" jars. The question is how it would be used, and the answer might be Gradle dependency substitution done conditionally combined with a dependency repository referencing the temp repo.

[undermyumbrella1]

agree that it is risky, not sure if my approach was right
any ideas on how to only execute test aka compile once test all? as gradlew test still runs build and compile tasks due to dependent tasks

based on the link, the jar files are published to a local folder, hence they cannot be shared between jobs. also compiled classes and jars are needed together so that compiled classes are used to run the test aka compile once test all. is the idea to upload compiled classes and jars in a remote maven repo to be shared between jobs?

[TWiStErRob]

Based on my current knowledge there are two ways to go about this:

1. substitute the main sourceSet of each project with a pre-built .jar file (based on a -P flag), unsure if this is actually feasible. Gradle has a pretty powerful configuration and dependencies API, so it should be possible. This way when running tests, there's no need to have production .class files, because the tests would use .jar files from the temp repo.
2. Set up toolchains and create multiple test tasks (see how Gradle Stutter plugin creates multiple test tasks for each Java * Gradle version).

The second one would create multiple builds (because of the maintainer's requirements to keep the matrix), but the .jar files would be exactly the same. As far as I read it, in the referenced blog post this is not a problem because there's no matrix and the toolchain approach is used.

The first approach would optimize the build pipeline for matrix, but I've never seen an example like this. At my previous company with did a similar -x hack for a while, but it fell apart during Gradle/Android Gradle Plugin upgrades. Dependency substitution is a less-hacky way of achieving what -x does.

[undermyumbrella1]

the second approach creates different gradle task on the same runner/gh job. From the response in the original issue, it seems different gh job is needed to test for each java version

[TWiStErRob]

Yeah, that works pretty well with matrix: see how to dynamically use a task.

[undermyumbrella1]

Sorry, I am still trying to understand this. Starting with build artifacts, instead of upload-artifacts, use publishToAllPublicationsMavenRepository.

1. this uploads artifacts into a local maven repo. How are these artifacts shared between github jobs? as local repo is local to each runner
2. jar + compiled classes is needed for testing. So is it advisable to upload the entire build folder?

[TWiStErRob]

Still need to upload the artifact of course, otherwise there's no sharing :) But the artifact shouldn't contain build folders, because those are Gradle intermediates, and the files inside won't all be reused if the JVM changes or they're but the tests will be run on an environment that's not natural and will never happen in real life (therefore tests lose their value). The compiled classes are not necessary for testing if the test sourceSet depends on the jars from the repo, instead of the actual classes from the build intermediate folders. I'm not sure how to do this though... this is why I never really started on the issue. Gradle dependency substitution might be an answer, but maybe something else is needed.

With the toolchains approach this is not a problem because Gradle internally and automatically takes care of building the jar with one Java and testing with another. In that case sharing the build folder might be an option, because the environment in all jobs would be the same.

[codecov-commenter]

Codecov Report

Patch and project coverage have no change.

Comparison is base (2300e2b) 85.70% compared to head (29970ed) 85.70%.

Additional details and impacted files

@@            Coverage Diff            @@
##               main    #3028   +/-   ##
=========================================
  Coverage     85.70%   85.70%           
  Complexity     2876     2876           
=========================================
  Files           326      326           
  Lines          8733     8733           
  Branches       1078     1078           
=========================================
  Hits           7485     7485           
  Misses          971      971           
  Partials        277      277           

see 2 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[TWiStErRob]

Tip: when you have just one command you can do this too:

Suggested change

	run: |
	./gradlew test \
	-x compileTestJava -x compileTestGroovy -x compileTestKotlin -x compileTestBundleJava \
	-x classes -x compileJava -x compileGroovy -x compileKotlin -x compileOtherBundleJava \
	-x processResources -x processOtherBundleResources \
	-x preBuild -x preDebugBuild \
	-x jar
	run: >
	./gradlew test
	-x compileTestJava -x compileTestGroovy -x compileTestKotlin -x compileTestBundleJava
	-x classes -x compileJava -x compileGroovy -x compileKotlin -x compileOtherBundleJava
	-x processResources -x processOtherBundleResources
	-x preBuild -x preDebugBuild
	-x jar

probably worth keeping the \ version though for consistency with other run: blocks.