Build logic classpath instrumentation should produce reproducible jars

[eskatos]

Expected Behavior

Jars produced by the build logic classpath instrumentation are reproducible.

Current Behavior

Jars produced by the build logic classpath instrumentation are not reproducible.

When using Java 8, their manifest differs. The entries are in a different order.
That's because in Java 8, the java.util.jar.Manifest class uses HashMaps.
This is fixed in Java 11 but we need to support Java 8.

Context

https://github.com/gradle/gradle/blob/v6.5.1/subprojects/core/src/main/java/org/gradle/internal/classpath/InstrumentingClasspathFileTransformer.java#L81

[eskatos]

Note that with 6.6 normalizing manifest entries order by default this won't cause cache misses. But always producing the same jars would reduce confusion when investigating cache misses due to other problems.

[walkerburgin]

We recently tracked down an unexpected cache miss between our local dev environment and the remote build cache populated by CI to five dependency jars containing .class files with different contents (e.g. differences in constant pools/offsets). We're on Gradle 7.6.1, local dev is on OSX, and CI is on Linux if it matters.

[mlopatkin]

@walkerburgin

Sorry for the late response.

We recently tracked down an unexpected cache miss between our local dev environment and the remote build cache populated by CI to five dependency jars containing .class files with different contents (e.g. differences in constant pools/offsets). We're on Gradle 7.6.1, local dev is on OSX, and CI is on Linux if it matters.

It can be a case of #24031 if someone poisoned the cache with Gradle 8.0 or 8.0.1 somewhere (these had bugs with incorrect cache keys potentially clashing with 7.6). Otherwise more info about the jar differences would be appreciated, please file a new issue for this.

[mlopatkin]

As of 6.8 we no longer rewrite manifests, so this part is determenistic. However, input normalization added in 8.4 muddies the waters. Suppose you have two variations of input, A1.jar and A2.jar, both have the same fingerprint (e.g. they differ in the archive entry order). Instrumented JARs inherit the entry order, so instrumentation is going to produce different outputs, but store them with the same cache key (key is the fingerprint). This might be problematic for the same reasons it was problematic in 8.0 line. Or might be not, because instrumented JARs are normalized the same way when used as inputs.