Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

InstrumentingTransformer generates different class files in Gradle 8 and 7.6 which leads to Remote Build-Cache misses #24031

Closed
AndreasTu opened this issue Feb 24, 2023 · 11 comments
Assignees
Labels
Milestone

Comments

@AndreasTu
Copy link

AndreasTu commented Feb 24, 2023

The InstrumentingTransformer generates different class files in Gradle8 and 7.6, which are generated in the jars-9 folder, which is shared by multiple Gradle-Versions. These are causing Remote-Build-Cache misses.

Expected Behavior

Builds executed in any order and any Gradle version on multiple machines, do not lead to Remote-Build-Caches, due to different transformed classes in the jars-9 folder.

Current Behavior

The InstrumentingTransformer generates different class files in Gradle8 and 7.6, which are generated in the jars-9 folder.
Resulting in different jar files in the jars-9 depending on the order of Gradle build executions.
These lead to different Task imlementations on different machines, which produce Remote-Build-Cache misses.
Gradle Enterprise reports this as "Task implementation changed.".

Context

Remote-Build-Cache does not work as expected, because Jenkins-Nodes push different artifacts.
Which leads to degraded build execution performance.

Steps to Reproduce

This happends when two machines execute build on different Gradle versions in different orders.
First machine Jenkins-Node executes a Gradle 7.6 job, then a Gradle 8 job.
Second machine "Developer-Machine" executes a Gradle 8 and gets a cache miss, because a Gradle-Build plugin from "jars-9" has a different hash.

I can also provide samples for differently generated .class files and/or deassembled sections of the class files.

Local:

  • Delete your .gradle/caches/jars-9
  • Start a Gradle 8 build with plugin id 'com.gradle.enterprise' version '3.12.3'
  • Move the .gradle/caches/jars-9 folder into a different location
  • Start a Gradle 7.6 build with plugin id 'com.gradle.enterprise' version '3.12.3'
  • Diff both folders, and the content are not the same for the com.gradle.enterprise plugin.

Your Environment

Two Windows 11 machines.

Gradle 8.0.1 and 7.6
Build scan URL:
I could provide a Build-Scan dump from an air-gapped Gradle-Enterprise installation.

@AndreasTu
Copy link
Author

@wolfs Could you please have a look at that? And check if you guys need more information on that.
Thank you very much for your time.

@jbartok jbartok added a:regression This used to work and removed a:bug labels Feb 27, 2023
@ljacomet ljacomet added in:configuration-cache Configuration Caching and removed to-triage labels Feb 28, 2023
@ljacomet ljacomet added this to the 8.0.2 (unconfirmed) milestone Feb 28, 2023
@ljacomet ljacomet added the 👋 team-triage Issues that need to be triaged by a specific team label Feb 28, 2023
@ljacomet
Copy link
Member

Thank you for your interest in Gradle!

This issue needs a decision from the team responsible for that area. They have been informed, response time may vary.

@mlopatkin mlopatkin added @configuration-cache and removed 👋 team-triage Issues that need to be triaged by a specific team labels Feb 28, 2023
bot-gradle added a commit that referenced this issue Feb 28, 2023
The change in lambda serialization sneaked in without a version bump. This caused false cache hits, when 8.0 was picking up instrumented jars produced by 7.6.

We bump straight to 23 as 8.1 is already at 22 and we don't want to pick up jars produced by 8.1 by mistake.

Fixes #24031

Co-authored-by: Mikhail Lopatkin <mlopatkin@gradle.com>
bot-gradle added a commit that referenced this issue Feb 28, 2023
There's no instrumentation changes going into 8.1, but we had to bump version for 8.0.2 to 23. To avoid clashes when incrementing this one eventually, this commit proactively bumps the version here.

Part of #24031

Co-authored-by: Mikhail Lopatkin <mlopatkin@gradle.com>
@mlopatkin
Copy link
Member

Fixed in #24073

@technoir42
Copy link

@mlopatkin I'm seeing weird behaviour with Gradle API jar being outdated when trying to upgrade from 7.5.1 to 7.6.1 or downgrade back from 7.6.1 to 7.5.1. Can 7.6.1 be affected by this bug?

@mlopatkin
Copy link
Member

@mlopatkin I'm seeing weird behaviour with Gradle API jar being outdated when trying to upgrade from 7.5.1 to 7.6.1 or downgrade back from 7.6.1 to 7.5.1. Can 7.6.1 be affected by this bug?

@technoir42 this bug is 8.0-specific, so it is unlikely. Could you please file another bug and describe the weirdness you see in more detail?

@AndreasTu
Copy link
Author

@mlopatkin My observation was, that if you execute a Gradle 8 build before a 7.6, this poisons the cache also for Gradle 7.6 builds, because it uses the differently generated file with the same cache key. Which will lead to different cache keys also in Gradle 7.6 on that particular machine in regard to a machine only executing 7.6 builds. The thing which helped on my side was to delete the jars-9 folder on the poisoned machine.

@mlopatkin
Copy link
Member

@mlopatkin My observation was, that if you execute a Gradle 8 build before a 7.6, this poisons the cache also for Gradle 7.6 builds, because it uses the differently generated file with the same cache key. Which will lead to different cache keys also in Gradle 7.6 on that particular machine in regard to a machine only executing 7.6 builds. The thing which helped on my side was to delete the jars-9 folder on the poisoned machine.

Your observation is correct, and the solution is the only one that can help 7.6 to get out of this state, thank you for the info.

Basically, what we have in this issue is that despite producing slightly different JARs, both 7.6 and 8.0 are using the same cache key for these JARs, so one picks up cached JARs of the other. If your build runs fine on 7.6, then these JARs should be interchangeable, so the build is most likely safe, only cache misses are the problem.

Cleaning jars-9 on affected machines is the only remedy for 7.6. Gradle 8.0.2 produces different cache keys, and no longer messes up with 7.6 outputs.

@ljacomet
Copy link
Member

ljacomet commented Mar 7, 2023

@mlopatkin If there is something to be fixed in Gradle 7.x to prevent that problem from happening, could you file an issue against the 7.x backport milestone?

@mlopatkin
Copy link
Member

@mlopatkin If there is something to be fixed in Gradle 7.x to prevent that problem from happening, could you file an issue against the 7.x backport milestone?

I don't think we need anything there, just don't mix it with 8.0 and 8.0.1 anymore.

@eskatos
Copy link
Member

eskatos commented Mar 7, 2023

IIUC as soon as you run a build with 8.0 or 8.0.1 you pollute the local cache and any subsequent build with 7.6 on the same machine might get cache misses, no build failure expected, correct?

I'm asking because just don't mix it with 8.0 and 8.0.1 anymore sounds scary to me.

@mlopatkin
Copy link
Member

IIUC as soon as you run a build with 8.0 or 8.0.1 you pollute the local cache and any subsequent build with 7.6 on the same machine might get cache misses, no build failure expected, correct?

I'm asking because just don't mix it with 8.0 and 8.0.1 anymore sounds scary to me.

Correct, no failures. Gradle 8.0 makes all lambdas in build scripts serializable, but the change is self-contained and Gradle 7.6 is able to load these just fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

6 participants