Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add internal properties to disable caching of non-task outputs #28827

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

bigdaz
Copy link
Member

@bigdaz bigdaz commented Apr 15, 2024

Adds properties that can be used to experimentally disable caching of:

  • Kotlin and Groovy script compilation: -Dorg.gradle.experimental.script-caching-disabled
  • Artifact Transform execution: -Dorg.gradle.internal.transform-caching-disabled

It is also possible to disable caching for select artifact transforms:
-Dorg.gradle.internal.transform-caching-disabled=org.example.Transform1,org.example.Transform2

These properties exist solely for the purpose of investigating the impact of build-cache on non-task outputs. They are not intended to be supported long term and will be removed (or replaced) in a future Gradle version without deprecation.

@bigdaz bigdaz added this to the 8.8 RC1 milestone Apr 15, 2024
@bigdaz bigdaz force-pushed the dd/remote-cache-experiment branch from 068c76d to efd3260 Compare April 15, 2024 20:14
@bigdaz
Copy link
Member Author

bigdaz commented Apr 15, 2024

@bot-gradle test this

@bot-gradle
Copy link
Collaborator

I've triggered the following builds for you. Click here to see all build failures.

@bigdaz bigdaz force-pushed the dd/remote-cache-experiment branch from efd3260 to 3abbd21 Compare April 15, 2024 22:15
@bigdaz bigdaz marked this pull request as ready for review April 16, 2024 03:43
@bigdaz bigdaz requested review from a team as code owners April 16, 2024 03:43
@bigdaz bigdaz requested review from tresat and asodja and removed request for a team and tresat April 16, 2024 03:43
@asodja asodja requested a review from lptr April 16, 2024 08:15
@bigdaz
Copy link
Member Author

bigdaz commented Apr 16, 2024

@asodja Do you think we need integration tests for these experimental properties? My preference is to keep it minimal.

@@ -291,6 +294,10 @@ public void markLegacySnapshottingInputsFinished(CachingState cachingState) {

@Override
public Optional<CachingDisabledReason> shouldDisableCaching(@Nullable OverlappingOutputs detectedOverlappingOutputs) {
if (System.getProperty(CACHING_DISABLED_PROPERTY) != null) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ Should we also check if value is true or false?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had that at first, but instead I decided to make it a no-value property.
Usage is ./gradlew -Dorg.gradle.experimental.transform-caching-disabled task.

@@ -61,6 +67,15 @@ public BuildScriptCompilationAndInstrumentation(
this.transformFactory = transformFactory;
}

@Override
public Optional<CachingDisabledReason> shouldDisableCaching(@Nullable OverlappingOutputs detectedOverlappingOutputs) {
if (System.getProperty(CACHING_DISABLED_PROPERTY) != null) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ Also here, should we also check if value is true or false?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The plan is to support usage like gradle -Dorg.gradle.experimental.script-caching-disabled foo (without requiring a value).

For transforms, I've updated this to allow a comma-separated list of transforms to disbaled, with empty string meaning disable all transforms. I suggest we keep this logic for scripts.

Since this is an experimental property, I'm not overly concerned with usability.

@asodja
Copy link
Member

asodja commented Apr 16, 2024

@asodja Do you think we need integration tests for these experimental properties? My preference is to keep it minimal.

I think it's fine. I guess we have coverage that build cache works without a flag, at least for artifact transform?

FYI: I will leave @lptr to review and approve this PR, since he said he will also look in to the spec.

@bigdaz bigdaz requested a review from a team April 17, 2024 18:05
@bigdaz bigdaz force-pushed the dd/remote-cache-experiment branch from 979df49 to 5c763b4 Compare April 17, 2024 18:15
Adds properties that can be used to experimentally disable caching of:
- Kotlin and Groovy script compilation: `-Dorg.gradle.experimental.script-caching-disabled`
- Artifact Transform execution: `-Dorg.gradle.experimental.transform-caching-disabled`

These properties exist solely for the purpose of investigating the impact of build-cache
on non-task outputs. They are not intended to be supported long term and will be removed
(or replaced) in a future Gradle version without deprecation.
It will be useful to assess the build-cache impact on specific artifact transforms.
Such measurements may be useful to inform developers of 3rd party plugins as to
whether caching is useful for their transform implementations.

Signed-off-by: daz <daz@gradle.com>
@bigdaz bigdaz force-pushed the dd/remote-cache-experiment branch from 5c763b4 to 68477b3 Compare April 17, 2024 18:18
@bigdaz
Copy link
Member Author

bigdaz commented Apr 17, 2024

@bot-gradle test rfm

@gradle gradle deleted a comment from bigdaz Apr 17, 2024
@bot-gradle
Copy link
Collaborator

I've triggered the following builds for you. Click here to see all build failures.

@lptr lptr changed the title Add experimental properties to disable caching of non-task outputs Add internal properties to disable caching of non-task outputs Apr 19, 2024
@bigdaz bigdaz requested a review from a team as a code owner April 19, 2024 14:16
@lptr lptr force-pushed the dd/remote-cache-experiment branch from c81475e to 4c1bbd7 Compare April 19, 2024 14:18
@bigdaz bigdaz modified the milestones: 8.8 RC1, 8.9 RC1 Apr 19, 2024
@ljacomet ljacomet modified the milestones: 8.9 RC1, 8.8 RC1 Apr 22, 2024
@ljacomet ljacomet changed the base branch from release to master April 23, 2024 16:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants