Gradle crashes if GRADLE_RO_DEP_CACHE is set and it cannot create modules-2 directory within it

[maartenh]

Expected Behavior

If the GRADLE_RO_DEP_CACHE is set, the build should never fail because it cannot be written to.

Current Behavior

When I create a directory for the shared read-only cache, and make it available as an empty read-only directory, with no modules-2 subdirectory present, the build crashes when it tries to create the modules-2 subdirectory.
Please find the full failure report including stacktrace in the attached file.

Context

I'm using Jenkins 2 with builds occurring in ephemeral containers. Using directories mapped from the host server into the container for the read-only dependency cache.
The goal is to minimise the required downloads and the build time increases caused by those downloads.
My current workaround for this issue is the pre-create the modules-2 directory, but that will cause a failure if Gradle ever moves to a modules-3 version.

Steps to Reproduce

Please run the following script to reproduce:

mkdir gradle-shared-cache-test
cd gradle-shared-cache-test
mkdir shared-cache
chmod 444 shared-cache
export GRADLE_RO_DEP_CACHE="`pwd`/shared-cache"
gradle init --stacktrace --info

Your Environment

Gradle 6.2
Running on Linux in docker container, or MacOS 10.15.3 locally.

[maartenh]

failure-exception.txt

[melix]

Thanks for the report. Maybe it's unclear but the shared read-only directory should always be pre-populated. You mustn't have one build with write access to it and others with read-access: all builds must have read-only access so don't let any Gradle build write directly to it.

[maartenh]

Thanks for the report. Maybe it's unclear but the shared read-only directory should always be pre-populated. You mustn't have one build with write access to it and others with read-access: all builds must have read-only access so don't let any Gradle build write directly to it.

Thank you. The requirement for having it pre-populated was not clear to me from the documentation.
And indeed, I do give all Gradle builds read-only access to it. The intend (once I see actual build time improvements from it) is to have only the master branch build update the cache after the Gradle build is finished.

[ljacomet]

Explicit notes have been added to the documentation to clarify the need of having a populated read-only cache.

[ChristianCiach]

I ran into the same issue and I still think it's bad that Gradle crashes if the directory is empty.

This directory is always read-only. This is even reflected by the property name. So I don't see why Gradle should ever even attempt to write to it (by attempting to create directories). I think Gradle should just behave as if the property hasn't been set (maybe logging a warning or info) if the directory is empty, doesn't exist at all or cannot be used for any other reason.

My use case: I tried to define "GRADLE_RO_DEP_CACHE" at the Jenkins system configuration, making it available to all builds. This works fine for all builds running directly on the host, but unfortunately, this immediately caused all builds that are executed inside a docker container to fail. I really think Gradle should just swallow a non-existing (sub)directory in this case.

[ljacomet]

That's a fair point. Let us work on fixing that for the next release.

[melix]

-1 to silently failing. This indicates a mis-configuration. It's important to fix the configuration problem. We can avoid crashing, but we shouldn't silence problems.

[ChristianCiach]

-1 to silently failing. This indicates a mis-configuration.

Of course I agree with you! This was badly worded on my part. By "swallowing" I just meant "not crashing". A log message is perfectly adequate for this situation.

[prakashul]

Hi folks,
I am facing a related issue. I have pre-populated the to be read-only dependency directory. I made it read-only and set variables GRADLE_HOME (separate writable local cache dir) and GRADLE_RO_DEP_CACHE (for read-only cache).
Turns out Gradle tries to rewrite modules-2 dir in GRADLE_RO_DEP_CACHE and ignores and/or overrides the GRADLE_HOME variable. Thus the process crashes with exceptions of failed to write to the read-only directory. The concept of local cache (writable) and a read-only cache is not honoured.

Gradle version used: 6.2.1

[ljacomet]

Hey @prakashul ,

Could you share more specific information about what you tried exactly?
Things like:

1. What did you copy?
2. Where did you copy it?
3. Which value did you put for GRADLE_RO_DEP_CACHE?

For example, this would be a valid setup:

1. Copy /users/foo/.gradle/caches/modules-2
2. Into /shared/gradle-cache/ so I have /shared/gradle-cache/modules-2
3. Add GRADLE_RO_DEP_CACHE=/shared/gradle-cache/ to system env

[prakashul]

Hi @ljacomet

Thanks for clearing that up, the solution worked. By your example, i had copied /users/foo/.gradle/caches to a new location.
Copying /users/foo/.gradle/caches/modules-2 and setting the env var to <CACHE_DIR> solved the problem.

Thanks :)