Gradle 7.6: high memory usage (android project)

[rmarma]

After upgrading to version 7.6, memory consumption has increased significantly.

Empty project with 1000 android modules.

Gradle 7.5.1: sync is successful in 3 minutes (10gb memory used)

Gradle 7.6: Syncing failed after 15 minutes (memory needs more than 16gb)

Expected Behavior

Memory consumption as in 7.5.1.

Current Behavior

Memory consumption has increased significantly.

Context

Sync project in Android Studio.

Steps to Reproduce

1. Download this empty project with 1000 android modules https://github.com/rmarma/project-with-android-modules
2. Open in Android Studio (Dolphin)
3. Try to sync

Your Environment

• Windows 10
• JDK corretto-11.0.17
• Android Studio Dolphin | 2021.3.1 Patch 1
• Gradle 7.6
• AGP 7.3.1
• Kotlin 1.7.21

[famod]

FWIW, over at Quarkus we are also seeing memory issues: quarkusio/quarkus#29508 (comment) (no android involved there, though)

[big-guy]

@famod @rmarma I think there is still more I can do here, but could you give 7.6.1-20221222053644+0000 a try? I can get the reproducer project to sync with this nightly.

You can update with ./gradlew wrapper --gradle-version=7.6.1-20221222053644+0000

[rmarma]

@big-guy, I tried version 7.6.1-20221222053644+0000 and got this result:
Sync is successful in 4 minutes (16gb used)

It's better than 7.6, but still worse than 7.5.1.

[famod]

@big-guy

@famod @rmarma I think there is still more I can do here, but could you give 7.6.1-20221222053644+0000 a try?

I have been trying to use that snapshot in the Quarkus CI but for some reason a part of it is still using 7.6. I had to give up for now due to lack of time, sorry!

[bcmedeiros]

I'm getting a java.lang.OutOfMemoryError: Java heap space in 7.6 for my relatively small project with standard Gradle config.
I can confirm that after ./gradlew wrapper --gradle-version=7.6.1-20221222053644+0000 the problem is gone.

[big-guy]

@rmarma @famod could you give 7.6.1-branch-sg_761_reduce_memory_consumption-20230118002823+0000 a try? This contains changes that aren't merged yet, but locally this doesn't run out of memory anymore.

[big-guy]

I updated the comment above to suggest 7.6.1-branch-sg_761_reduce_memory_consumption-20230118002823+0000

[bcmedeiros]

Version 7.6.1-branch-sg_761_reduce_memory_consumption-20230118002823+0000 works for me.

[rmarma]

@big-guy, yes, it works like 7.5.1
7.6.1-branch-sg_761_reduce_memory_consumption-20230118002823+0000
Sync is successful in 3 minutes (10gb used)

Thanks

[trask]

hi! we're also running into OutOfMemoryErrors trying to update from Gradle 7.5 to 7.6.

I tried 7.6.1-branch-sg_761_reduce_memory_consumption-20230118002823+0000 but still running OOM, even after bumping from the default -Xmx512m to -Xmx2g:

open-telemetry/opentelemetry-java-instrumentation#7515

[big-guy]

@trask could you give 7.6.1-branch-sg_761_reduce_memory_consumption_2-20230123043022+0000 a try too?

[trask]

sorry for the delayed response, we're still seeing Expiring Daemon because JVM heap space is exhausted with the new snapshot

[yogurtearl]

@big-guy CI logs from the failures are here: https://github.com/open-telemetry/opentelemetry-java-instrumentation/actions/runs/4056334446/jobs/6980700983

[big-guy]

@yogurtearl hmm, these errors look a little different. I see StackOverflowErrors happening in CodeNarc.

If you disable codenarc on :instrumentation:apache-camel-2.20:javaagent, does the build no longer fail?

We changed the version of Groovy between 7.5 and 7.6, so I wonder if that's impacting CodeNarc somehow.

[yogurtearl]

I was just providing a direct link to the logs to the use case that @trask was pointing out here: #23215 (comment)

But that is not a project I work on. :)

[trask]

thx @big-guy, i'll try disabling codenarc in our PR that updates to gradle 7.6 and report back

[trask]

still getting OOMs, e.g. https://github.com/open-telemetry/opentelemetry-java-instrumentation/actions/runs/4180629061/jobs/7241802271

[ljacomet]

A number of improvements have been released in Gradle 8 and will be part of an upcoming Gradle 7.6.1 release.

Closing as resolved for now.

[trask]

@ljacomet we're still seeing OOM in 7.6.1: https://github.com/open-telemetry/opentelemetry-java-instrumentation/actions/runs/4271958494/jobs/7436787553

would you like us to open a new issue?

[rogerhu]

Based on input from @big-guy, we did have to disable this option in our CI though: (https://cs.android.com/android-studio/platform/tools/base/+/mirror-goog-studio-main:build-system/gradle-core/src/main/java/com/android/build/gradle/internal/utils/GradlePluginUtils.kt;l=110-111

ext["AGP_INTERNAL__MIN_PLUGIN_VERSION_CHECK_STARTED] = true

[ljacomet]

@trask We know that 7.6.1 might require a bit more memory than 7.5. We will address this in upcoming 8.x releases. If you believe that the memory bump you need to do is not reasonable, please open a new issue with as much information as possible and the different things you tried to resolve the issue.

[jjohannes]

We have a ~1200 subprojects build that has 25.567 Configuration objects. Standard Java + some customization on top.

• With Gradle 7.4.2 the Configurations take a total size of 475MB
• With Gradle 8.0.1 the Configurations take a total size of 2.066MB

So in this case the Configurations need more than 4x the memory compared to before.

Maybe there is something specific in our setup which makes this difference so huge.
I would certainly not qualify this as "a bit more". 🙁

Can you maybe share which feature improvements we have in 7.6 and 8.0 so that we know for what we pay the memory costs?

[trask]

We know that 7.6.1 might require a bit more memory than 7.5. We will address this in upcoming 8.x releases.

thx! we were able to get up and running with 7.6.1 by doubling our heap (1.5g to 3g). we are working on getting to Gradle 8 currently, and great to hear you are still making improvements to this in 8.x

[SheliakLyr]

I have a very similar experience to @jjohannes.

My largest gradle build consists of 400 modules. With gradle 7.5.x I am able to build it with heap = 3700MB. After upgrade to 7.6.1 or 8.0.2 it requires significantly more heap (>5GB). Heapdump analysis points to configurations objects.

Are further improvements to memory usage planned in 8.x line beyond those introduced in 8.0.2? Should I create a new bug report with profiler screenshots? I guess that's the only data (apart from Xmx values) that I can provide.

[mklueh]

I'm running a monorepo project with Quarkus, Gradle and Vue + Postgres in Docker on Windows 11 + WSL2.

Allocated 6GB to WSL and now 4GB to Gradle for a demo project that just exposes a simple REST API, composed by some submodules. Nothing special or complex code-wise, except for the module structure maybe.

With 3GB it won't launch at all, with 4GB the daemon is often killed when I run the frontend / database in parallel and something takes memory away.

Imo, this kind of application should not take up more than 250-500mb for what it does, but it sure has a few dependencies included and consists of 7 Gradle submodules that in some cases depend on other submodules.

However, on my 16GB Laptop using IntelliJ as an IDE which itself loves a little bit more memory development currently isn't fun anymore.

I've launched the application with 3GB and let it dump the heap. While the application code itself takes up ~2Kb memory, 4.09GB is allocated by Gradle itself, especially by one class :

org.gradle.internal.component.external.model.ConfigurationBoundExternalDependencyMetadata

Looking into the "Biggest Objects" and the artifacts of each, I find my submodules in several of these Objects, for example the "payment" module and it seems each time new memory is allocated for those exact same modules and their dependencies. Not sure if this is right or wrong, or if those objects are the same but just references in several lists:

Is the submodule structure responsible for a large memory overhead?

Is there any rule of thumb how to get the memory down and is there a difference between using "implementation" vs "api" memory-wise?

Edit: I'm now running my project with just 1GB memory instead of 4GB and pretty fast build time with disabled daemon

org.gradle.daemon=false
org.gradle.priority=low
org.gradle.caching=true
org.gradle.configureondemand=true
org.gradle.jvmargs=-Xmx1g -XX:MaxMetaspaceSize=1g -XX:+HeapDumpOnOutOfMemoryError -Dfile.encoding=UTF-8

will try that for a bit, but it looks like the best solution for my case so far and it looks like it is even faster than with daemon enabled

[filipatbnp]

will it be fixed in some release of gradle 7.6.X?

[jjohannes]

RE #23215 (comment)

With the current 8.3 nightly, the situation has improved for us (large standard Java project). Thanks for working on this.

[trask]

just wanted to report that 8.3 resolved the memory issues we had been seeing since 7.6, thanks!! (open-telemetry/opentelemetry-java-instrumentation#9241)