Shrinking down executable image

[aalmiray]

I'm currently looking at options to generate a native image executable for the jreleaser/jreleaser CLI. Current settings create an image close to 100M in size and I wonder what could be done to shrink it down.

All classes found int he org.jreleaser.* packages should be included in the image. There are a few places where proxy and reflection config must be declared. Most of the codebase is reflection free but external configuration is read through Jackson (hence reflection), some network services require reflection/proxies as well (Feign, Hub4j, Twitter4j).

I used the agent to gather information and ended up manually updating the generated files into https://github.com/jreleaser/jreleaser/tree/graal/apps/jreleaser/src/main/resources/META-INF/native-image/org.jreleaser/jreleaser
This manual update is likely responsible for the growth in size.

Additionally every project generates reflect and proxy config files using custom APT. See attached ZIP with all generated files
native-image.zip

The executable is created passing these additional command flags:

-H:Name=jreleaser \
-Duser.language=en -H:IncludeLocales=en,ca,de,es,fr,hi,it,ja,nl,pt_BR,zh_TW,ru \
-H:Optimize=2 -H:+RemoveUnusedSymbols

[neomatrix369]

Hey @aalmiray I have a couple of resources for you to look at:

• Compressed GraalVM Native Images: the best startup for Java apps comes in tiny packages
• UPX to compress GraalVM Images - in the thread you can see I have shared a working example of how to compress a native-image using UPX

You can also find other native-image and other GraalVM related posts here https://github.com/neomatrix369/awesome-graal/blob/master/blog-posts.md

I hope this helps - maybe you wish to accomplish this in another way, but UPX seems to be the most portable and one of the widely tested ways to get around this. It is of course after the fact the native-image builder creates the native-image, and does not solve the issue at build/image creation level.

[fniephaus]

Ha, you beat me to it @neomatrix369! However, I think it's a bit too early to apply UPX. Instead, I'd suggest to look at the cause, which seems to be the newly introduced reflection configuration. Including all fields, methods, and constructors may not be necessary to create a working binary.

[neomatrix369]

Yes that's true, but UPX can be applied only after the fact -- i.e. after the image has been created.

Good suggestion about the steps you suggested.

[neomatrix369]

Also, I think somehow the UPX step can be made part of the native-image build process - via an additional switch for those who wish to go that path, saves them all the extra steps they would need to do - my example programs are available for everyone to reuse.

[aalmiray]

Hi @neomatrix36. Thanks. I'm aware of UPX and @fniephaus mentioned it's a cure for the symptoms but not for the actual cause, which is why this discussion thread started.

[neomatrix369]

Ok thats fine you are looking to solve it at the internal levels.

Even after this issue is fixed I feel, the image can be further reduced in size using UPX, which either can be done externally or be built into the native-image builder.

[fniephaus]

Thanks for opening the discussion, @aalmiray. What's the easiest way to reproduce such a build? I'd like to run it through the Dashboard.

[aalmiray]

It does not. It's an annotation processor triggered on every build. The only manually created files are located inside the apps/jreleaser/src/main/resources/META-INF directory. Every other reflect/proxy config files found in the final artifacts are created using the annotation processor. You can see them on every module at build/classes/java/main/META-INF.

[fniephaus]

Ok, so in general, I believe it's not a good idea to force-include all public methods, fields, and constructors. Instead, the reachability analysis should be able to find out what's needed and what's not. If it doesn't, then there's something wrong with either the setup or the analysis itself.
Looking at the number of dependencies, I'm not surprised that the binary ends up being quite large. Nonetheless, it may be possible to generate much smaller binaries that are functional.

So again, how did you use the agent to create the reflect-config,json files in your project? Did you run it as part of a releasing process with your demo app? Also, are there any development dependencies that may accidentally be included in the binary?

[aalmiray]

There are no development dependencies included in the resulting binary nor Java distro.

I ran the agent configuring JAVA_OPTS per the guide instructions when running the binary as a Java distribution.

I'm aware that the agent defines a smaller set of reflection elements but I wasn't sure all codepaths were exercised during live tests hence why a wider set of methods/constructors were manually defined in the resulting reflect config file found inside apps/jreleaser.

OTOH the automatically generated reflect config files must remain as they are.

[fniephaus]

So with your current setup, you can create a functional +100MB jreleaser binary that you could potentially ship at some point, right? And you can shrink it down to something like 30MB using UPX, correct? If so, I will do some more experiments when I have some time or are you currently blocked because of this?

In summary, I think marking too many things as reachable by hand can certainly blow up image sizes. However, it seems this is the only way you found to generate a working binary, so maybe jreleaser is a good example for where NI tooling support could be improved.

BTW: Are you aware of the Gradle plugin for GraalVM Native Image building? If you could give that a try and report your experience that'd be great. Maybe it even improves image sizes! :)

[aalmiray]

Yes, it seems that the current setup may work. Now that I've figure out how to run UPX as part of the build pipeline for all hosted runners on GitHub I can build with more confidence.

The Gradle native plugin could be used instead of delegating to JReleaser's assemble which wraps the native-image executable, but only if it offers more than just using the native-image executable, does it?

The current explicit reflect config is quite wide in its reach as an overabundance of caution that some codepaths were not exercised with the agent. I'd rather have a big binary than a broken binary. Time will tell how that size can be trimmed down with more accurate configuration.

Thanks.