Java: Better support for GraalVM native images

[jroper]

Currently, Java generated messages use reflection to implement equals/hashCode/toString. This places a large burden on anyone using protobuf with GraalVM native images, since any class that gets reflected over needs to have an explicit reflection configuration created for it so that GraalVM native images can know to provide that information during reflection. Replacing the reflection with generated code would be of a big help to GraalVM native image users.

[rafi-kamal]

Java generated messages use reflection to implement equals/hashCode/toString

By default generated classes have generated code for equals and hashCode. Are you using java-lite?

[ST-DDT]

If I try to compile a dummy application to a native image using graalvm I get the following error during compilation. (protobuf 3.10.0)

The error is caused by the usage of MethodHandles, which are unsupported by graalvm native images.

[test.test:13252]     analysis:   9,826.99 ms
Error: com.oracle.svm.hosted.substitute.DeletedElementException: Unsupported type java.lang.invoke.MemberName is reachable: All methods from java.lang.invoke should have been replaced during image building.
To diagnose the issue, you can add the option --report-unsupported-elements-at-runtime. The unsupported element is then reported at run time when it is accessed the first time.
Detailed message:
Trace:
        at parsing java.lang.invoke.MethodHandles$Lookup.unreflect(MethodHandles.java:1186)
Call path from entry point to java.lang.invoke.MethodHandles$Lookup.unreflect(Method):
        at java.lang.invoke.MethodHandles$Lookup.unreflect(MethodHandles.java:1182)
        at com.google.protobuf.GeneratedMessageV3$FieldAccessorTable$SingularFieldAccessor$MethodHandleInvoker.<init>(GeneratedMessageV3.java:2259)
        at com.google.protobuf.GeneratedMessageV3$FieldAccessorTable$SingularFieldAccessor.tryGetMethodHandleInvoke(GeneratedMessageV3.java:2367)
        at com.google.protobuf.GeneratedMessageV3$FieldAccessorTable$SingularFieldAccessor.<init>(GeneratedMessageV3.java:2359)
        at com.google.protobuf.GeneratedMessageV3$FieldAccessorTable.ensureFieldAccessorsInitialized(GeneratedMessageV3.java:2008)
        at com.google.protobuf.Empty.internalGetFieldAccessorTable(Empty.java:89)
        at com.google.protobuf.GeneratedMessageV3.getDescriptorForType(GeneratedMessageV3.java:141)
        at com.google.protobuf.AbstractMessage.equals(AbstractMessage.java:151)
        at java.util.HashMap.getNode(HashMap.java:579)
        at java.util.HashMap.get(HashMap.java:557)
        at com.oracle.svm.jni.access.JNIReflectionDictionary.getFieldNameByID(JNIReflectionDictionary.java:260)
        at com.oracle.svm.jni.functions.JNIFunctions.ToReflectedField(JNIFunctions.java:848)
        at com.oracle.svm.core.code.IsolateEnterStub.JNIFunctions_ToReflectedField_80d8233579d5215df0227b770e5c01228a0de9b9(generated:0)

If I try to use the --report-unsupported-elements-at-runtime as suggested, then it shows 4-5 errors.

If I try to set GeneratedMessageV3#forTestUseReflection to true and invoke it again, the same error is shown.

I have tried to substitute the call to tryGetMethodHandle with a NOP call, but I failed to do so (due to private classes), because I'm not experienced enough in graalvm yet.

It works with protobuf-java 3.9.2 (Haven't tested all aspects yet).

[rafi-kamal]

@ST-DDT the issue seems to be related to #6718, in that case it's already fixed in master and will be released in 3.11.

[huntc]

Having just used pb in the context of GraalVM native images for the first time, I ended up configuring native-image for reflection code around this project. I'm surprised that pb for Java uses reflection. I'm sure there are good reasons from a historical perspective, but a reflection-free pb sounds like something to strive for.

[perezd]

@huntc I recommend the "lite" runtime if you want to be reflection free:
https://github.com/protocolbuffers/protobuf/blob/master/java/lite.md

[huntc]

@huntc I recommend the "lite" runtime if you want to be reflection free:
https://github.com/protocolbuffers/protobuf/blob/master/java/lite.md

Good to know! Thanks! However, perhaps avoiding reflection is still worth striving for with the main code base given the caveats associated with Lite?

[perezd]

There are still good reasons to use reflection in the main implementation (based on APIs we've published that effectively require it), so we'll likely leave it as is for now, but this is why we have the lite flavor, to allow folks to decide what is best for them.

[huntc]

Thanks for the further insight. Should this issue therefore be closed given your answer of using lite for when dealing with GraalVM?

Btw, and for more info, there are transitive dependency situations that aren’t avoidable. For this, perhaps guidance docs for GraalVM usage might be useful. Thoughts?

[perezd]

If you'd like to contribute said docs, I'd be happy to approve! I'll go ahead and close this as suggested. Thanks!

[huntc]

Thanks. Happy to raise a PR. Can you recommend a link to an appropriate doc page source?

[perezd]

Here would be perfect:
https://github.com/protocolbuffers/protobuf/tree/master/docs

[huntc]

@perezd Here you go: #8294

[pavelorehov]

FYI https://stackoverflow.com/questions/77256791/using-protobuf-java-in-glaalvm-native-image