Skip to content
This repository has been archived by the owner on Apr 2, 2023. It is now read-only.

Implement a working sample for Quarkus #30

Closed
dzou opened this issue Oct 14, 2020 · 3 comments · Fixed by #33
Closed

Implement a working sample for Quarkus #30

dzou opened this issue Oct 14, 2020 · 3 comments · Fixed by #33
Labels
enhancement New feature or request p3

Comments

@dzou
Copy link
Contributor

dzou commented Oct 14, 2020 •
edited

Quarkus does not work out of the box with this library.

I think the root issue is that Quarkus brings its own version of netty and the Google client libraries bring in their version of grpc-netty-shaded. Ordinarily there would be no conflicts, but both libraries also register native libraries, and these I think conflict with each other somehow.

If you try building with both deps on the classpath you get the error:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f6201df00b0, pid=11180, tid=11315
#
# JRE version: OpenJDK Runtime Environment GraalVM CE 20.2.0 (11.0.8+10) (build 11.0.8+10-jvmci-20.2-b03)
# Java VM: OpenJDK 64-Bit Server VM GraalVM CE 20.2.0 (11.0.8+10-jvmci-20.2-b03, mixed mode, tiered, jvmci, jvmci compiler, compressed oops, parallel gc, linux-amd64)
# Problematic frame:
# C  0x00007f6201df00b0
#
# No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /home/dzou/IdeaProjects/quarkus-sandbox/target/getting-started-1.0-SNAPSHOT-native-image-source-jar/hs_err_pid11180.log
#
# If you would like to submit a bug report, please visit:
#   https://github.com/oracle/graal/issues
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.

If you go down the path of excluding grpc-netty-shaded and relying on Quarkus's version of netty, this gets you past this build error but fails at runtime since our configurations target grpc-netty-shaded.

I think it is worth first investigating if grpc-netty-shaded can be made compatible with quarkus. Apparently this should have been solved already: netty/netty#7272

Otherwise, probably will move forward with using non-shaded grpc-netty which requires users manually adding an dep exclusion:

    <dependency>
      <groupId>com.google.cloud</groupId>
      <artifactId>google-cloud-pubsub</artifactId>

      <exclusions>
        <exclusion>
          <groupId>io.grpc</groupId>
          <artifactId>grpc-netty-shaded</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
@dzou
Copy link
Contributor Author

dzou commented Oct 15, 2020

Running the native image build through the debugger and here is the stack trace:

C  0x00007f005c1c30b0
j  io.grpc.netty.shaded.io.netty.internal.tcnative.Library.initialize(Ljava/lang/String;Ljava/lang/String;)Z+31
j  io.grpc.netty.shaded.io.netty.handler.ssl.OpenSsl.initializeTcNative(Ljava/lang/String;)Z+4
j  io.grpc.netty.shaded.io.netty.handler.ssl.OpenSsl.<clinit>()V+240
v  ~StubRoutines::call_stub
V  [libjvm.so+0x880096]  JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, Thread*)+0x366

Somehow it fails on the aprMajorVersion() call in Library.java from netty TCNative. Note that this call succeeds during the normal build and only fails during the native image build.

Not sure why it fails. It's triggered by initialization of OpenSsl.

io.grpc.netty.shaded.io.netty.handler.ssl.OpenSsl.<clinit>()V+240

my guess is that the OpenSsl class must be initialized at runtime; I think not all classes using native libraries can be loaded at build time.

Tried and now getting different errors regarding class initialization.

Started a work-in-progress repo here: https://github.com/dzou/quarkus-gcp-experiments/blob/main/pom.xml

@ttomsu ttomsu added enhancement New feature or request p3 and removed enhancement New feature or request labels Oct 19, 2020
@dzou
Copy link
Contributor Author

dzou commented Oct 19, 2020

Made some new progress in the sample. I got past the native library errors, however now am getting an error about grpc service providers:

ConfigurationError: io.grpc.NameResolverProvider: Provider io.grpc.internal.DnsNameResolverProvider not found
	at org.jboss.resteasy.core.ExceptionHandler.handleApplicationException(ExceptionHandler.java:106)
	at org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:372)
	at org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:218)
	at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:519)
	at org.jboss.resteasy.core.SynchronousDispatcher.lambda$invoke$4(SynchronousDispatcher.java:261)
	at org.jboss.resteasy.core.SynchronousDispatcher.lambda$preprocess$0(SynchronousDispatcher.java:161)
	at org.jboss.resteasy.core.interception.jaxrs.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:364)
	at org.jboss.resteasy.core.SynchronousDispatcher.preprocess(SynchronousDispatcher.java:164)
	at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:247)
	at io.quarkus.resteasy.runtime.standalone.RequestDispatcher.service(RequestDispatcher.java:73)
	at io.quarkus.resteasy.runtime.standalone.VertxRequestHandler.dispatch(VertxRequestHandler.java:131)
	at io.quarkus.resteasy.runtime.standalone.VertxRequestHandler.access$000(VertxRequestHandler.java:37)
	at io.quarkus.resteasy.runtime.standalone.VertxRequestHandler$1.run(VertxRequestHandler.java:94)
	at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
	at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:2046)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1578)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1452)
	at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29)
	at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29)
	at java.lang.Thread.run(Thread.java:834)
	at org.jboss.threads.JBossThread.run(JBossThread.java:479)
	at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:517)
	at com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine(PosixJavaThreads.java:192)
Caused by: java.util.ServiceConfigurationError: io.grpc.NameResolverProvider: Provider io.grpc.internal.DnsNameResolverProvider not found
	at java.util.ServiceLoader.fail(ServiceLoader.java:588)
	at java.util.ServiceLoader$LazyClassPathLookupIterator.nextProviderClass(ServiceLoader.java:1211)
	at java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1220)
	at java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1264)
	at java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1299)
	at java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1384)
	at io.grpc.ServiceProviders.getCandidatesViaServiceLoader(ServiceProviders.java:111)
	at io.grpc.ServiceProviders.loadAll(ServiceProviders.java:64)
	at io.grpc.NameResolverRegistry.getDefaultRegistry(NameResolverRegistry.java:101)
	at io.grpc.internal.AbstractManagedChannelImplBuilder.<init>(AbstractManagedChannelImplBuilder.java:107)
	at io.grpc.netty.shaded.io.grpc.netty.NettyChannelBuilder.<init>(NettyChannelBuilder.java:147)
	at io.grpc.netty.shaded.io.grpc.netty.NettyChannelBuilder.<init>(NettyChannelBuilder.java:142)
	at io.grpc.netty.shaded.io.grpc.netty.NettyChannelBuilder.forAddress(NettyChannelBuilder.java:128)
	at io.grpc.netty.shaded.io.grpc.netty.NettyChannelProvider.builderForAddress(NettyChannelProvider.java:37)
	at io.grpc.netty.shaded.io.grpc.netty.NettyChannelProvider.builderForAddress(NettyChannelProvider.java:23)
	at io.grpc.ManagedChannelBuilder.forAddress(ManagedChannelBuilder.java:39)
	at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createSingleChannel(InstantiatingGrpcChannelProvider.java:280)
	at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.access$1600(InstantiatingGrpcChannelProvider.java:71)
	at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider$1.createSingleChannel(InstantiatingGrpcChannelProvider.java:210)
	at com.google.api.gax.grpc.ChannelPool.create(ChannelPool.java:72)
	at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createChannel(InstantiatingGrpcChannelProvider.java:217)
	at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.getTransportChannel(InstantiatingGrpcChannelProvider.java:200)
	at com.google.api.gax.rpc.ClientContext.create(ClientContext.java:166)
	at com.google.cloud.pubsub.v1.stub.GrpcPublisherStub.create(GrpcPublisherStub.java:195)
	at com.google.cloud.pubsub.v1.stub.PublisherStubSettings.createStub(PublisherStubSettings.java:219)
	at com.google.cloud.pubsub.v1.TopicAdminClient.<init>(TopicAdminClient.java:165)
	at com.google.cloud.pubsub.v1.TopicAdminClient.create(TopicAdminClient.java:147)
	at com.google.cloud.pubsub.v1.TopicAdminClient.create(TopicAdminClient.java:139)
	at org.acme.getting.started.GreetingResource.createTopic(GreetingResource.java:69)
	at org.acme.getting.started.GreetingResource.hello(GreetingResource.java:57)
	at java.lang.reflect.Method.invoke(Method.java:566)
	at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:170)
	at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:130)
	at org.jboss.resteasy.core.ResourceMethodInvoker.internalInvokeOnTarget(ResourceMethodInvoker.java:643)
	at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTargetAfterFilter(ResourceMethodInvoker.java:507)
	at org.jboss.resteasy.core.ResourceMethodInvoker.lambda$invokeOnTarget$2(ResourceMethodInvoker.java:457)
	at org.jboss.resteasy.core.interception.jaxrs.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:364)
	at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:459)
	at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:419)
	at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:393)
	at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:68)
	at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:492)
	... 19 more

@dzou
Copy link
Contributor Author

dzou commented Oct 20, 2020 •
edited

I got this working finally; latest changes pushed to: https://github.com/dzou/quarkus-gcp-experiments/blob/main/pom.xml

To get this to work:

  • Did more work to figure out what classes can be built run-time vs. build-time and these configs are updated. The time when classes get initialized (build-time/run-time) can cause the severe native library linkage errors shown above.

  • I had to copy over Netty's configuration and update the package names to be prefixed with io.grpc.netty.shaded. Filed an issue in grpc-java requesting additional shading of native-image config resources: Apply package name shading to native-image config files in grpc-netty-shaded grpc/grpc-java#7540 . This is necessary to get Quarkus working. (In the meanwhile, I've duplicated the configuration locally to test).

  • Also relying on Quarkus's grpc extension which provides many reflection configs so i did not have to reinvent the wheel much.

This was referenced Oct 20, 2020
Merged
Merged
@dzou dzou closed this as completed in #33 Oct 29, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
enhancement New feature or request p3
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add Quarkus Pub/Sub Sample GoogleCloudPlatform/native-image-support-java
2 participants
@dzou @ttomsu