[GR-39609] Deadlock breaks the native image building process #8685
Comments
|
This issue seems to be related to this one #4822, so I will ask the same question posted there:
Is this your case? |
It is a single-module maven project, so the structure is very simple. And I use the following command for building: Project structure: |
|
is there a way you can create a minimal version of the project that can produce the same error? |
|
I just found that after excluding an internal runtime dependency which depends on javaassist and objectasm (not sure if they are related), building process can complete successfully. However, since it's an internal library, I'm unable to share it here. But is it anyway to debug the building process, or see where does it get stuck? And I'm using Oracle GraalVM now, because we need to use G1 GC. Should I try GraalCE to see if the problem is reproducible? |
|
You can use the debugging tools explained here: https://www.graalvm.org/latest/reference-manual/native-image/debugging-and-diagnostics/DebugInfo/ |
Thanks for your help, Fernando. I tried with GraalCE, and it worked normally. However, the link you shared above seems to be related to "adding debug information to a generated native image". I'm not sure if it is helpful to this issue. And I will try to find if there is any verbose or debug flag in the native-image executable. |
|
Just excluding I also tried to add In addition, it seems that only one thread in the ForkJoinPool is running during the building process. Does the following stacktrace ring a bell? "ForkJoinPool-3-worker-8" Id=116 in RUNNABLE
at app/org.graalvm.collections/org.graalvm.collections.EconomicMapImpl.findHash(EconomicMapImpl.java:292)
at app/org.graalvm.collections/org.graalvm.collections.EconomicMapImpl.find(EconomicMapImpl.java:253)
at app/org.graalvm.collections/org.graalvm.collections.EconomicMapImpl.get(EconomicMapImpl.java:244)
at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.InterproceduralPartialEscapeAnalysisPhase$a.a(stripped:241)
at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:140)
at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:147)
at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:147)
at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:147)
at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:147)
at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:147)
at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:147)
at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:147)
at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:147)
at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:147)
at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:85)
at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d$$Lambda/0x00000007c29ad670.accept(Unknown Source)
at app/com.oracle.graal.graal_enterprise/com.oracle.graal.phases.preciseinline.priorityinline.nodes.a.c(stripped:452)
at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.d.a(stripped:76)
at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.SubstratePolicyFactory$a.a(stripped:150)
at app/com.oracle.graal.graal_enterprise/com.oracle.graal.phases.preciseinline.priorityinline.d.a(stripped:71)
at app/com.oracle.graal.graal_enterprise/com.oracle.graal.phases.preciseinline.priorityinline.PriorityInliningPhase$a.d(stripped:582)
at app/com.oracle.graal.graal_enterprise/com.oracle.graal.phases.preciseinline.priorityinline.PriorityInliningPhase$a.c(stripped:537)
at app/com.oracle.graal.graal_enterprise/com.oracle.graal.phases.preciseinline.priorityinline.PriorityInliningPhase.d(stripped:474)
at app/com.oracle.graal.graal_enterprise/com.oracle.graal.phases.preciseinline.priorityinline.PriorityInliningPhase.a(stripped:469)
at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.SubstratePriorityInliningPhase.a(stripped:191)
at app/com.oracle.svm.svm_enterprise/com.oracle.svm.enterprise.hosted.phases.priorityinline.SubstratePriorityInliningPhase.run(stripped:76)
at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:434)
at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:322)
at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.PhaseSuite.run(PhaseSuite.java:390)
at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:434)
at platform/jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:322)
at platform/jdk.internal.vm.compiler/org.graalvm.compiler.core.GraalCompiler.emitFrontEnd(GraalCompiler.java:294)
at platform/jdk.internal.vm.compiler/org.graalvm.compiler.core.GraalCompiler.compile(GraalCompiler.java:169)
at platform/jdk.internal.vm.compiler/org.graalvm.compiler.core.GraalCompiler.compileGraph(GraalCompiler.java:137)
at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.code.CompileQueue.defaultCompileFunction(CompileQueue.java:1310)
at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.code.CompileQueue$$Lambda/0x00000007c2aae408.compile(Unknown Source)
at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.code.CompileQueue.doCompile(CompileQueue.java:1258)
at app/org.graalvm.nativeimage.builder/com.oracle.svm.hosted.code.CompileQueue$CompileTask.run(CompileQueue.java:306)
at app/org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.executeCommand(CompletionExecutor.java:187)
at app/org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.lambda$executeService$0(CompletionExecutor.java:171)
at app/org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor$$Lambda/0x00000007c238d648.run(Unknown Source)
at java.base@21.0.2/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1423)
at java.base@21.0.2/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
at java.base@21.0.2/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312)
at java.base@21.0.2/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843)
at java.base@21.0.2/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808)
at java.base@21.0.2/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) |
|
What happens if you use |
|
Luckily (or saddly), after I cleaned my local Maven cache, the whole build process worked as expected. I will keep tracking this problem. If it doesn't re-occur this week. I will close this issue. Thanks for your help. |
The previous problem reoccurred today, and adding |
Yes, it can affect the performance. I only suggested it as a workaround and to see whether it really is related to the priority inliner. Another test that shouldn't affect the performance as much is to use Nonetheless, it would be great if you could provide us with a reproducer so we can take a closer look at what's going wrong. Thanks! |
Thanks. I will try |
|
|
|
Here is a minimal reproducer, deadlock.zip. Please try the following steps to reproduce this issue: cd deadlock-agent
./mvnw clean install -Dmaven.test.skip
cd ../deadlock-reproducer
./mvnw clean package native:compile-no-fork -P native -Dmaven.test.skip |
fniephaus
changed the title
|
BTW, removing |
Yes, that is correct.
That's weird because it's defaulting to Shouldn't this be: <buildArgs>
<buildArg>-march=x86-64-v2</buildArg>
<buildArg>--add-opens java.management/sun.management=ALL-UNNAMED</buildArg>
<buildArg>--add-opens java.base/java.lang=ALL-UNNAMED</buildArg>
<buildArg>--add-opens java.base/java.lang.reflect=ALL-UNNAMED</buildArg>
<buildArg>--add-opens java.base/sun.reflect.annotation=ALL-UNNAMED</buildArg>
<buildArg>--add-opens java.base/java.math=ALL-UNNAMED</buildArg>
<buildArg>--add-opens java.base/java.util=ALL-UNNAMED</buildArg>
<buildArg>--add-opens java.base/sun.util.calendar=ALL-UNNAMED</buildArg>
<buildArg>--add-opens java.base/sun.util.calendar=ALL-UNNAMED</buildArg>
<buildArg>--add-opens java.base/java.io=ALL-UNNAMED</buildArg>
<buildArg>--add-opens java.base/java.net=ALL-UNNAMED</buildArg>
<buildArg>--add-opens java.xml/com.sun.org.apache.xerces.internal.jaxp.datatype=ALL-UNNAMED</buildArg>
</buildArgs> |
|
Oh, finally got the correct way of configuring multiple buildArgs.... Because some of our K8s nodes use a relatively old CPU architecture, we have to use v2, otherwise it won't work. And on my laptop with v3 support, using v3 can result a successful build, but using v2 can't. |
Also when you configure multiple buildArgs correctly? |
No, it doesn't work, either.
And I tested all the <!-- Goods -->
<!-- <buildArg>-march=native</buildArg>-->
<!-- <buildArg>-march=haswell</buildArg>-->
<!-- <buildArg>-march=skylake</buildArg>-->
<!-- <buildArg>-march=skylake-avx512</buildArg>-->
<!-- <buildArg>-march=x86-64-v4</buildArg>-->
<!-- <buildArg>-march=x86-64-v3</buildArg>-->
<!-- Bads -->
<!-- <buildArg>-march=x86-64-v2</buildArg>-->
<!-- <buildArg>-march=x86-64-v1</buildArg>-->
<!-- <buildArg>-march=x86-64</buildArg>-->
<!-- <buildArg>-march=compatibility</buildArg>-->
|
|
Hi guys, any new findings on this issue? Thanks. |
|
Update: It seems not to be related to the |
With optimization enabled the native image build process fails because of a deadlock. The issue might be related to oracle/graal#8685
Describe the issue
Following error is shown when building a native image using Maven plugin.
Steps to reproduce the issue
Sorry, the project uses some internal libraries, so I can't share the entire project. Here are the configuration of native-maven-plugin.
Describe GraalVM and your environment:
❯ java -version java version "21.0.2" 2024-01-16 LTS Java(TM) SE Runtime Environment Oracle GraalVM 21.0.2+13.1 (build 21.0.2+13-LTS-jvmci-23.1-b30) Java HotSpot(TM) 64-Bit Server VM Oracle GraalVM 21.0.2+13.1 (build 21.0.2+13-LTS-jvmci-23.1-b30, mixed mode, sharing)More details
The text was updated successfully, but these errors were encountered: