Many Micronaut builds failing with NPE with Gradle 7.1 & JDK 8

[graemerocher]

Many of our builds are failing with Gradle 7.1 with the following stack trace:

Caused by: java.lang.NullPointerException
        at org.gradle.internal.compiler.java.listeners.constants.ConstantsTreeVisitor.visitMemberSelect(ConstantsTreeVisitor.java:110)
        at org.gradle.internal.compiler.java.listeners.constants.ConstantsTreeVisitor.visitMemberSelect(ConstantsTreeVisitor.java:39)
        at com.sun.tools.javac.tree.JCTree$JCFieldAccess.accept(JCTree.java:1903)
        at com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68)
        at com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:81)
        at com.sun.source.util.TreeScanner.visitAssignment(TreeScanner.java:307)
        at org.gradle.internal.compiler.java.listeners.constants.ConstantsTreeVisitor.visitAssignment(ConstantsTreeVisitor.java:58)
        at org.gradle.internal.compiler.java.listeners.constants.ConstantsTreeVisitor.visitAssignment(ConstantsTreeVisitor.java:39)

Expected Behavior

Should compile

Current Behavior

Fails with NPE

Context

Blocks upgrading to 7.1

Steps to Reproduce

1. git clone git@github.com:micronaut-projects/micronaut-kafka.git
2. git checkout sync-files-master
3. ./gradlew cJ --stacktrace

Your Environment

------------------------------------------------------------
Gradle 7.1
------------------------------------------------------------

Build time:   2021-06-14 14:47:26 UTC
Revision:     989ccc9952b140ee6ab88870e8a12f1b2998369e

Kotlin:       1.4.31
Groovy:       3.0.7
Ant:          Apache Ant(TM) version 1.10.9 compiled on September 27 2020
JVM:          1.8.0_282 (GraalVM Community 25.282-b07-jvmci-21.0-b06)
OS:           Mac OS X 10.16 x86_64

• See https://github.com/micronaut-projects/micronaut-kafka/runs/2863213070?check_suite_focus=true#step:8:54 and https://github.com/micronaut-projects/micronaut-kafka/blob/f73b9c48e465cebe5cad0dd9769faacf89da9018/.github/workflows/gradle.yml#L38-L42

[asodja]

Since I wrote this part of code I took a quick look. I checked it with JDK8, JDK 11 and JDK 16. It works on JDK 11 and 16, but not on JDK 8.

It looks like the issue is that we start with null context in line:

gradle/subprojects/java-compiler-plugin/src/main/java/org/gradle/internal/compiler/java/listeners/constants/ConstantsCollector.java

Line 57 in 12254f2

visitor.scan(path, null);

Relevant stacktrace:

Caused by: java.lang.NullPointerException
        at org.gradle.internal.compiler.java.listeners.constants.ConstantsTreeVisitor.visitMemberSelect(ConstantsTreeVisitor.java:110)
        at org.gradle.internal.compiler.java.listeners.constants.ConstantsTreeVisitor.visitMemberSelect(ConstantsTreeVisitor.java:39)
        at com.sun.tools.javac.tree.JCTree$JCFieldAccess.accept(JCTree.java:1903)
        at com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68)
        at com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:81)
        at com.sun.source.util.TreeScanner.visitAssignment(TreeScanner.java:307)
        at org.gradle.internal.compiler.java.listeners.constants.ConstantsTreeVisitor.visitAssignment(ConstantsTreeVisitor.java:58)
        at org.gradle.internal.compiler.java.listeners.constants.ConstantsTreeVisitor.visitAssignment(ConstantsTreeVisitor.java:39)
        at com.sun.tools.javac.tree.JCTree$JCAssign.accept(JCTree.java:1693)
        at com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68)
        at com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:81)
        at com.sun.source.util.TreeScanner.scan(TreeScanner.java:91)
        at com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:99)
        at com.sun.source.util.TreeScanner.visitAnnotation(TreeScanner.java:401)
        at com.sun.tools.javac.tree.JCTree$JCAnnotation.accept(JCTree.java:2327)
        at com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68)
        at com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:81)
        at com.sun.source.util.TreeScanner.scan(TreeScanner.java:91)
        at com.sun.source.util.TreeScanner.visitCompilationUnit(TreeScanner.java:117)
        at org.gradle.internal.compiler.java.listeners.constants.ConstantsTreeVisitor.visitCompilationUnit(ConstantsTreeVisitor.java:53)
        at org.gradle.internal.compiler.java.listeners.constants.ConstantsTreeVisitor.visitCompilationUnit(ConstantsTreeVisitor.java:39)
        at com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:550)
        at com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:50)
        at org.gradle.internal.compiler.java.listeners.constants.ConstantsCollector.finished(ConstantsCollector.java:57)
        at com.sun.tools.javac.api.ClientCodeWrapper$WrappedTaskListener.finished(ClientCodeWrapper.java:681)
        at com.sun.tools.javac.api.MultiTaskListener.finished(MultiTaskListener.java:111)
        at com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1342)
        at com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1296)
        at com.sun.tools.javac.main.JavaCompiler.compile2(JavaCompiler.java:901)
        at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:860)
        at com.sun.tools.javac.main.Main.compile(Main.java:523)

At the start we don't know what class is currently visited until class node or package node is visited. And we need to know class name, so we can mark it as dependant to some constant.

But it looks like that on JDK 8 in that case annotation node is visited before class node or package node. In that case we would most likely need to find out visited class from CompilationUnit node or some other node.

[asodja]

After more digging, I think I found the problem. Problematic are package-info.java classes with annotations that has some constant in it. In case of Micronaut-kafka that is: kafka/src/main/java/io/micronaut/configuration/kafka/package-info.java . If you remove this file or just remove constants from annotations it compiles with JDK8.

The problem is that visitPackage method in TreePathScanner is supported after JDK9. So fix for JDK8 would be something like:

--- a/subprojects/java-compiler-plugin/src/main/java/org/gradle/internal/compiler/java/listeners/constants/ConstantsTreeVisitor.java
+++ b/subprojects/java-compiler-plugin/src/main/java/org/gradle/internal/compiler/java/listeners/constants/ConstantsTreeVisitor.java
@@ -49,8 +49,15 @@ public ConstantsTreeVisitor(Elements elements, Trees trees, ConstantDependentsCo
     }
 
     @Override
-    public ConstantsVisitorContext visitCompilationUnit(CompilationUnitTree node, ConstantsVisitorContext constantConsumer) {
-        return super.visitCompilationUnit(node, constantConsumer);
+    public ConstantsVisitorContext visitCompilationUnit(CompilationUnitTree node, ConstantsVisitorContext context) {
+        if (context == null) {
+            // Basically same code as for visitPackage method  
+            Element element = trees.getElement(getCurrentPath());
+            String visitedClass = ((PackageElement) element).getQualifiedName().toString();
+            consumer.consumePrivateDependent(visitedClass, visitedClass);
+            return super.visitCompilationUnit(node, new ConstantsVisitorContext(visitedClass, consumer::consumePrivateDependent));
+        } else {
+            return super.visitCompilationUnit(node, context);
+        }
     }

I did not manage to test this with gradle, since I could not compile it. I had some issues with 'com.gradle.internal.test-selection', version: '0.6.0-rc-1', also it seems repositories are not set correctly, so some dependencies could not be resolved for me.

[wolfs]

@asodja Thank you for looking into this! Could you provide a PR with your fix and a test?

I did not manage to test this with gradle, since I could not compile it. I had some issues with 'com.gradle.internal.test-selection', version: '0.6.0-rc-1', also it seems repositories are not set correctly, so some dependencies could not be resolved for me.

We had problems with repo.gradle.org, causing some artifacts not being available. It seems like it should work again now.

[asodja]

Of course, I can check it later today.

[wolfs]

This has been fixed by @asodja in #17518. I created a 7.1.1 nightly containing the fix: 7.1.1-20210624152759+0000

@graemerocher Can you try out this version to verify that the problem has been fixed?

[wolfs]

@graemerocher Can you confirm this is fixed now? Please try out 7.1.1-20210624152759+0000.

[wolfs]

I verified the fix on micronaut-kafka.