Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

adding jqwik-kotlin causes existing tests written in java to fail with NPE #557

Closed
twentylemon opened this issue Mar 8, 2024 · 18 comments
Closed

adding jqwik-kotlin causes existing tests written in java to fail with NPE #557

twentylemon opened this issue Mar 8, 2024 · 18 comments
Labels
bug

Comments

@twentylemon
Copy link

Testing Problem

I have a java code base that we recently starting adding some kotlin to. When adding jqwik-kotlin as a dependency, and seemingly doing the rest of the setup, our existing tests in java fail before even starting to run anything.

I've a minimal test written in java,

class SomeTest {
  @Property
  void test(@ForAll int i) {
    System.out.println(i);
  }
}

Snips of pom,

<kotlin.version>1.9.22</kotlin.version>
<jqwik.version>1.8.3</jqwik.version>
...

      <dependency>
        <groupId>net.jqwik</groupId>
        <artifactId>jqwik</artifactId>
        <version>${jqwik.version}</version>
        <scope>test</scope>
      </dependency>
      <dependency>
        <groupId>net.jqwik</groupId>
        <artifactId>jqwik-kotlin</artifactId>
        <version>${jqwik.version}</version>
        <scope>test</scope>
      </dependency>
...
      <plugin>
        <groupId>org.jetbrains.kotlin</groupId>
        <artifactId>kotlin-maven-plugin</artifactId>
        <version>${kotlin.version}</version>
        <extensions>true</extensions>
        <configuration>
          <compilerPlugins>
            <plugin>lombok</plugin>
          </compilerPlugins>
          <pluginOptions>
            <option>lombok:config=${project.basedir}/lombok.config</option>
          </pluginOptions>
          <args>
            <arg>-java-parameters</arg> <!-- Get correct parameter names in jqwik reporting -->
            <arg>-Xjsr305=strict</arg> <!-- Strict interpretation of nullability annotations in jqwik API -->
            <arg>-Xemit-jvm-type-annotations</arg> <!-- Enable annotations on type variables -->
          </args>
        </configuration>
...
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.5.1</version>
        <configuration>
          <compilerArgs>
            <arg>-parameters</arg> <!-- required if you want to report source code names of property method parameters -->
          </compilerArgs>
        </configuration>

Removing jqwik-kotlin and re-running the build, the tests start to pass again.

Stack trace I am getting,

java.lang.NullPointerException: Parameter specified as non-null is null: method kotlin.reflect.jvm.internal.impl.descriptors.runtime.structure.ReflectJavaAnnotation.<init>, parameter annotation

	at kotlin.reflect.jvm.internal.impl.descriptors.runtime.structure.ReflectJavaAnnotation.<init>(ReflectJavaAnnotation.kt)
	at kotlin.reflect.jvm.internal.impl.descriptors.runtime.structure.ReflectJavaAnnotationOwnerKt.getAnnotations(ReflectJavaAnnotationOwner.kt:37)
	at kotlin.reflect.jvm.internal.impl.descriptors.runtime.structure.ReflectJavaTypeParameter.getAnnotations(ReflectJavaTypeParameter.kt:27)
	at kotlin.reflect.jvm.internal.impl.descriptors.runtime.structure.ReflectJavaTypeParameter.getAnnotations(ReflectJavaTypeParameter.kt:23)
	at kotlin.reflect.jvm.internal.impl.load.java.lazy.LazyJavaAnnotations.iterator(LazyJavaAnnotations.kt:40)
	at kotlin.collections.CollectionsKt__MutableCollectionsKt.addAll(MutableCollections.kt:117)
	at kotlin.collections.CollectionsKt___CollectionsKt.plus(_Collections.kt:3250)
	at kotlin.reflect.jvm.internal.impl.load.java.typeEnhancement.AbstractSignatureParts.extractQualifiersFromAnnotations(AbstractSignatureParts.kt:90)
	at kotlin.reflect.jvm.internal.impl.load.java.typeEnhancement.AbstractSignatureParts.computeIndexedQualifiers(AbstractSignatureParts.kt:181)
	at kotlin.reflect.jvm.internal.impl.load.java.typeEnhancement.SignatureEnhancement.enhance(signatureEnhancement.kt:223)
	at kotlin.reflect.jvm.internal.impl.load.java.typeEnhancement.SignatureEnhancement.enhance$default(signatureEnhancement.kt:217)
	at kotlin.reflect.jvm.internal.impl.load.java.typeEnhancement.SignatureEnhancement.enhanceTypeParameterBounds(signatureEnhancement.kt:169)
	at kotlin.reflect.jvm.internal.impl.load.java.lazy.descriptors.LazyJavaTypeParameterDescriptor.processBoundsWithoutCycles(LazyJavaTypeParameterDescriptor.kt:68)
	at kotlin.reflect.jvm.internal.impl.descriptors.impl.AbstractTypeParameterDescriptor$TypeParameterTypeConstructor.processSupertypesWithoutCycles(AbstractTypeParameterDescriptor.java:220)
	at kotlin.reflect.jvm.internal.impl.types.AbstractTypeConstructor$supertypes$3.invoke(AbstractTypeConstructor.kt:107)
	at kotlin.reflect.jvm.internal.impl.types.AbstractTypeConstructor$supertypes$3.invoke(AbstractTypeConstructor.kt:77)
	at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$5.doPostCompute(LockBasedStorageManager.java:235)
	at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedLazyValueWithPostCompute.postCompute(LockBasedStorageManager.java:491)
	at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedLazyValue.invoke(LockBasedStorageManager.java:412)
	at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedLazyValueWithPostCompute.invoke(LockBasedStorageManager.java:481)
	at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedNotNullLazyValueWithPostCompute.invoke(LockBasedStorageManager.java:512)
	at kotlin.reflect.jvm.internal.impl.types.AbstractTypeConstructor.getSupertypes(AbstractTypeConstructor.kt:27)
	at kotlin.reflect.jvm.internal.impl.descriptors.impl.AbstractTypeParameterDescriptor.getUpperBounds(AbstractTypeParameterDescriptor.java:121)
	at kotlin.reflect.jvm.internal.impl.types.checker.ClassicTypeSystemContext$DefaultImpls.getUpperBounds(ClassicTypeSystemContext.kt:282)
	at kotlin.reflect.jvm.internal.impl.types.checker.SimpleClassicTypeSystemContext.getUpperBounds(NewKotlinTypeChecker.kt:29)
	at kotlin.reflect.jvm.internal.impl.load.java.typeEnhancement.AbstractSignatureParts.getBoundsNullability(AbstractSignatureParts.kt:151)
	at kotlin.reflect.jvm.internal.impl.load.java.typeEnhancement.AbstractSignatureParts.extractQualifiersFromAnnotations(AbstractSignatureParts.kt:129)
	at kotlin.reflect.jvm.internal.impl.load.java.typeEnhancement.AbstractSignatureParts.computeIndexedQualifiers(AbstractSignatureParts.kt:181)
	at kotlin.reflect.jvm.internal.impl.load.java.typeEnhancement.SignatureEnhancement.enhance(signatureEnhancement.kt:223)
	at kotlin.reflect.jvm.internal.impl.load.java.typeEnhancement.SignatureEnhancement.enhance(signatureEnhancement.kt:214)
	at kotlin.reflect.jvm.internal.impl.load.java.typeEnhancement.SignatureEnhancement.enhance$default(signatureEnhancement.kt:204)
	at kotlin.reflect.jvm.internal.impl.load.java.typeEnhancement.SignatureEnhancement.enhanceSignature(signatureEnhancement.kt:125)
	at kotlin.reflect.jvm.internal.impl.load.java.typeEnhancement.SignatureEnhancement.enhanceSignatures(signatureEnhancement.kt:56)
	at kotlin.reflect.jvm.internal.impl.load.java.lazy.descriptors.LazyJavaScope$functions$1.invoke(LazyJavaScope.kt:125)
	at kotlin.reflect.jvm.internal.impl.load.java.lazy.descriptors.LazyJavaScope$functions$1.invoke(LazyJavaScope.kt:118)
	at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$MapBasedMemoizedFunction.invoke(LockBasedStorageManager.java:578)
	at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$MapBasedMemoizedFunctionToNotNull.invoke(LockBasedStorageManager.java:681)
	at kotlin.reflect.jvm.internal.impl.load.java.lazy.descriptors.LazyJavaScope.getContributedFunctions(LazyJavaScope.kt:273)
	at kotlin.reflect.jvm.internal.impl.load.java.lazy.descriptors.LazyJavaClassMemberScope.getContributedFunctions(LazyJavaClassMemberScope.kt:865)
	at kotlin.reflect.jvm.internal.impl.load.java.lazy.descriptors.LazyJavaScope.computeDescriptors(LazyJavaScope.kt:378)
	at kotlin.reflect.jvm.internal.impl.load.java.lazy.descriptors.LazyJavaScope$allDescriptors$1.invoke(LazyJavaScope.kt:64)
	at kotlin.reflect.jvm.internal.impl.load.java.lazy.descriptors.LazyJavaScope$allDescriptors$1.invoke(LazyJavaScope.kt:63)
	at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedLazyValue.invoke(LockBasedStorageManager.java:408)
	at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedNotNullLazyValue.invoke(LockBasedStorageManager.java:527)
	at kotlin.reflect.jvm.internal.impl.load.java.lazy.descriptors.LazyJavaScope.getContributedDescriptors(LazyJavaScope.kt:357)
	at kotlin.reflect.jvm.internal.impl.resolve.scopes.ResolutionScope$DefaultImpls.getContributedDescriptors$default(ResolutionScope.kt:50)
	at kotlin.reflect.jvm.internal.KDeclarationContainerImpl.getMembers(KDeclarationContainerImpl.kt:58)
	at kotlin.reflect.jvm.internal.KClassImpl$Data$declaredNonStaticMembers$2.invoke(KClassImpl.kt:173)
	at kotlin.reflect.jvm.internal.KClassImpl$Data$declaredNonStaticMembers$2.invoke(KClassImpl.kt:173)
	at kotlin.reflect.jvm.internal.ReflectProperties$LazySoftVal.invoke(ReflectProperties.java:70)
	at kotlin.reflect.jvm.internal.ReflectProperties$Val.getValue(ReflectProperties.java:32)
	at kotlin.reflect.jvm.internal.KClassImpl$Data.getDeclaredNonStaticMembers(KClassImpl.kt:173)
	at kotlin.reflect.jvm.internal.KClassImpl$Data$allNonStaticMembers$2.invoke(KClassImpl.kt:182)
	at kotlin.reflect.jvm.internal.KClassImpl$Data$allNonStaticMembers$2.invoke(KClassImpl.kt:182)
	at kotlin.reflect.jvm.internal.ReflectProperties$LazySoftVal.invoke(ReflectProperties.java:70)
	at kotlin.reflect.jvm.internal.ReflectProperties$Val.getValue(ReflectProperties.java:32)
	at kotlin.reflect.jvm.internal.KClassImpl$Data.getAllNonStaticMembers(KClassImpl.kt:182)
	at kotlin.reflect.jvm.internal.KClassImpl$Data$allMembers$2.invoke(KClassImpl.kt:188)
	at kotlin.reflect.jvm.internal.KClassImpl$Data$allMembers$2.invoke(KClassImpl.kt:188)
	at kotlin.reflect.jvm.internal.ReflectProperties$LazySoftVal.invoke(ReflectProperties.java:70)
	at kotlin.reflect.jvm.internal.ReflectProperties$Val.getValue(ReflectProperties.java:32)
	at kotlin.reflect.jvm.internal.KClassImpl$Data.getAllMembers(KClassImpl.kt:188)
	at kotlin.reflect.jvm.internal.KClassImpl.getMembers(KClassImpl.kt:206)
	at kotlin.reflect.full.KClasses.getFunctions(KClasses.kt:89)
	at kotlin.reflect.jvm.ReflectJvmMapping.getKotlinFunction(ReflectJvmMapping.kt:144)
	at net.jqwik.kotlin.internal.SuspendedPropertyMethodsHook.isSuspendFunction(SuspendedPropertyMethodsHook.kt:41)
	at net.jqwik.kotlin.internal.SuspendedPropertyMethodsHook.access$isSuspendFunction(SuspendedPropertyMethodsHook.kt:19)
	at net.jqwik.kotlin.internal.SuspendedPropertyMethodsHook$appliesTo$1.invoke(SuspendedPropertyMethodsHook.kt:24)
	at net.jqwik.kotlin.internal.SuspendedPropertyMethodsHook$appliesTo$1.invoke(SuspendedPropertyMethodsHook.kt:24)
	at net.jqwik.kotlin.internal.SuspendedPropertyMethodsHook.appliesTo$lambda$0(SuspendedPropertyMethodsHook.kt:24)
	at java.base/java.util.Optional.map(Optional.java:260)
	at net.jqwik.kotlin.internal.SuspendedPropertyMethodsHook.appliesTo(SuspendedPropertyMethodsHook.kt:24)
	at net.jqwik.kotlin.internal.SuspendedPropertyMethodsHook.appliesTo(SuspendedPropertyMethodsHook.kt:19)
	at net.jqwik.engine.execution.lifecycle.LifecycleHooksRegistry.hookAppliesTo(LifecycleHooksRegistry.java:109)
	at net.jqwik.engine.execution.lifecycle.LifecycleHooksRegistry.lambda$findHooks$1(LifecycleHooksRegistry.java:102)
	at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:178)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
	at net.jqwik.engine.execution.lifecycle.LifecycleHooksRegistry.findHooks(LifecycleHooksRegistry.java:104)
	at net.jqwik.engine.execution.lifecycle.LifecycleHooksRegistry.resolveParameterHook(LifecycleHooksRegistry.java:58)
	at net.jqwik.engine.execution.PropertyTaskCreator.createLifecycleContext(PropertyTaskCreator.java:87)
	at net.jqwik.engine.execution.PropertyTaskCreator.lambda$createTask$2(PropertyTaskCreator.java:34)
	at net.jqwik.engine.execution.pipeline.ExecutionTask$1.lambda$execute$0(ExecutionTask.java:31)
	at net.jqwik.engine.execution.lifecycle.CurrentTestDescriptor.runWithDescriptor(CurrentTestDescriptor.java:17)
	at net.jqwik.engine.execution.pipeline.ExecutionTask$1.execute(ExecutionTask.java:31)
	at net.jqwik.engine.execution.pipeline.ExecutionPipeline.runToTermination(ExecutionPipeline.java:82)
	at net.jqwik.engine.execution.JqwikExecutor.execute(JqwikExecutor.java:46)
	at net.jqwik.engine.JqwikTestEngine.executeTests(JqwikTestEngine.java:70)
	at net.jqwik.engine.JqwikTestEngine.execute(JqwikTestEngine.java:53)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:147)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:127)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:90)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:55)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:102)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:54)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
	at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
	at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53)
	at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:57)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
	at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:232)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55)

That's from intellij, but the trace is essentially the same as running from mvn test, just the root changes to surefire.

Suggested Solution

Discussion
@jlink
Copy link
Collaborator

jlink commented Mar 9, 2024

@twentylemon Thanks for reporting this issue.
Have you tried older versions of jqwik and/or Kotlin?
Have you tried without Lombok being present?

@twentylemon twentylemon reopened this Mar 9, 2024
@twentylemon
Copy link
Author

twentylemon commented Mar 9, 2024 •
edited

I have tried with jqwik 1.8.1, I upgraded as part of debugging. The project contains a lot of Lombok, so removing that isn't a realistic option -- maybe in a few months once most code becomes kotlin.

About Lombok, there isn't any used in the test class. Makes me feel like it shouldn't be an issue. But Lombok has definitely surprised me many times before.

I can try downgrading kotlin, but I am pinned to 1.9. It's a work project, so I'll report back after the weekend 🙂

@jlink
Copy link
Collaborator

jlink commented Mar 9, 2024

Do you have an isolated repo to replicate the problem?

I cannot replicate your problem. I duplicated your setup except lombok being enabled since that seems to require a config file that I haven't.

Maybe the java version plays a role. My properties look like that:

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>${maven.compiler.source}</maven.compiler.target>
        <jqwik.version>1.8.3</jqwik.version>
        <kotlin.version>1.9.22</kotlin.version>
    </properties>

@jlink
Copy link
Collaborator

jlink commented Mar 9, 2024

But something seems to be wrong with the kotlin-maven-plugin. At least on my machine, IntelliJ no longer marks the Kotlin test classes and methods with the "Run Test" triangles; that's never a problem in my Kotlin Gradle set ups.

@twentylemon
Copy link
Author

But something seems to be wrong with the kotlin-maven-plugin. At least on my machine, IntelliJ no longer marks the Kotlin test classes and methods with the "Run Test" triangles; that's never a problem in my Kotlin Gradle set ups.

I haven't noticed anything like that. I've used the hotkey for run tests many times without fail. The kotlin-lombok plugin is experimental, but the overall kotlin-maven one -- at least I haven't noticed anything unusual.

I duplicated your setup except lombok being enabled since that seems to require a config file that I haven't.

IIRC that file is effectively empty. I can confirm after the weekend though.

Do you have an isolated repo to replicate the problem?

I don't. I'll try to whip one up; again, after the weekend though.

Thanks for quick responses!

twentylemon added a commit to twentylemon/jqwik-kotlin-npe that referenced this issue Mar 11, 2024
@twentylemon
🧪 minimal jqwik-kotlin npe example jqwik-team/jqwik#557
5c19f60
@twentylemon
Copy link
Author

@jlink I was able to get a minimal repo together: https://github.com/twentylemon/jqwik-kotlin-npe -- lombok seems to be a red herring, I was able to remove that altogether and still see the error.

I tried downgraded kotlin to 1.8.0, but then my tests down run at all 🤔

@jlink
Copy link
Collaborator

jlink commented Mar 12, 2024

@twentylemon I can replicate! Thanks.

@jlink
Copy link
Collaborator

jlink commented Mar 12, 2024

@twentylemon Looks like a bug in Kotlin library. I'll probably have to find a workaround.

jlink added a commit that referenced this issue Mar 12, 2024
@jlink
Trying a fix for #557
e4409d4
@jlink
Copy link
Collaborator

jlink commented Mar 12, 2024

@twentylemon Please Try release "1.8.4-SNAPSHOT"

@twentylemon
Copy link
Author

twentylemon commented Mar 12, 2024 •
edited

Seems my workplace doesn't have snapshot releases. They lock down maven central for """security reasons.""" I'll be able to try later tonight on my personal machine.

@twentylemon
Copy link
Author

twentylemon commented Mar 12, 2024 •
edited

Using 1.8.4 in my example repo worked! Unfortunately I wasn't able to test it out on my work machine in a larger repo, though I think I realize now the issue was the wrong snapshots url. I didn't realize you used s01.oss.yadayadayada. I'll try again at work tomorrow, but it does seem like it should be solved 😄

@jlink
Copy link
Collaborator

jlink commented Mar 13, 2024

s01.oss

It's a faster instance than the default one.
I switched over a year ago but probably forgot to update some references in the jqwik-samples repository.

@twentylemon
Copy link
Author

Even with the correct snapshots repo, seems I am still blocked on my work pc. Oddly though, I can just download the damn jar, so that makes sense.

After adding that as a system dependency, tests are indeed running normally on the big repo as well 🎉

@jlink jlink added bug and removed in progress labels Mar 14, 2024
@jlink
Copy link
Collaborator

jlink commented Mar 14, 2024

@twentylemon I'll mark this as closed. Feel free to reopen if some problem with the current solution shows up later.

@jlink jlink closed this as completed Mar 14, 2024
@twentylemon
Copy link
Author

@jlink Would this fix be part of the 1.8.4 release? Any clue when that would be available? I'd like to use the library for the big project, but I can't rely on a static jar or a snapshot release for that.

@jlink
Copy link
Collaborator

jlink commented Mar 14, 2024

I could release it at the weekend if that helps you.

@twentylemon
Copy link
Author

I don't mean to rush you. Whatever your release schedule normally is, that's good by me. Would just like to know so I can start using it 😄

@jlink
Copy link
Collaborator

jlink commented Mar 15, 2024

1.8.4 has just been released.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jlink @twentylemon