Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1.8.0 contracts regression #519

Closed
SimY4 opened this issue Sep 5, 2023 · 16 comments
Closed

1.8.0 contracts regression #519

SimY4 opened this issue Sep 5, 2023 · 16 comments
Labels

Comments

@SimY4
Copy link

SimY4 commented Sep 5, 2023

Testing Problem

interface EqualityContract<T> {
  @Property
  default boolean reflexivityEq(@ForAll("anyT") T value) {
    return Objects.equals(value, value);
  }
  Arbitrary<T> anyT();
}

class MyTypeTest implements EqualityContract<MyType> {
  @Provide("anyT")
  Arbitrary<MyType> anyT() { ... }
}

fails with:

net.jqwik.api.CannotFindArbitraryException: Cannot find an Arbitrary [anyT] for Parameter of type [T]
	at net.jqwik.engine.properties.RandomizedShrinkablesGenerator.resolveArbitraries(RandomizedShrinkablesGenerator.java:152)
	at net.jqwik.engine.properties.RandomizedShrinkablesGenerator.resolveEdgeCases(RandomizedShrinkablesGenerator.java:124)
	at net.jqwik.engine.properties.RandomizedShrinkablesGenerator.listOfEdgeCases(RandomizedShrinkablesGenerator.java:93)
	at net.jqwik.engine.properties.RandomizedShrinkablesGenerator.forParameters(RandomizedShrinkablesGenerator.java:28)
	at net.jqwik.engine.execution.CheckedProperty.createRandomizedShrinkablesGenerator(CheckedProperty.java:236)
	at net.jqwik.engine.execution.CheckedProperty.createForAllParametersGenerator(CheckedProperty.java:176)
	at net.jqwik.engine.execution.CheckedProperty.createGenericProperty(CheckedProperty.java:144)
	at net.jqwik.engine.execution.CheckedProperty.check(CheckedProperty.java:67)
	at net.jqwik.engine.execution.PropertyMethodExecutor.executeProperty(PropertyMethodExecutor.java:90)
	at net.jqwik.engine.execution.PropertyMethodExecutor.executeMethod(PropertyMethodExecutor.java:69)
	at net.jqwik.engine.execution.PropertyMethodExecutor.lambda$execute$0(PropertyMethodExecutor.java:49)
	at net.jqwik.api.lifecycle.AroundPropertyHook.lambda$static$0(AroundPropertyHook.java:46)
	at net.jqwik.engine.execution.lifecycle.HookSupport.lambda$wrap$0(HookSupport.java:26)
	at net.jqwik.api.lifecycle.PropertyExecutor.executeAndFinally(PropertyExecutor.java:39)
	at net.jqwik.engine.hooks.lifecycle.PropertyLifecycleMethodsHook.aroundProperty(PropertyLifecycleMethodsHook.java:56)
	at net.jqwik.engine.execution.lifecycle.HookSupport.lambda$wrap$1(HookSupport.java:31)
	at net.jqwik.engine.execution.lifecycle.HookSupport.lambda$wrap$0(HookSupport.java:26)
	at net.jqwik.engine.hooks.statistics.StatisticsHook.aroundProperty(StatisticsHook.java:37)
	at net.jqwik.engine.execution.lifecycle.HookSupport.lambda$wrap$1(HookSupport.java:31)
	at net.jqwik.engine.execution.lifecycle.HookSupport.lambda$wrap$0(HookSupport.java:26)
	at net.jqwik.engine.hooks.lifecycle.AutoCloseableHook.aroundProperty(AutoCloseableHook.java:13)
	at net.jqwik.engine.execution.lifecycle.HookSupport.lambda$wrap$1(HookSupport.java:31)
	at net.jqwik.engine.execution.PropertyMethodExecutor.execute(PropertyMethodExecutor.java:47)
	at net.jqwik.engine.execution.PropertyTaskCreator.executeTestMethod(PropertyTaskCreator.java:166)
	at net.jqwik.engine.execution.PropertyTaskCreator.lambda$createTask$1(PropertyTaskCreator.java:51)
	at net.jqwik.engine.execution.lifecycle.CurrentDomainContext.runWithContext(CurrentDomainContext.java:28)
	at net.jqwik.engine.execution.PropertyTaskCreator.lambda$createTask$2(PropertyTaskCreator.java:50)
	at net.jqwik.engine.execution.pipeline.ExecutionTask$1.lambda$execute$0(ExecutionTask.java:31)
	at net.jqwik.engine.execution.lifecycle.CurrentTestDescriptor.runWithDescriptor(CurrentTestDescriptor.java:17)
	at net.jqwik.engine.execution.pipeline.ExecutionTask$1.execute(ExecutionTask.java:31)
	at net.jqwik.engine.execution.pipeline.ExecutionPipeline.runToTermination(ExecutionPipeline.java:82)
	at net.jqwik.engine.execution.JqwikExecutor.execute(JqwikExecutor.java:46)
	at net.jqwik.engine.JqwikTestEngine.executeTests(JqwikTestEngine.java:70)
	at net.jqwik.engine.JqwikTestEngine.execute(JqwikTestEngine.java:53)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
	at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:118)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:93)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:88)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:62)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at jdk.proxy2/jdk.proxy2.$Proxy5.stop(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:193)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
	at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:113)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:65)
	at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
	at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
@jlink
Copy link
Collaborator

jlink commented Sep 6, 2023

I just tried this successfully with the latest version of jqwik (openjdk 17):

interface EqualityContract<T> {
	@Property
	@Report(Reporting.GENERATED)
	default boolean reflexivityEq(@ForAll("anyT") T value) {
		return Objects.equals(value, value);
	}

	Arbitrary<T> anyT();
}

class MyTypeTest implements EqualityContract<MyType> {

	@Provide("anyT")
	public Arbitrary<MyType> anyT() {
		return Arbitraries.strings().alpha().map(MyType::new);
	}
}

class MyType {
	private final String name;

	MyType(String name) {
		this.name = name;
	}

	@Override
	public String toString() {
		return "My(" + name + ")";
	}
}

Maybe an old version is loaded somehow? Or Java version has an influence?

@SimY4
Copy link
Author

SimY4 commented Sep 6, 2023

@jlink Sorry, getting more details as I look into this problem more. My simplified example is not representable and in reality was a bit more nuanced:

interface EqualityContract<T> {
  @Property
  default boolean reflexivityEq(@ForAll("anyT") T value) {
    return Objects.equals(value, value);
  }
  Arbitrary<T> anyT();
}

interface ComparableContract<T extends Comparable<? super T>> extends EqualityContract<T> {}

class MyTypeTest implements ComparableContract<String> {
  @Provide("anyT")
  Arbitrary<String> anyT() { return Arbitraries.strings(); }
}

@jlink
Copy link
Collaborator

jlink commented Sep 6, 2023

Still seems to work with the extended contract. I assume something else is going on.

@SimY4
Copy link
Author

SimY4 commented Sep 7, 2023

@jlink It's yet another weird one... If I try to debug it and stall an execution it'll pass for me too. But running a test in a class would make it fail 100% of time on my machine. This makes me wonder where this race could come from. 🤔

In my case, I see that CachingArbitraryResolver cached the value of an empty set for my type parameter. But if I stall it on compute if absent it'll return 5 arbitraries.

@jlink
Copy link
Collaborator

jlink commented Sep 7, 2023

Maybe you can set up a repo that I can use to replicate the problem.

@SimY4
Copy link
Author

SimY4 commented Sep 8, 2023

Found minimal example that breaks:

interface A<T> {
  @Property
  default boolean test1(@ForAll("anyT") T t) {
    return true;
  }

  Arbitrary<T> anyT();
}

interface B<T extends Comparable<? super T>> extends A<T> {
  @Property
  default boolean test2(@ForAll("anyT") T t) {
    return true;
  }
}

interface MyType extends Comparable<MyType> { }

interface C<T extends MyType> extends B<T> {
  @Property
  default boolean test3(@ForAll("anyT") T t) {
    return true;
  }
}

class MyRecord implements MyType {
  @Override
  public int compareTo(MyType o) {
    return 0;
  }
}

public class Test implements C<MyRecord> {
  @Provide("anyT")
  @Override
  public Arbitrary<MyRecord> anyT() {
    return Arbitraries.of(new MyRecord());
  }
}

@jlink
Copy link
Collaborator

jlink commented Sep 8, 2023

@SimY4 Have you seen this work in older versions of jqwik?

@SimY4
Copy link
Author

SimY4 commented Sep 8, 2023

@jlink Yeah, it works on 1.7.4 but it could be by chance. IDK...

@jlink
Copy link
Collaborator

jlink commented Sep 30, 2023

@SimY4 Sorry for the delay. I can reproduce. Looks like a bug. Will try to dive into it in the week to come.

jlink added a commit that referenced this issue Sep 30, 2023
@SimY4
Copy link
Author

SimY4 commented Sep 30, 2023

@jlink no problem. Thank you for looking into this

jlink added a commit that referenced this issue Oct 19, 2023
@jlink jlink added the bug label Oct 19, 2023
@jlink
Copy link
Collaborator

jlink commented Oct 19, 2023

Should be fixed in d6de16d

@jlink
Copy link
Collaborator

jlink commented Oct 19, 2023

Fix released in 1.8.1-SNAPSHOT

@jlink jlink closed this as completed Oct 19, 2023
@jlink jlink removed the in progress label Oct 19, 2023
@SimY4
Copy link
Author

SimY4 commented Oct 19, 2023

Awesome work! Thank you.🙏

@jlink
Copy link
Collaborator

jlink commented Oct 20, 2023

@SimY4 Do you want to update to 1.8? In that case I could publish 1.8.1 soon.

@SimY4
Copy link
Author

SimY4 commented Oct 20, 2023

@jlink i did check the 1.8.1 snapshot on my project. Worked great, no problems detected, all tests are green. 😀

@jlink
Copy link
Collaborator

jlink commented Oct 20, 2023

1.8.1 has been released

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants