Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Does Pitest work with the junit-quickcheck property test tool? #337

Open
ricardosm opened this issue Apr 20, 2021 · 2 comments
Open

Does Pitest work with the junit-quickcheck property test tool? #337

ricardosm opened this issue Apr 20, 2021 · 2 comments

Comments

@ricardosm
Copy link

I am trying to use the junit-quickcheck property testing tool together with Pirtest 1.6.4. However, I am encountering a problem. Pitest doesn't seem to be able to find the tests in my test class.

The message "PIT >> INFO: Found 0 tests" is always returned.

I believe that this problem occurs because the junit-quickcheck tool requires @RunWith (JUnitQuickcheck.class).

Does anyone have a solution to this problem?

Below is my code, test code and pom.xml:

package sort_selection_sort;

public class SelectionSort {

	public int[] sort(int arr[]) {
		int n = arr.length;

		// One by one move boundary of unsorted subarray
		for (int i = 0; i < n - 1; i++) {
			// Find the minimum element in unsorted array
			int min_idx = i;
			for (int j = i + 1; j < n; j++)
				if (arr[j] < arr[min_idx])
					min_idx = j;

			// Swap the found minimum element with the first
			// element
			int temp = arr[min_idx];
			arr[min_idx] = arr[i];
			arr[i] = temp;
		}
		
		return arr;
	}
}

package sort_selection_sort;

import static org.junit.Assert.assertTrue;

import java.util.Arrays;

import org.junit.runner.RunWith;

import com.pholser.junit.quickcheck.Property;
import com.pholser.junit.quickcheck.runner.JUnitQuickcheck;

@RunWith(JUnitQuickcheck.class)
public class SelectionSortPropertiesTest {
	
	@Property(trials = 10)
	public void property(int arr[]) {
		SelectionSort ob = new SelectionSort();
		int[] copyOfArr = Arrays.copyOf(arr, arr.length);
		Arrays.sort(copyOfArr);
		int[] sort = ob.sort(arr);
			
		assertTrue(Arrays.equals(copyOfArr, sort));
	}
}


	<build>
		<plugins>
			<plugin>
				<artifactId>maven-compiler-plugin</artifactId>
				<version>3.8.1</version>
			</plugin>

			<!-- Configuration of Maven Surefire Plugin for support to jUnit 5 (Unit 
				tests) -->
			<plugin>
				<artifactId>maven-surefire-plugin</artifactId>
				<version>2.22.2</version>
			</plugin>

			<!-- Configuration of Pitest and Pitest jUnit 5 plugin -->
			<plugin>
				<groupId>org.pitest</groupId>
				<artifactId>pitest-maven</artifactId>
				<version>1.6.4</version>
				
				<configuration>
					<targetClasses>
						<param>sort_selection_sort.*</param>
					</targetClasses>
					<targetTests>
						<param>sort_selection_sort.*</param>
					</targetTests>
					<mutators>
						<mutator>ALL</mutator>
					</mutators>
					<verbose>true</verbose>
				</configuration>
			</plugin>
		</plugins>
	</build>
	<!-- ... -->
	<dependencies>
		
		<dependency>
			<groupId>junit</groupId>
			<artifactId>junit</artifactId>
			<version>4.13</version>
			<scope>test</scope>
		</dependency>
		
		<dependency>
			<groupId>com.pholser</groupId>
			<artifactId>junit-quickcheck-core</artifactId>
			<version>1.0</version>
		</dependency>

		<dependency>
			<groupId>com.pholser</groupId>
			<artifactId>junit-quickcheck-generators</artifactId>
			<version>1.0</version>
		</dependency>

		<dependency>
			<groupId>com.pholser</groupId>
			<artifactId>junit-quickcheck-guava</artifactId>
			<version>1.0</version>
		</dependency>
				
		<dependency>
			<groupId>org.slf4j</groupId>
			<artifactId>slf4j-api</artifactId>
			<version>1.7.30</version>
		</dependency>
		<dependency>
			<groupId>org.slf4j</groupId>
			<artifactId>slf4j-simple</artifactId>
			<version>1.7.30</version>
		</dependency>
		<!--  -->
	</dependencies>

16:55:16 PIT >> FINE : Maximum available memory is 3612 mb
16:55:16 PIT >> FINE : MINION : Installing PIT agent
16:55:16 PIT >> INFO : Sending 2 test classes to minion
16:55:16 PIT >> INFO : Sent tests to minion
16:55:17 PIT >> INFO : MINION : 16:55:16 PIT >> FINE : Expecting 2 tests classes from parent
16:55:17 PIT >> INFO : MINION : 16:55:16 PIT >> FINE : Tests classes received
16:55:17 PIT >> INFO : MINION : 16:55:16 PIT >> INFO : Checking environment
16:55:17 PIT >> INFO : MINION : 16:55:17 PIT >> INFO : Found  0 tests
16:55:17 PIT >> INFO : MINION : 16:55:17 PIT >> INFO : Dependency analysis reduced number of potential tests by 0
16:55:17 PIT >> INFO : MINION : 16:55:17 PIT >> INFO : 0 tests received
16:55:17 PIT >> INFO : MINION : 16:55:17 PIT >> FINE : Running 0 units
16:55:17 PIT >> FINE : Coverage generator Minion exited ok
16:55:17 PIT >> INFO : Calculated coverage in 0 seconds.
16:55:17 PIT >> FINE : Used memory after coverage calculation 23 mb
16:55:17 PIT >> FINE : Free Memory after coverage calculation 219 mb
16:55:17 PIT >> FINE : Incremental analysis set 153 mutations to a status of NOT_STARTED
16:55:17 PIT >> INFO : Incremental analysis reduced number of mutations by 0
16:55:17 PIT >> INFO : Created  1 mutation test units
16:55:17 PIT >> FINE : Used memory before analysis start 47 mb
16:55:17 PIT >> FINE : Free Memory before analysis start 136 mb
16:55:17 PIT >> FINE : Running 1 units
16:55:17 PIT >> INFO : Completed in 0 seconds
================================================================================
- Mutators
================================================================================
> org.pitest.mutationtest.engine.gregor.mutators.rv.AOR3Mutator
>> Generated 2 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 2 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.AOR2Mutator
>> Generated 2 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 2 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.AOR1Mutator
>> Generated 2 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 2 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.AOR4Mutator
>> Generated 2 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 2 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.RemoveConditionalMutator_ORDER_IF
>> Generated 3 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 3 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.MathMutator
>> Generated 2 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 2 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.CRCR3Mutator
>> Generated 3 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 3 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.ConditionalsBoundaryMutator
>> Generated 3 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 3 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.IncrementsMutator
>> Generated 2 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 2 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.experimental.RemoveIncrementsMutator
>> Generated 1 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 1 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.CRCR4Mutator
>> Generated 2 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 2 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.CRCR2Mutator
>> Generated 2 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 2 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.NullReturnValsMutator
>> Generated 1 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 1 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.AOD1Mutator
>> Generated 2 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 2 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.InlineConstantMutator
>> Generated 3 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 3 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.AOD2Mutator
>> Generated 2 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 2 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.ReturnValsMutator
>> Generated 1 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 1 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.CRCR6Mutator
>> Generated 3 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 3 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.CRCR1Mutator
>> Generated 1 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 1 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.CRCR5Mutator
>> Generated 3 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 3 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.ABSMutator
>> Generated 18 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 18 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.RemoveConditionalMutator_ORDER_ELSE
>> Generated 3 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 3 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.ROR3Mutator
>> Generated 3 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 3 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.ROR5Mutator
>> Generated 3 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 3 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.ROR1Mutator
>> Generated 3 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 3 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.ROR2Mutator
>> Generated 3 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 3 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.ROR4Mutator
>> Generated 3 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 3 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.UOI1Mutator
>> Generated 18 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 18 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.NegateConditionalsMutator
>> Generated 3 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 3 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.UOI3Mutator
>> Generated 18 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 18 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.UOI2Mutator
>> Generated 18 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 18 
--------------------------------------------------------------------------------
> org.pitest.mutationtest.engine.gregor.mutators.rv.UOI4Mutator
>> Generated 18 Killed 0 (0%)
> KILLED 0 SURVIVED 0 TIMED_OUT 0 NON_VIABLE 0 
> MEMORY_ERROR 0 NOT_STARTED 0 STARTED 0 RUN_ERROR 0 
> NO_COVERAGE 18 
--------------------------------------------------------------------------------
================================================================================
- Timings
================================================================================
> scan classpath : < 1 second
> coverage and dependency analysis : < 1 second
> build mutation tests : < 1 second
> run mutation analysis : < 1 second
--------------------------------------------------------------------------------
> Total  : < 1 second
--------------------------------------------------------------------------------
================================================================================
- Statistics
================================================================================
>> Generated 153 mutations Killed 0 (0%)
>> Mutations with no coverage 153. Test strength 0%
>> Ran 0 tests (0 tests per mutation)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.596 s
[INFO] Finished at: 2021-04-20T16:55:17-03:00
[INFO] ------------------------------------------------------------------------
@pholser
Copy link
Owner

pholser commented Apr 20, 2021

@hcoles Any advice you can offer? TBH I haven't tried running junit-quickcheck tests under pitest.

@tj-byun
Copy link

tj-byun commented Jul 17, 2021 •
edited

Pitest + junit-quickcheck will be super cool..! Strong test generator & oracle meets strong test adequacy criterion.
@ricardosm Did you get it to work yet, by any chance?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@pholser @tj-byun @ricardosm