Introduce skipNativeTests flag in Maven plugin (#193)

Prior to this commit, the NativeTestMojo had a `skipTests` property.
This meant that you could skip the tests, but that skipped the JVM
tests as well as the native tests, since Maven Surefire looks for the
same `skipTests` flag. In other words, there was previously no way to
execute the JVM tests and skip only the native tests.

This commit addresses this issue by introducing a new `skipNativeTests`
flag that allows the user to explicitly disable only native testing
support.

For example, this new feature enables workflows that need to execute
tests on the JVM using the agent in order to generate native
configuration files without having to execute the tests within a native
image. See the updated reference documentation for additional use cases.

Closes #179

docs/src/docs/asciidoc/gradle-plugin.adoc

@@ -229,16 +229,21 @@ Currently, this feature requires the execution of the tests in the classic "JVM"
 
 NOTE: This plugin requires JUnit Platform 1.8 or higher.
 
-=== Disabling test support
+[[testing-support-disabling]]
+=== Disabling testing support
 
-There are cases where you might want to disable test support:
+There are cases where you might want to disable native testing support:
 
-- you don't actually want to run your tests in native mode
-- your library or application uses a testing framework that is not supported on the JUnit Platform
+- You don't actually want to run your tests in native mode.
+- Your library or application uses a testing framework that is not supported on the JUnit
+  Platform.
+- You need to use the <<agent-support, agent>> when running tests on the JVM but do not
+  wish to run those same tests in native mode.
 
-In this case, you can disable test support by configuring the `graalvmNative` extension:
+In this case, you can disable native testing support by configuring the `graalvmNative`
+extension as follows:
 
-.Disabling test support
+.Disabling testing support
 [source,groovy,role="multi-language-sample"]
 ----
 include::../snippets/gradle/groovy/build.gradle[tags=disable-test-support]

docs/src/docs/asciidoc/index.adoc

@@ -15,6 +15,13 @@ If you are interested in contributing, please refer to our https://github.com/gr
 [[changelog]]
 == Changelog
 
+=== Release 0.9.10
+
+==== Maven plugin
+
+* Native testing support can now be explicitly disabled via `skipNativeTests`.
+   - See <<maven-plugin.adoc#testing-support-disabling, Disabling testing support>> for details.
+
 === Release 0.9.9
 
 ==== Gradle plugin

docs/src/docs/asciidoc/maven-plugin.adoc

@@ -202,6 +202,25 @@ NOTE: This plugin requires JUnit Platform 1.8 or higher.
 
 NOTE: This plugin provides integration with Maven Surefire 2.22.0 or higher.
 
+[[testing-support-disabling]]
+=== Disabling testing support
+
+If you wish to disable tests on the JVM as well as tests within a native image, you can
+invoke Maven with the `-DskipTests` flag. This flag is supported by Maven Surefire and
+Native Build Tools. Several examples in <<agent-support-running-application>> demonstrate
+the use of this flag.
+
+If you wish to run tests on the JVM with Maven Surefire but skip testing within a native
+image, you can invoke Maven with the `-DskipNativeTests` flag. This flag is specific to
+Native Build Tools. For example, you might wish to disable only native testing support for
+use cases such as the following:
+
+- You don't actually want to run your tests in native mode.
+- Your library or application uses a testing framework that is not supported on the JUnit
+  Platform.
+- You need to use the <<agent-support, agent>> when running tests on the JVM but do not
+  wish to run those same tests in native mode.
+
 [[long_classpath_and_shading_support]]
 == Long classpath and shading support
 
@@ -404,23 +423,23 @@ include::../../../../samples/java-application-with-reflection/pom.xml[tag=java-a
 Then you can execute your application with the agent by running:
 
 ```bash
-mvn -Pnative -Dagent=true -DskipTests=true -DskipNativeBuild=true package exec:exec@java-agent
+mvn -Pnative -Dagent=true -DskipTests -DskipNativeBuild=true package exec:exec@java-agent
 ```
 
 To execute your application with custom agent options, supply the `-DagentOptions=<NAME>`
 command-line argument to Maven as follows. See the documentation for
 <<agent-support-configuring-options, agent options>> for details.
 
 ```bash
-mvn -Pnative -Dagent=true -DagentOptions=periodic-config -DskipTests=true -DskipNativeBuild=true package exec:exec@java-agent
+mvn -Pnative -Dagent=true -DagentOptions=periodic-config -DskipTests -DskipNativeBuild=true package exec:exec@java-agent
 ```
 
 Both of the above commands will generate configuration files in the
 `target/native/agent-output/main` directory. If you want to run your native application
 with those configuration files, you then need to execute the following command:
 
 ```bash
-mvn -Pnative -Dagent=true -DskipTests=true package exec:exec@native
+mvn -Pnative -Dagent=true -DskipTests package exec:exec@native
 ```
 
 WARNING: If the agent is enabled, the `--allow-incomplete-classpath` option is

native-maven-plugin/src/functionalTest/groovy/org/graalvm/buildtools/maven/JavaApplicationWithAgentFunctionalTest.groovy

@@ -46,6 +46,38 @@ import spock.lang.Unroll
 
 class JavaApplicationWithAgentFunctionalTest extends AbstractGraalVMMavenFunctionalTest {
 
+    @Issue("https://github.com/graalvm/native-build-tools/issues/179")
+    def "agent is used for JVM tests when native image tests are skipped via -DskipNativeTests"() {
+        given:
+        withSample("java-application-with-reflection")
+
+        when:
+        // Run Maven in debug mode (-X) in order to capture the command line arguments
+        // used to launch Surefire with the agent.
+        mvn '-X', '-Pnative', 'test', '-DskipNativeTests'
+
+        then:
+        // Agent is used with Surefire
+        outputContains '-agentlib:native-image-agent='
+
+        and:
+        // Agent generates files
+        ['jni', 'proxy', 'reflect', 'resource', 'serialization'].each { name ->
+            assert file("target/native/agent-output/test/${name}-config.json").exists()
+        }
+
+        and:
+        // Surefire / JVM tests run
+        buildSucceeded
+        outputContains "SurefirePlugin - Running org.graalvm.demo.ApplicationTest"
+        outputContains "SurefirePlugin - Running org.graalvm.demo.CalculatorTest"
+
+        and:
+        // Native tests do not run
+        outputContains "Skipping native-image tests (parameter 'skipTests' or 'skipNativeTests' is true)."
+        outputDoesNotContain "containers found"
+    }
+
     def "agent is used for tests when enabled in POM without custom options"() {
         given:
         withSample("java-application-with-reflection")

native-maven-plugin/src/functionalTest/groovy/org/graalvm/buildtools/maven/JavaApplicationWithTestsFunctionalTest.groovy

@@ -41,7 +41,10 @@
 
 package org.graalvm.buildtools.maven
 
+import spock.lang.Issue
+
 class JavaApplicationWithTestsFunctionalTest extends AbstractGraalVMMavenFunctionalTest {
+
     def "can run tests in a native image with the Maven plugin"() {
         withSample("java-application-with-tests")
 
@@ -91,4 +94,33 @@ class JavaApplicationWithTestsFunctionalTest extends AbstractGraalVMMavenFunctio
 [         0 tests failed          ]
 """.trim()
     }
+
+    @Issue("https://github.com/graalvm/native-build-tools/issues/179")
+    def "can skip JVM tests and native image tests with the Maven plugin with -DskipTests"() {
+        withSample("java-application-with-tests")
+
+        when:
+        mvn '-Pnative', 'test', '-DskipTests'
+
+        then:
+        buildSucceeded
+        outputDoesNotContain "SurefirePlugin - Running org.graalvm.demo.CalculatorTest"
+        outputContains "Skipping native-image tests (parameter 'skipTests' or 'skipNativeTests' is true)."
+        outputDoesNotContain "containers found"
+    }
+
+    @Issue("https://github.com/graalvm/native-build-tools/issues/179")
+    def "can skip native image tests with the Maven plugin with -DskipNativeTests"() {
+        withSample("java-application-with-tests")
+
+        when:
+        mvn '-Pnative', 'test', '-DskipNativeTests'
+
+        then:
+        buildSucceeded
+        outputContains "SurefirePlugin - Running org.graalvm.demo.CalculatorTest"
+        outputContains "Skipping native-image tests (parameter 'skipTests' or 'skipNativeTests' is true)."
+        outputDoesNotContain "containers found"
+    }
+
 }

native-maven-plugin/src/main/java/org/graalvm/buildtools/maven/NativeBuildMojo.java

@@ -102,9 +102,10 @@ public class NativeBuildMojo extends AbstractNativeMojo {
 
     private PluginParameterExpressionEvaluator evaluator;
 
+    @Override
     public void execute() throws MojoExecutionException {
         if (skip) {
-            getLog().info("Skipping native-image generation (parameter 'skip' is true).");
+            getLog().info("Skipping native-image generation (parameter 'skipNativeBuild' is true).");
             return;
         }
         evaluator = new PluginParameterExpressionEvaluator(session, mojoExecution);

native-maven-plugin/src/main/java/org/graalvm/buildtools/maven/NativeTestMojo.java

@@ -81,6 +81,9 @@ public class NativeTestMojo extends AbstractNativeMojo {
     @Parameter(property = "skipTests", defaultValue = "false")
     private boolean skipTests;
 
+    @Parameter(property = "skipNativeTests", defaultValue = "false")
+    private boolean skipNativeTests;
+
     @Parameter(property = "classpath")
     private List<String> classpath;
 
@@ -89,8 +92,8 @@ public class NativeTestMojo extends AbstractNativeMojo {
 
     @Override
     public void execute() throws MojoExecutionException, MojoFailureException {
-        if (skipTests) {
-            logger.info("Tests are skipped.");
+        if (skipTests || skipNativeTests) {
+            logger.info("Skipping native-image tests (parameter 'skipTests' or 'skipNativeTests' is true).");
             return;
         }
         if (!hasTests()) {