Add restore annotations for system properties and environment variabl…

…es (#574 / #700)

To allow users to modify environment variables and system properties
directly in their test code, outside of the existing clear and set
annotations, this PR  introduces `RestoreEnvironmentVariables` and
`RestoreSystemProperties` to revert these resources to their pre-test state.

Closes: #574
PR: #700

README.md

@@ -116,6 +116,9 @@ Thank you for your efforts! 🙏
 
 The least we can do is to thank them and list some of their accomplishments here (in lexicographic order).
 
+#### 2023
+* [Eric Everman](https://github.com/eeverman) added `@RestoreSystemProperties` and `@RestoreEnvironmentVariables` annotations to the [System Properties](https://junit-pioneer.org/docs/system-properties/) and [Environment Variables](https://junit-pioneer.org/docs/environment-variables/) extensions (#574 / #700)
+
 #### 2022
 
 * [Filip Hrisafov](https://github.com/filiphr) contributed the [JSON Argument Source](https://junit-pioneer.org/docs/json-argument-source/) support (#101 / #492)

docs/docs-nav.yml

@@ -5,9 +5,9 @@
     children:
       - title: "Cartesian Product of Parameters"
         url: /docs/cartesian-product/
-      - title: "Clearing or Setting System Properties"
+      - title: "Clear, Set, and Restore System Properties"
         url: /docs/system-properties/
-      - title: "Clearing or Setting Environment Variables"
+      - title: "Clear, Set, and Restore Environment Variables"
         url: /docs/environment-variables/
       - title: "Default Locale and TimeZone"
         url: /docs/default-locale-timezone/

docs/environment-variables.adoc

@@ -1,12 +1,13 @@
-:page-title: Clearing or Setting Environment Variables
-:page-description: The JUnit 5 (Jupiter) extensions `@ClearEnvironmentVariable`/`@SetEnvironmentVariable` clear/set the values of environment variables for the duration of a test
+:page-title: Clear, Set, and Restore Environment Variables
+:page-description: The JUnit 5 (Jupiter) extensions `@ClearEnvironmentVariable`, `@SetEnvironmentVariable` and `@RestoreEnvironmentVariables` clear/set/restore the values of environment variables for the duration of a test, and/or restore them after
 :xp-demo-dir: ../src/demo/java
 :demo: {xp-demo-dir}/org/junitpioneer/jupiter/EnvironmentVariablesExtensionDemo.java
 
+== `@ClearEnvironmentVariable` and `@SetEnvironmentVariable`
 The `@ClearEnvironmentVariable` and `@SetEnvironmentVariable` annotations can be used to clear and set, respectively, the values of environment variables for a test execution.
 Both annotations work on the test method and class level, are repeatable, combinable, and inherited from higher-level containers.
 After the annotated method has been executed, the variables mentioned in the annotation will be restored to their original value or the value of the higher-level container, or will be cleared if they didn't have one before.
-Other environment variables that are changed during the test, are *not* restored.
+Other environment variables that are changed during the test, are *not* restored (unless the `@RestoreEnvironmentVariables` is used).
 
 [WARNING]
 ====
@@ -51,6 +52,59 @@ Method-level configurations are visible in both `@BeforeEach` setup methods and
 Since v1.7.0, a class-level configuration means that the specified environment variables are cleared/set before and reset after each individual test in the annotated class.
 ====
 
+== `@RestoreEnvironmentVariables`
+`@RestoreEnvironmentVariables` can be used to restore changes to environment variables made directly in code.
+While `@ClearEnvironmentVariable` and `@SetEnvironmentVariable` set or clear specific variables and values, they don't allow values to be calculated or parameterized, thus there are times you may want to directly set them in your test code.
+`@RestoreEnvironmentVariables` can be placed on test methods or test classes and will completely restore all environment variables to their original state after a test or test class is complete.
+
+In this example, `@RestoreEnvironmentVariables` is used on a test method, ensuring any changes made in that method are restored:
+
+[source,java,indent=0]
+----
+include::{demo}[tag=environment_method_restore_test]
+----
+
+[NOTE]
+====
+Modifying environment variables of a running JVM requires several lines of code and the Java reflection API.
+The two `@RestoreEnvironmentVariables` examples leave out that detail and use a hypothetical `setEnvVar()` method.
+====
+
+When `@RestoreEnvironmentVariables` is used on a test class, any environment variable changes made during the entire lifecycle of the test class, including test methods, `@BeforeAll`, `@BeforeEach` and 'after' methods, are restored after the test class' lifecycle is complete.
+In addition, the annotation is inherited by each test method just as if each one was annotated with `@RestoreEnvironmentVariables`.
+
+In the following example, both test methods see the environment variable changes made in `@BeforeAll` and `@BeforeEach`, however, the test methods are isolated from each other (`isolatedTest2` does not 'see' changes made in `isolatedTest1`).
+As shown in the second example below, the class-level `@RestoreEnvironmentVariables` ensures that environment variable changes made within the annotated class are completely restored after the class's lifecycle, ensuring that changes are not visible to `SomeOtherTestClass`.
+Note that `SomeOtherTestClass` uses the `@ReadsEnvironmentVariable` annotation: This ensures that JUnit does not schedule the class to run during any test known to modify environment variables (see <<Thread-Safety>>).
+
+[source,java,indent=0]
+----
+include::{demo}[tag=environment_class_restore_setup]
+----
+
+Some other test class, running later:
+
+[source,java,indent=0]
+----
+include::{demo}[tag=environment_class_restore_isolated_class]
+----
+
+== Using `@ClearEnvironmentVariable`, `@SetEnvironmentVariable`, and `@RestoreEnvironmentVariables` together
+All three annotations can be combined, which could be used when some environment values are parameterized (i.e. need to be set in code) and others are not.
+For instance, imagine testing an image generation utility that takes configuration from environment variables.
+Basic configuration can be specified using `Set` and `Clear` and the image size parameterized:
+
+[source,java,indent=0]
+----
+include::{demo}[tag=environment_method_combine_all_test]
+----
+
+[NOTE]
+====
+Using `@RestoreEnvironmentVariables` is not necessary to restore environment variables modified via `@ClearEnvironmentVariable` or `@SetEnvironmentVariable` - they each automatically restore the referenced variables.
+'Restore', is only needed if environment variables are modified in some way _other than_ Clear and Set during a test.
+====
+
 == Warnings for Reflective Access
 
 As explained above, this extension uses reflective access to change the otherwise immutable environment variables.
@@ -91,13 +145,13 @@ These command line options need to be added to the JVM that executes the tests:
 == Thread-Safety
 
 Since environment variables are global state, reading and writing them during https://junit.org/junit5/docs/current/user-guide/#writing-tests-parallel-execution[parallel test execution] can lead to unpredictable results and flaky tests.
-The environment variable extension is prepared for that and tests annotated with `@ClearEnvironmentVariable` or `@SetEnvironmentVariable` will never execute in parallel (thanks to https://junit.org/junit5/docs/current/api/org.junit.jupiter.api/org/junit/jupiter/api/parallel/ResourceLock.html[resource locks]) to guarantee correct test results.
+The environment variable extension is prepared for that and tests annotated with `@ClearEnvironmentVariable`, `@SetEnvironmentVariable`, or `@RestoreEnvironmentVariables` will never execute in parallel (thanks to https://junit.org/junit5/docs/current/api/org.junit.jupiter.api/org/junit/jupiter/api/parallel/ResourceLock.html[resource locks]) to guarantee correct test results.
 
 However, this does not cover all possible cases.
 Tested code that reads or writes environment variables _independently_ of the extension can still run in parallel to it and may thus behave erratically when, for example, it unexpectedly reads a variable set by the extension in another thread.
 Tests that cover code that reads or writes environment variables need to be annotated with the respective annotation:
 
 * `@ReadsEnvironmentVariable`
-* `@WritesEnvironmentVariable`
+* `@WritesEnvironmentVariable` (though consider using `@RestoreEnvironmentVariables` instead)
 
-Tests annotated in this way will never execute in parallel with tests annotated with `@ClearEnvironmentVariable` or `@SetEnvironmentVariable`.
+Tests annotated in this way will never execute in parallel with tests annotated with `@ClearEnvironmentVariable`, `@SetEnvironmentVariable`, or `@RestoreEnvironmentVariables`.

docs/system-properties.adoc

@@ -1,12 +1,13 @@
-:page-title: Clearing or Setting System Properties
-:page-description: The JUnit 5 (Jupiter) extensions `@ClearSystemProperty`/`@SetSystemProperty` clear/set the values of system properties for the duration of a test
+:page-title: Clear, Set, and Restore System Properties
+:page-description: The JUnit 5 (Jupiter) extensions `@ClearSystemProperty`, `@SetSystemProperty` and `@RestoreSystemProperties` clear/set/restore the values of system properties for the duration of a test, and/or restore them after
 :xp-demo-dir: ../src/demo/java
 :demo: {xp-demo-dir}/org/junitpioneer/jupiter/SystemPropertyExtensionDemo.java
 
+== `@ClearSystemProperty` and `@SetSystemProperty`
 The `@ClearSystemProperty` and `@SetSystemProperty` annotations can be used to clear and set, respectively, the values of system properties for a test execution.
 Both annotations work on the test method and class level, are repeatable, combinable, and inherited from higher-level containers.
 After the annotated method has been executed, the properties mentioned in the annotation will be restored to their original value or the value of the higher-level container, or will be cleared if they didn't have one before.
-Other system properties that are changed during the test, are *not* restored.
+Other system properties that are changed during the test, are *not* restored (unless the `@RestoreSystemProperties` is used).
 
 For example, clearing a system property for a test execution can be done as follows:
 
@@ -44,24 +45,71 @@ Method-level configurations are visible in both `@BeforeEach` setup methods and
 Since v1.7.0, a class-level configuration means that the specified system properties are cleared/set before and reset after each individual test in the annotated class.
 ====
 
-Sometimes, you might also need to set a system property to a value that is not a constant expression, which is required for annotation values.
-In this case, you can still leverage the restore mechanism:
+== `@RestoreSystemProperties`
+`@RestoreSystemProperties` can be used to restore changes to system properties made directly in code.
+While `@ClearSystemProperty` and `@SetSystemProperty` set or clear specific properties and values, they don't allow property values to be calculated or parameterized, thus there are times you may want to directly set properties in your test code.
+`@RestoreSystemProperties` can be placed on test methods or test classes and will completely restore all system properties to their original state after a test or test class is complete.
+
+In this example, `@RestoreSystemProperties` is used on a test method, ensuring any changes made in that method are restored:
+
+[source,java,indent=0]
+----
+include::{demo}[tag=systemproperty_restore_test]
+----
+
+When `@RestoreSystemProperties` is used on a test class, any system properties changes made during the entire lifecycle of the test class, including test methods, `@BeforeAll`, `@BeforeEach` and 'after' methods, are restored after the test class' lifecycle is complete.
+In addition, the annotation is inherited by each test method just as if each one was annotated with `@RestoreSystemProperties`.
+
+In the following example, both test methods see the system property changes made in `@BeforeAll` and `@BeforeEach`, however, the test methods are isolated from each other (`isolatedTest2` does not 'see' changes made in `isolatedTest1`).
+As shown in the second example below, the class-level `@RestoreSystemProperties` ensures that system property changes made within the annotated class are completely restored after the class's lifecycle, ensuring that changes are not visible to `SomeOtherTestClass`.
+Note that `SomeOtherTestClass` uses the `@ReadsSystemProperty` annotation: This ensures that JUnit does not schedule the class to run during any test known to modify system properties (see <<Thread-Safety>>).
+
+[source,java,indent=0]
+----
+include::{demo}[tag=systemproperty_class_restore_setup]
+----
+
+Some other test class, running later:
 
 [source,java,indent=0]
 ----
-include::{demo}[tag=systemproperty_parameter]
+include::{demo}[tag=systemproperty_class_restore_isolated_class]
 ----
 
+== Using `@ClearSystemProperty`, `@SetSystemProperty`, and `@RestoreSystemProperties` together
+All three annotations can be combined, which could be used when some system properties are parameterized (i.e. need to be set in code) and others are not.
+For instance, imagine testing an image generation utility that takes configuration from system properties.
+Basic configuration can be specified using `Set` and `Clear` and the image size parameterized:
+
+[source,java,indent=0]
+----
+include::{demo}[tag=systemproperty_method_combine_all_test]
+----
+
+[NOTE]
+====
+Using `@RestoreSystemProperties` is not necessary to restore system properties modified via `@ClearSystemProperty` or `@SetSystemProperty` - they each automatically restore the referenced properties.
+'Restore', is only needed if system properties are modified in some way _other than_ Clear and Set during a test.
+====
+
+=== `@RestoreSystemProperties` Limitations
+The system `Properties` object is normally just a hashmap of strings, however, it is technically possible to store non-string values and create https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/Properties.html#%3Cinit%3E(java.util.Properties)[nested `Properties` with inherited default values].
+`@RestoreSystemProperties` restores the original `Properties` object with all of its potential richness _after_ the annotated scope is complete.
+However, for use during the test _within_ the test scope it provides a cloned `Properties` object with these limitations:
+
+- Properties with non-string values are removed
+- Nested `Properties` are flattened into a non-nested instance that has the same effective values, but not necessarily the same structure
+
 == Thread-Safety
 
 Since system properties are global state, reading and writing them during https://junit.org/junit5/docs/current/user-guide/#writing-tests-parallel-execution[parallel test execution] can lead to unpredictable results and flaky tests.
-The system property extension is prepared for that and tests annotated with `@ClearSystemProperty` or `@SetSystemProperty` will never execute in parallel (thanks to https://junit.org/junit5/docs/current/api/org.junit.jupiter.api/org/junit/jupiter/api/parallel/ResourceLock.html[resource locks]) to guarantee correct test results.
+The system property extension is prepared for that and tests annotated with `@ClearSystemProperty`, `@SetSystemProperty`, or `@RestoreSystemProperties` will never execute in parallel (thanks to https://junit.org/junit5/docs/current/api/org.junit.jupiter.api/org/junit/jupiter/api/parallel/ResourceLock.html[resource locks]) to guarantee correct test results.
 
 However, this does not cover all possible cases.
 Tested code that reads or writes system properties _independently_ of the extension can still run in parallel to it and may thus behave erratically when, for example, it unexpectedly reads a property set by the extension in another thread.
 Tests that cover code that reads or writes system properties need to be annotated with the respective annotation:
 
 * `@ReadsSystemProperty`
-* `@WritesSystemProperty`
+* `@WritesSystemProperty` (though consider using `@RestoreSystemProperties` instead)
 
-Tests annotated in this way will never execute in parallel with tests annotated with `@ClearSystemProperty` or `@SetSystemProperty`.
+Tests annotated in this way will never execute in parallel with tests annotated with `@ClearSystemProperty`, `@SetSystemProperty`, or `@RestoreSystemProperties`.

src/demo/java/org/junitpioneer/jupiter/EnvironmentVariablesExtensionDemo.java

@@ -12,9 +12,19 @@
 
 import static org.assertj.core.api.Assertions.assertThat;
 
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.ClassOrderer;
+import org.junit.jupiter.api.Nested;
+import org.junit.jupiter.api.Order;
 import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.TestClassOrder;
+import org.junit.jupiter.api.TestInstance;
 import org.junit.jupiter.api.condition.EnabledForJreRange;
 import org.junit.jupiter.api.condition.JRE;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.ValueSource;
+import org.junitpioneer.jupiter.params.IntRangeSource;
 
 @EnabledForJreRange(max = JRE.JAVA_16, disabledReason = "See: https://github.com/junit-pioneer/junit-pioneer/issues/509")
 public class EnvironmentVariablesExtensionDemo {
@@ -60,4 +70,90 @@ void clearedAtClasslevel() {
 	}
 	// end::environment_using_at_class_level[]
 
+	// tag::environment_method_restore_test[]
+	@ParameterizedTest
+	@ValueSource(strings = { "foo", "bar" })
+	@RestoreEnvironmentVariables
+	void parameterizedTest(String value) {
+		setEnvVar("some parameterized property", value);
+		setEnvVar("some other dynamic property", "my code calculates somehow");
+	}
+	// end::environment_method_restore_test[]
+
+	@Nested
+	@TestClassOrder(ClassOrderer.OrderAnnotation.class)
+	class EnvironmentVariableRestoreExample {
+
+		@Nested
+		@Order(1)
+		@TestInstance(TestInstance.Lifecycle.PER_CLASS) // Allow non-static @BeforeAll
+		// tag::environment_class_restore_setup[]
+		@RestoreEnvironmentVariables
+		class EnvironmentVarRestoreTest {
+
+			@BeforeAll
+			void beforeAll() {
+				setEnvVar("A", "A value");
+			}
+
+			@BeforeEach
+			void beforeEach() {
+				setEnvVar("B", "B value");
+			}
+
+			@Test
+			void isolatedTest1() {
+				setEnvVar("C", "C value");
+			}
+
+			@Test
+			void isolatedTest2() {
+				assertThat(System.getenv("A")).isEqualTo("A value");
+				assertThat(System.getenv("B")).isEqualTo("B value");
+
+				// Class-level @RestoreEnvironmentVariables restores "C" to original state
+				assertThat(System.getenv("C")).isNull();
+			}
+
+		}
+		// end::environment_class_restore_setup[]
+
+		@Nested
+		@Order(2)
+		// tag::environment_class_restore_isolated_class[]
+		// A test class that runs later
+		@ReadsEnvironmentVariable
+		class SomeOtherTestClass {
+
+			@Test
+			void isolatedTest() {
+				// Class-level @RestoreEnvironmentVariables restores all changes made in EnvironmentVarRestoreTest
+				assertThat(System.getenv("A")).isNull();
+				assertThat(System.getenv("B")).isNull();
+				assertThat(System.getenv("C")).isNull();
+			}
+
+			// Changes to A, B, C have been restored to their values prior to the above test
+		}
+
+		// end::environment_class_restore_isolated_class[]
+	}
+
+	// tag::environment_method_combine_all_test[]
+	@ParameterizedTest
+	@IntRangeSource(from = 0, to = 10000, step = 500)
+	@RestoreEnvironmentVariables
+	@SetEnvironmentVariable(key = "DISABLE_CACHE", value = "TRUE")
+	@ClearEnvironmentVariable(key = "COPYWRITE_OVERLAY_TEXT")
+	void imageGenerationTest(int imageSize) {
+		setEnvVar("IMAGE_SIZE", String.valueOf(imageSize)); // Requires restore
+
+		// Test your image generation utility with the current environment variables
+	}
+	// end::environment_method_combine_all_test[]
+
+	public static void setEnvVar(String name, String value) {
+		EnvironmentVariableUtils.set(name, value);
+	}
+
 }