best practices to set env variables

[aubertaa]

Hi,
i've not found this in the documentation and i wonder what the best practices are when needing to define env var for testing. Here is an example:

    Describe 'Call MyMethodReadingEnvVar with env variable MY_VAR = "my_value"'
      MY_VAR="my_value"
      It 'should output its ok'
        When call MyMethodReadingEnvVar
        The output should equal 'its ok !'
        The status should be success
      End
    End

    Describe 'Call MyMethodReadingEnvVar without env variable MY_VAR'
      It 'should output missing env var'
        When call MyMethodReadingEnvVar
        The output should equal 'MY_VAR is missing in environment !'
        The status should be success
      End
    End

• Is this syntax correct, with setting the MY_VAR at 'Describe level' ?

  • Will this var well be destroyed when first test ends ?
  • Is this compliant with parallelizing tests runs (using 'call' doesn't pop a new shell, what about this MY_VAR in env ?) ?

• Is there a better way to scope MY_VAR assignment to one test ?

I've read this discussion but it's not crystal clear for me if it's required to use BeforeCall and AfterCall each time we want to do that or if simplier syntax (as in my example above) is also OK.

Thanks for clarifications on that and maybe adding this to documentation could be nice :) !

[LukeSavefrogs]

TL;DR;

It depends on how you want to handle your tests.

Personal preference

Personally, i like to follow the following rules:

• If it's an environment variable used throughout the whole test (i.e. the path to a specific file) i place it at global scope (outside of every block);
• If it's an environment variable that needs to be reset before every test (i.e. a counter of some sorts) i place it in the BeforeAll or BeforeEach hooks;
• If it's going to be used only for a specific test (i.e. a DEBUG to test the verbosity) i place it in the Describe or It blocks;

But as i've said before, it really depends on what you're trying to make.

Demonstration

In the following example i have taken into consideration 4 major scopes:

• Describe
• It
• BeforeEach
• BeforeAll

Notice how we can overwrite the variable in the inner scope and the result changes accordingly (see the > Unsetting: and > Overwriting: tests)

#!/bin/env bash

function MyMethodReadingEnvVar () {
	if [[ "${MY_VAR}" ]]; then
		echo "its ok. Variable is '${MY_VAR}'!";
		return;
	fi
	echo "MY_VAR is missing in environment!";
}

# Environment variable defined at `Describe` level
Describe "ENV at 'describe' level"
	MY_VAR="describe"

	It "> should output its ok"
		When call MyMethodReadingEnvVar
		The output should equal "its ok. Variable is 'describe'!"
		The status should be success
	End

	It "> Unsetting: should output missing env var"
		unset MY_VAR
		When call MyMethodReadingEnvVar
		The output should equal 'MY_VAR is missing in environment!'
		The status should be success
	End
End

# Environment variable defined at `Test` level
Describe "ENV at 'test' level"
	It "> should output its ok"
		MY_VAR="test"
		When call MyMethodReadingEnvVar
		The output should equal "its ok. Variable is 'test'!"
		The status should be success
	End

	It "> should output missing env var"
		When call MyMethodReadingEnvVar
		The output should equal 'MY_VAR is missing in environment!'
		The status should be success
	End
End

# Environment variable defined at `BeforeEach` level
Describe "ENV at 'hook' (BeforeEach) level"
	BeforeEach 'MY_VAR="BeforeEach hook"'

	It "> should output its ok"
		When call MyMethodReadingEnvVar
		The output should equal "its ok. Variable is 'BeforeEach hook'!"
		The status should be success
	End

	It "> Overwriting: should output its ok"
		MY_VAR="BeforeEach Overwritten"
		When call MyMethodReadingEnvVar
		The output should equal "its ok. Variable is 'BeforeEach Overwritten'!"
		The status should be success
	End
End

# Environment variable defined at `BeforeAll` level
Describe "ENV at 'hook' (BeforeEach) level"
	BeforeAll 'MY_VAR="BeforeAll hook"'

	It "> should output its ok"
		When call MyMethodReadingEnvVar
		The output should equal "its ok. Variable is 'BeforeAll hook'!"
		The status should be success
	End

	It "> Overwriting: should output its ok"
		MY_VAR="BeforeAll Overwritten"
		When call MyMethodReadingEnvVar
		The output should equal "its ok. Variable is 'BeforeAll Overwritten'!"
		The status should be success
	End
End

This gives us the following output:

Feel free to respond, as i'll be happy to help 😄

[aubertaa]

hi,
Thanks for your prompt and detailed answer !

Are your 2 examples perfectly equivalent or is there any difference (var kept after test, impacting parallel run or anything else) ?

• Environment variable defined at Describe level
• Environment variable defined at BeforeAll level

What do you think about this one ? It is strange to me that both tests are successful whereas MY_VAR is set twice at Describe level, is this a bug ?

# Environment variable defined TWICE at `Describe` level
Describe "ENV at 'describe' level"

	MY_VAR="describe"
	It "> should output its ok"
		When call MyMethodReadingEnvVar
		The output should equal "its ok. Variable is 'describe'!"
		The status should be success
	End


	MY_VAR="new_value"
	It "> should output new_value"
		When call MyMethodReadingEnvVar
		The output should equal "its ok. Variable is 'new_value'!"
		The status should be success
	End

End

[LukeSavefrogs]

Hi @aubertaa,
i'll try to answer as best as i can 😄

What do you think about this one ? It is strange to me that both tests are successful whereas MY_VAR is set twice at Describe level, is this a bug ?

I don't know 😃

Your example is indeed interesting...
As you said in the output both the tests are successful, which is something i wouldn't have expected, given how other test frameworks work (personally i use Jest in my projects)...

#!/bin/env bash

function MyMethodReadingEnvVar () {
	if [[ "${MY_VAR}" ]]; then
		echo "its ok. Variable is '${MY_VAR}'!";
		return;
	fi
	echo "MY_VAR is missing in environment!";
}

# Environment variable defined TWICE at `Describe` level
Describe "ENV at 'describe' level"
	MY_VAR="describe"
	It "> should output its ok"
		When call MyMethodReadingEnvVar
		The output should equal "its ok. Variable is 'describe'!"
		The status should be success
	End

	MY_VAR="new_value"
	It "> should output new_value"
		When call MyMethodReadingEnvVar
		The output should equal "its ok. Variable is 'new_value'!"
		The status should be success
	End
End

Output:

Normally i would expect the Describe block to be evaluated first all at once (leading to the MY_VAR variable having the value new_value) before the It blocks get executed, however this situation gets me thinking... 🤔

With a bit of time it would be interesting to see how this is handled in the source code.

UPDATE

ALWAYS PREFERE HOOKS over instructions directly in the Describe block.
The following snippet will fail because it won't find the MY_VAR variable, just because we declared it after the It block!

# Environment variable defined TWICE at `Describe` level
Describe "ENV at 'describe' level"
	It "> should output its ok"
		When call MyMethodReadingEnvVar
		The output should equal "its ok. Variable is 'describe'!"
		The status should be success
	End

	MY_VAR="describe"

	It "> should output new_value"
		MY_VAR="new_value"
		When call MyMethodReadingEnvVar
		The output should equal "its ok. Variable is 'new_value'!"
		The status should be success
	End
End

Output:

Using the BeforeAll instead tells us there is a Syntax error and requires us to place the BeforeAll hook before every test definition (probably for the same reason we got the strange output before):

Describe "ENV at 'describe' level"
	It "> should output its ok"
		When call MyMethodReadingEnvVar
		The output should equal "its ok. Variable is 'describe'!"
		The status should be success
	End

	BeforeAll 'MY_VAR="describe"'

	It "> should output new_value"
		MY_VAR="new_value"
		When call MyMethodReadingEnvVar
		The output should equal "its ok. Variable is 'new_value'!"
		The status should be success
	End
End

---

Are your 2 examples perfectly equivalent or is there any difference (var kept after test, impacting parallel run or anything else) ?

Yes, they are equivalent, but only as long as you use them like they should.

I'll explain myself...

• You should place the environment variables in the Before[...] blocks whenever you want to ensure that your tests run under a consistent environment (wether per-test with BeforeEach or per example group_*_ using BeforeAll)
• As said before, you should avoid doing too many things directly in the Describe block and if you need to, place them right under the declaration and not scattered around. This will help keeping your tests clean and avoiding headaches like in your example.

* Test/Example group is the actual name of the Describe block.

Neither of them should have an impact on parallel run, since the parallelism is used only on files (this is what I could understand, but i could be wrong).

Since you're setting the variables before the It block, i suspect you need them only there, so the snippet you've provided could be then rewritten like this:

Describe "ENV at 'describe' level"
	It "> should output its ok"
		MY_VAR="describe"
		When call MyMethodReadingEnvVar
		The output should equal "its ok. Variable is 'describe'!"
		The status should be success
	End


	It "> should output new_value"
		MY_VAR="new_value"
		When call MyMethodReadingEnvVar
		The output should equal "its ok. Variable is 'new_value'!"
		The status should be success
	End
End

This way, the code will be more readable and most importantly does not rely on something we don't know if it's an actual feature.

[aubertaa]

Thanks, it's really clear with this answer !
And it's also confirmed that behavior of having both tests successful in my example is not consistent with what UT frameworks usualy do (jest, mocha....) but still, it is not recommended to write vars at describe scope this way !
Regards,