Override env var

[John0x]

Hey, I did expect env_files to behave like typical dotnet files, which surprisingly isn't the case. Is that intended?
My env_file has a variable "CONNECTION_STRING", which should default to the value set in the env_file but be overwritten, if it's defined as an environment variable.
It's not being overwritten tho, is it a bug?

Usecase:
Having default values defined in the env_file that will be overwritten in the docker machine by global environment variables

[sagiegurari]

env files do override existing vars.
can you give me a simole test case to try it out?

[sagiegurari]

@John0x I created the following env file:

CONNECTION_STRING=envfile

and makefile:

env_files = [
    "./myenv.env"
]

[tasks.envtest]
command = "echo"
args = ["connection string: ${CONNECTION_STRING}"]

ran the following command: CONNECTION_STRING=CLI cargo make envtest
and the output was:

CONNECTION_STRING=cli cargo make envtest
[cargo-make] INFO - cargo make 0.35.10
[cargo-make] INFO - Build File: Makefile.toml
[cargo-make] INFO - Task: envtest
[cargo-make] INFO - Profile: development
[cargo-make] INFO - Running Task: legacy-migration
[cargo-make] INFO - Execute Command: "echo" "connection string: envfile"
connection string: envfile
[cargo-make] INFO - Build Done in 0.21 seconds.

which shows that env file overridden the current env value just fine.
so this seems ok to me unless you have a test case that shows otherwise.

[John0x]

The environment variable isn't being properly propagated to the applications that are run.

envfile:

CONNECTION_STRING=envfile

makefile:

env_files = ["./.env"]

[env]
CARGO_MAKE_EXTEND_WORKSPACE_MAKEFILE = true

[tasks.testo]
script_runner = "@shell"
script = '''
echo connection string: $CONNECTION_STRING
'''

Being invoked with:
cross-env CONNECTION_STRING=fromcli cargo make testo
(same behavior without cross-env)

Output:

connection string: envfile

[sagiegurari]

'envfile' is what you defined in your env file so... isn't that the right value?
i see you have a workspace, maybe thats the issue, can you setup a test case for me to check?

[John0x]

Nope. Output should be "connection string: fromcli".
At least that's what I would expect from an .env file: https://www.npmjs.com/package/dotenv
Expected behavior: Environment Variable takes precedence over .env file

I'll check if the workspace is the problem

[sagiegurari]

ok now i got you. no, thats not how it works at the moment.
this is the logic: https://github.com/sagiegurari/cargo-make#usage-env-vars-loading-order
cargo-make overrides whats already defined. if you want to provide via cli you should use the -e cli param.
for example: cross-env cargo make -e CONNECTION_STRING=fromcli testo

[John0x]

ah, okay. But isn't that the whole purpose of dotenv files?
Declaring sane default that can then be overwritten by the environment that it's being run on.

For example by .env file will contain the connection string that is being used for local development (the default), which will then be overwritten by the docker image for production.

[sagiegurari]

i only used it as an alternative of defining everything in the makefile.toml and not as dotenv alternative.
basically a simple ability to also have different prod/dev env files and load the right one.
to have the ability to use different values for dev and production i have profiles: https://github.com/sagiegurari/cargo-make#usage-profiles
with profiles you can load different env files, env blocks and so on...
also if you define env vars inside the makefile.toml you can default to already defined values and if not set the value, for example:

PREFER_EXISTING = { value = "new", condition = { env_not_set = ["PREFER_EXISTING"] } }
OVERWRITE_EXISTING = { value = "new", condition = { env_set = ["OVERWRITE_EXISTING"] } }

[John0x]

Okay, thanks for the clarification. But I think it would be better to not name them .env files then. Dotenv isn't really a standard but it's pretty established and therefore can cause a lot of confusion if it doesn't behave the way you'd expect .env files to behave.

Or make them behave like typical dotenv files, which I think would be better

[sagiegurari]

at this point it would be a big backward break, so not sure i am for that change. also the original purpose was different and that's why i documented the order specifically.
i do understand the confusion.

[sagiegurari]

closing this one for now

[John0x]

@sagiegurari is there a way to use dotenv with cargo-make?
Otherwise, I'll have to find a new task runner :/

Just, for example, supports them. But I would really prefere to use cargo-make

[sagiegurari]

what do you mean exactly? to have a file that only contains 'defaults'?
you can do it via duckscript in env_script attribute. something like this (not tested):

env_scripts = [
'''
#!@duckscript
text = readfile ./test.env
handle = map
map_load_properties ${handle} ${text}
for key in ${handle}
    value = map_get ${handle} ${key}
    value_not_exists = is_empty ${value}
    if ${value_not_exists}
        set_env ${key} ${value}
    end
end
'''
]

[John0x]

Thank you @sagiegurari
But that doesn't seem to propagate the set env vars to the applications being run, does it?

They seem to work for commands that are directly in the Makefile but the program that is executed by the task isn't picking it up

[sagiegurari]

can you share an example?

[sagiegurari]

@John0x like i wrote in the example, its not tested.
i checked it and there was a bug there.
this should work

env_scripts = [
'''
#!@duckscript
text = readfile ./myenv.env
handle = map
map_load_properties ${handle} ${text}
keys = map_keys ${handle}
for key in ${keys}
    value = map_get ${handle} ${key}
    current_value = get_env ${key}
    value_not_exists = is_empty ${current_value}
    if ${value_not_exists}
        set_env ${key} ${value}
    end
end
'''
]

[sagiegurari]

apart of the script i gave you which should solve your issue, i could also add an attribute to enable that.
so it would be something like:

env_files = [
    { path = "./load_if_not_defined.env", defaults_only = true },
    { path = "./load_all.env" }
]

[sagiegurari]

ok, decided to go for it.
its as i wrote:

env_files = [
    { path = "./load_only_undefined.env", defaults_only = true },
    { path = "./load_all.env" }
]

you can see the docs at:
https://github.com/sagiegurari/cargo-make/tree/0.35.11#env-file

[sagiegurari]

@John0x this is now officially released.

[sagiegurari]

@John0x can you share if this resolved your issue?

[John0x]

Thank you very much :)
I'll try it tomorrow. Not in the office right now

[John0x]

Seems to be working fine
Thank you for your work :)

[sagiegurari]

thanks for checking