Support overriding configuration via environment variables / additional config files

[duckfullstop]

Is your feature request related to a problem? Please describe.
A common problem encountered when running any software in a container is how to handle passing secrets to the container. At present, secrets are stored in the main configuration file, which means that committing the configuration to public repositories (or even private ones) would leak any and all secrets.

At present, thanks to @andig's e220091, there is support for overriding most configuration using the environment (e.g EVCC_MQTT.PASSWORD - but it is not possible to override things such as the vehicles entry (i.e you can't do EVCC_VEHICLES[0].PASSWORD.

Describe the solution you'd like
There's a few possible options for alleviating this:

• Allow overriding of configuration arrays via environment variables (for example, EVCC_VEHICLE_NAME, EVCC_VEHICLE_PASSWORD, or maybe something like EVCC_VEHICLE_superexpensivetesla_PASSWORD) - this would require specific lookups on each key by hand
• Allow special configuration values that causes a lookup from the environment, e.g password: ENV_VEHICLE_PASSWORD would check VEHICLE_PASSWORD
  • We can't use the evcc_ space because that's already used for Viper's configuration overloading (as e220091)
• Allow for loading multiple configuration files, e.g evcc.yaml and secrets.yaml, then merge them to produce a final in-memory configuration that can be used freely
  • e.g secrets.yaml may contain its own Vehicles configuration

All of these methods have their upsides and downsides, hence why I'm raising this as an issue for further discussion instead of going straight to PR.

Describe alternatives you've considered
Treating my entire configuration file as a secret isn't ideal, as it means I'm obscuring a large part of my infrastructure from my public repositories.

Additional context
Discussed in Slack: https://evccgroup.slack.com/archives/C01321PUJAD/p1676019600376789

[StefanSchoof]

Related #405

[andig]

there is support for overriding most configuration using the environment

is that really true? That would already be great, but I didnt test it yet. Does that really work when reading from the unmarshaled config struct or does it require using the viper functions for reading?

[duckfullstop]

@andig Yup, try setting EVCC_SITE.TITLE in your environment and it'll override whatever's in your config (at runtime)!

[andig]

Wow, that's better then I thought. Seems I can remove any calls to viper.Get... and replace them with accessing the global config like everywhere else. That is really good news as it makes the code more consistent. Always wanted to experiment but never got around to try it.

For this issue the solution should be within Viper, I'd suggest to add an issue there:

EVCC_VEHICLE_0_PASSWORD

should work for things that are slices.

[StefanSchoof]

There are some issues on viper that sound similar: spf13/viper#761 spf13/viper#935

[andig]

@andig Yup, try setting EVCC_SITE.TITLE in your environment and it'll override whatever's in your config (at runtime)!

@duckfullstop OT, but

EVCC_DATABASE_DSN=/tmp/foo.db3 evcc

doesn't work for me.

Update solution:

viper.SetEnvKeyReplacer(strings.NewReplacer(".", "_"))

[duckfullstop]

@andig Yup, try setting EVCC_SITE.TITLE in your environment and it'll override whatever's in your config (at runtime)!

@duckfullstop OT, but

EVCC_DATABASE_DSN=/tmp/foo.db3 evcc

doesn't work for me.

Did you try EVCC_DATABASE.DSN? For whatever reason viper wants periods in the middle of environment variables

[andig]

See #6286. . doesn't work art least on mac.

[andig]

Also not recent comments like #9363 (reply in thread)

[StefanSchoof]

Es wurde gerade spf13/viper#1429 gemergt. Möglicherweise lässt sich das Problem jetzt lösen.

[andig]

Lustig- die Notification hab ich auch bekommen. Steht auf meiner Todoliste :)