Support using environment variables in configuration

[J7mbo]

Hi. #569 led me to creating a new task to discuss this.

Authelia's use of a configuration file currently requires all configuration to be hardcoded. Examples can be the domain under access_control, the user under authentication_backend.ldap etc.

Are there any reservations for a PR using the template package or similar to utilise environment variables in authelia configuration? Preferably without any unnecessary prefixes or suffixes like terraform does (these require users to use extra scripts just to use vendor-specific environment variables).

ldap:
        url: {{ LDAP_URL }}

No complexity with defaults - if the environment variable exists then place it otherwise it's an empty string.

[james-d-elliott]

We're actively discussing this. There are a few factors which affect this.. we're looking towards a management UI which would introduce in database configuration. Additionally we are about to deprecate our current use of env variables to replace values in config. I think though based on your description it sounds like you want to add it so you can replace the contents of the YAML prior to reading it into a struct?

I think we'd consider this (I can't speak for @clems4ever or @nightah so lets wait for them to comment) provided it had a CLI argument associated with it.

I think a good design is to have an argument like --templates="/var/folder/file1.yaml,/var/folder/file2.yaml" - an array of files to replace using the template package prior to reading configuration. We could theoretically utilize this with a database preload in the future with little to no adjustment.

Alternatively maybe --templates=/var/folder --templates-out=/var/folder2. Where --templates is a directory of *.tpl files, and --templates-out is the same file names with tpl removed (defaulting to either the /etc/authelia/ dir, or the same folder as the templates?

[J7mbo]

If there were CLI arguments as an options, I would just use --authentication_backend.ldap.url=$LDAP_URL in docker-compose command:. Were this to exist for every configurable option, this would negate the need for a YAML file being copied over and mounted to be used in the first place. But I think that's a different kettle of fish...

When parsing the YAML file to convert it into a configuration struct, for each option, if the option is in the format {{ envVarName }}, then instead of using the string use os.GetEnv() instead.

Looking at: reader.go, after this, my implementation for a PR would be: loop through all properties of the config struct (they're all public), if any {{ someString }} are found with, say, a regex, pull out the someString, do an os.GetEnv() lookup with a default value of an empty string (and in the default case, a warning log), and replace the value in the config struct with the value from the environment.

For docker-compose/docker this would work fine with the environment: key or -e in docker run which passes the env vars to the containers.

[clems4ever]

Hello @J7mbo , there is no reservation on community PRs, they are actually very welcome and you did the right thing by coming and discuss the design first. I'd be interested to know the use case first though. That's important to decide whether we would accept that change because as mentioned by @james-d-elliott , this change can conflict with the migration of the configuration values in database if we decide to do so eventually. It's still under discussion because I'm wondering if doing so wouldn't make setup automation harder.

Also please note that with viper, setting those config variables with env vars would be trivial if spf13/viper#761 was resolved. That would be the proper solution in order to delegate the config-specific complexity to the library instead of Authelia. In the meantime and for the reasons mentioned above, I'd like to hear more about the actual use case.

Can you elaborate on your situation right now?

[J7mbo]

@clems4ever Sure, happy to discuss the use-case.

As a user, I don't want to hardcode my domain, password or other configuration options such as the port, sensitive or otherwise, in configuration.yml. I use $DOMAIN when spinning up many services and currently have to hardcode the domain in authentication_backend.ldap.base_dn, authentication_backend.ldap.user, authentication_backend.ldap.password, totp.issuer etc.

All my environment variables are stored in a .env file which I read out and pass during deployment. Authelia can't handle this right now so my alternative is to place the placeholders in and script an on-the-fly replacement.

Perhaps irrelevant, but other tools that work well alongside authelia such as traefik allow the passing in of configuration options on the command line, which by proxy enables environment variables to be used and it's a popular feature for 100% setup automation which is my goal. OpenLDAP has this feature. Elasticsearch and Kibana too. Seems to be quite common to allow for dynamic setups.

Configuration values in a database (with the UI?) sounds great. I haven't looked at the PR and discussion for it, how having this addition would conflict with it? If configuration values were specified in a database, they will be used, with environment variables taking precedence if present, or the other way around.

[jlcummings]

For reference:
Traefik Go Templating and Sprig Functions
Sprig function documentation
Partial implementation source reference:
traefik/pkg/provider/file/file.go
traefik/pkg/provider/configuration.go

[J7mbo]

The main question here is - can a user set up and deploy their project with authelia, with their desired configuration, fully automatically in a script? At that time I could not.

Bit of an old one now - May 2020, almost a whole year ago. Is there still interest here?

[RoboMagus]

I for one would definitely be interested in such an addition to the configuration options.
People keep looking for workarounds now, but having the option to provide e.g. a domain name through environment variables, similar to the secrets, would be so much easier for scripted deployment!

[james-d-elliott]

I am working on a PR at present that supports this. Basically it's finished and just needs some tweaking and peer review.

[james-d-elliott]

@RoboMagus @jlcummings @J7mbo @SeraphimSerapis @noizwaves @martinl88 @tomachristian @josephcomisi @mlavi @snesbittsea @Chuckame @dartunian @m1kl

This is implemented in 4.30.0. The two things that don't currently work with it are access_control.rules, and identity_providers.oidc.clients.

I will aim to spend some time on getting that working in a couple weeks.

[ToshY]

@james-d-elliott Could you give an update on the possible progress you've made regarding access_control.rules? I also really don't want to hardcode my domains in the rules, so setting those through environment variables would be a really nice thing to have.

[james-d-elliott]

Oops I forgot to report back here. So I did a lot of work trying to get this to work, it seems quite difficult to do right. The issue is order, we either need to introduce a priority field or something similar. I plan to do some more work on configuration after getting a few major things done, I'll add this to my list of things to attempt to resolve.

The workaround to this would be to have a separate file that's meant to be used with envsubst or similar to configure it for just ACL's.

[ToshY]

Well in the meantime I ended up doing the following as a workaround, where in the Dockerfile I copy the initial configuration/user files, replace the placeholders and specify that the new config should be used instead.

docker-compose.yml

authelia:
  build:
    context: docker/authelia
    dockerfile: Dockerfile
    args:
      AUTHELIA_VERSION: ${AUTHELIA_VERSION}
      APP_DOMAIN: ${APP_DOMAIN}
      USER_NAME: ${AUTHELIA_ADMIN_NAME}
      USER_EMAIL: ${AUTHELIA_ADMIN_EMAIL}
      USER_DISPLAY_NAME: ${AUTHELIA_ADMIN_DISPLAY_NAME}
      USER_PASSWORD_HASH: ${AUTHELIA_ADMIN_PASSWORD_HASH}
  container_name: authelia

_{Note: AUTHELIA_ADMIN_PASSWORD_HASH should be surrounded with single quotes in your dotenv file for this to work}

docker/authelia/Dockerfile

ARG AUTHELIA_VERSION

FROM authelia/authelia:${AUTHELIA_VERSION}

ARG APP_DOMAIN
ARG USER_NAME
ARG USER_DISPLAY_NAME
ARG USER_PASSWORD_HASH
ARG USER_EMAIL

COPY ./config/configuration.yml /config/configuration.yml
COPY ./config/users.yml /config/users.yml

RUN sed "s:\$APP_DOMAIN:${APP_DOMAIN}:g" /config/configuration.yml > /app/configuration.yml
RUN sed -e "s:\$USER_NAME:${USER_NAME}:g" -e "s:\$USER_DISPLAY_NAME:${USER_DISPLAY_NAME}:g" -e "s:\$USER_PASSWORD_HASH:${USER_PASSWORD_HASH}:g" -e "s:\$USER_EMAIL:${USER_EMAIL}:g" /config/users.yml > /app/users.yml

ENTRYPOINT ["/app/entrypoint.sh"]
CMD ["--config", "/app/configuration.yml"]

docker/authelia/config/configuration.yml

...
authentication_backend:
  file:
    path: /app/users.yml

access_control:
  default_policy: deny
  rules:
    - domain:
        - "*.$APP_DOMAIN"
      policy: two_factor
...

docker/authelia/config/users.yml

users:
    $USER_NAME:
        displayname: $USER_DISPLAY_NAME
        password: $USER_PASSWORD_HASH
        email: $USER_EMAIL
        groups:
            - admins

[m4heshd]

It would be amazing if the configuration could handle a .env file just like docker-compose does. Hardcoding domain names, encryption keys, passwords.. etc is really an inconvenience.

[james-d-elliott]

The configuration does not require hard coding of these values at all. They can be configured via secrets or via environment variables both of which can be utilized in docker-compose and .env.

We're unlikely to implement configuration via .env especially without a compelling reason / use case, especially when it's likely to be complex to implement well.

[cellulosa]

how about setting up secrets that are nested under an array-like structure? Eg.

identity_providers:
  oidc:
    jwks:
      - key:

    clients:
      - client_id:

[james-d-elliott]

If it's an array, then it can only be done via the templating system.

[cellulosa]

So I guess something like this?

services:
  authelia:
    image: docker.io/authelia/authelia:latest
    volumes:
      - my_private_key_file:my_private_key_file
    environment:
      - X_AUTHELIA_CONFIG_FILTERS=expand-env,template
      - CLIENT_SECRET_ID=mysecretid

and then editing config.yml:

identity_providers:
  oidc:
    jwks:
      - key: |
        {{- fileContent '/my_private_key_file' | nindent 8 }}

    clients:
      - client_id: '{{ env "CLIENT_SECRET_ID" }}'

[james-d-elliott]

Not expand-env,template, just template. Then yea you can use the env like that. See the following resources for how you'd do multi line secrets as well as debugging these:

• Configuration > Prologue > Security Sensitive Values
• Reference > Guides > Templating
• Blog > 4.38 Release Notes

[cellulosa]

Heya I looked at the linked resources but still can't make the templating work - can I pass env variables or do I need to use secrets?

docker-compose.yml:

services:
  authelia:
    image: docker.io/authelia/authelia:latest
    environment:
      - X_AUTHELIA_CONFIG_FILTERS=template
      - MY_DOMAIN=website.com

and /config/configuration.yml:

session:
  cookies:
    - domain: '{{ env "MY_DOMAIN" }}'
      authelia_url: 'https://auth.{{ env "MY_DOMAIN" }}'
      default_redirection_url: 'https://www.{{ env "MY_DOMAIN" }}'

If I run docker-compose exec authelia authelia config template --config.experimental.filters template it returns:

---
##
## Authelia rendered configuration file (file filters).
##
## Filters: template
##

---
##
## File Source Path: /config/configuration.yml
##

session:
  cookies:
    - domain: ''
      authelia_url: 'https://auth.'
      default_redirection_url: 'https://www.'

[james-d-elliott]

Seems like you may want to open a discussion to troubleshoot this, seems to work for me:

services:
  authelia:
    image: docker.io/authelia/authelia:4.38.7
    environment:
      - X_AUTHELIA_CONFIG_FILTERS=template
      - MY_DOMAIN=website.com
    volumes:
      - ./configuration.yml:/config/configuration.yml

session:
  cookies:
    - domain: '{{ env "MY_DOMAIN" }}'
      authelia_url: 'https://auth.{{ env "MY_DOMAIN" }}'
      default_redirection_url: 'https://www.{{ env "MY_DOMAIN" }}'

$ docker compose run authelia authelia config template

---
##
## Authelia rendered configuration file (file filters).
##
## Filters: template
##

---
##
## File Source Path: /config/configuration.yml
##

session:
  cookies:
    - domain: 'website.com'
      authelia_url: 'https://auth.website.com'
      default_redirection_url: 'https://www.website.com'

[cellulosa]

done, thanks! #7059