NixOS modules: Secrets provided in arguments are exposed to unprivileged users

[TLATER]

Describe the bug

When specifying secrets via arguments, they will be exposed via /proc/<pid>/cmdline - this is also possible if passed indirectly via an environment variable, since the shell resolves the value. This can be trivially demonstrated like so:

tlater ~ $ export SAMPLE_SECRET=20
tlater ~ $ sleep $SAMPLE_SECRET & ps -u | grep sleep
[1] 8181
tlater      8181  0.0  0.0 227256  1336 pts/2    SN   21:54   0:00 sleep 20

Some modules seem to use shell scripts to feed values provided via passwordFile or similar to their applications at runtime (with constructs like SECRETKEY="$(head -n1 ${secretKey})"). Often, these are then fed to sed to replace values in template files, or passed to --password arguments (which are often, but not always, arguably an upstream problem).

I'm not sure how widespread exactly this is, some way to search for occurrences of this would be nice. It's difficult because not all secrets are named passwordFile and there would be a lot of false positives if we simply pinged all modules that have such options. It's also hard to tell when this needs an upstream fix, but at least I'd argue passwordFile args should not be created for modules that cannot keep the secrets from being world-readable outside the nix store either.

I'd like to point out replace-secret, which essentially exists to do this without leaking secrets.

For now, I believe at least these modules to do such things in 21.11 - I did not check if upstream have a better way of supplying secrets where applicable:

• nixos/modules/services/misc/gitea.nix (@srhb @Ma27) - nixos/gitea: Prevent secrets from being exposed at ExecStart time #156401
• nixos/modules/services/web-servers/ttyd.nix (@nagy @atopuzov) - Allow specifying HTTP basic auth credentials from a file tsl0922/ttyd#872
• nixos/modules/services/networking/hans.nix
• nixos/modules/services/networking/ddclient.nix (@felschr) - nixos/ddclient: don't leak password in process listings #181197
• nixos/modules/services/web-apps/photoprism.nix Config: Read secrets from file photoprism/photoprism#2302

There are also some modules that use bash heredocs and printf (notably, nixos/modules/config/ldap.nix), which I believe are safe, but could probably do with some comments to point this out.

I guess it will be fairly common though, since the requirement of passing secrets via files is somewhat more pronounced in NixOS, not always considered by upstreams, and leaking secrets via ps is an easy thing to overlook. Maybe we should consider some kind of support in lib for supplying secrets to applications to avoid things like this (and get more use out of systemd, which has some nice secret-passing functionality).

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-should-one-report-small-vulnerabilities-in-individual-modules/17279/5

[TLATER]

Ah, sorry @jtojnar, I must have misread that, infinoted is clearly using a heredoc as well.

[atopuzov]

Hi @TLATER i took a look at ttyd [1] and it does not provide a way of using a file for the password.

[1] https://github.com/tsl0922/ttyd

[nixinator]

Yeah, it certainly not the greatest idea to hot wire processes with secrets injected into Environment Variables.

However, a work around would be to limit normal users from actually seeing other users processes at all.

https://www.cyberciti.biz/faq/linux-hide-processes-from-other-users/

Unfortunately You'll find many upstream projects think it's okay to pass sensitive stuff on the command line.

[RaitoBezarius]

Another way to perform this operation securely is to use LoadCredential from systemd, see systemd/systemd#15778

[TLATER]

@nixinator yes, those are good points, and partially why I hesitated raising this. Still, most users run their systems without any hardening (hard to know what might be a good idea), and a lot of projects are sympathetic to this problem.

I'm more concerned that NixOS modules are lulling users into a false sense of security through the existence of passwordFile options, which are explicitly claimed not to make passwords world-readable via the nix store; It's easy to miss that this doesn't necessarily apply to everything. We're also actively breaking these protections for projects that do have them, like in the gitea case.

The latter should clearly be fixed, and the former probably warrants further discussion. At least this problem should be more common knowledge in the community than it currently appears to be.

[Pacman99]

Is this an issue when using envusbst too? Like would the following be insecure:

export SECRET=pass
envsubst -i /nix/store/... -o /var/lib/app/config.yaml

Does the act of just setting the secret variable reveal what it is? Or is only revealed when used in command line?

[TLATER]

Does the act of just setting the secret variable reveal what it is?

No, since export doesn't launch a separate process - variables are only leaked if they are passed as arguments to an actual application executed by the kernel, not when bash modifies its own model of what the environment contains. This is also why printf and heredocs are usually fine - these are all built-in bash functionality that look a little suspect, but they do not actually launch separate processes, and therefore do not reveal their arguments via the kernel.

Note that this isn't true for all shells. busybox' ash for example implements printf as a separate binary. export specifically can probably not be implemented as an external binary though, so that one should always be safe.

This does also mean it's not a shell-exclusive issue though, secrets are also revealed when passed as variables to os.execv or subprocess.call in python, or similar library functions in other languages.

For reference, when exporting secrets in an interactive shell (i.e., by typing it manually into a terminal), those are logged to the user's bash history, which might also not be intended. This isn't an issue for the non-interactive shells this issue is about, though.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/agenix-encrypted-plaintext-passwords-and-builtins-readfile/18425/3

[bjornfor]

Closed by accident, reopening now.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/set-password-for-a-postgresql-user-from-a-file-agenix/41377/5