docker-credential-helpers support for cri-o

[neverd1m]

What happened?

There is a cri-o parameter global_auth_file that can keep the credentials for private registries. It follows the dockerconfigjson format and apparently inherits all the configuration the docker login support. I want to store the credentials on a node but not in a base64 format but with credential storage, the pass for example.

What did you expect to happen?

When i create a file with such config {"auths": {"some.registry:5000":{}},"credsStore":"pass"} nothing really happens. However with docker client this configuration works.

How can we reproduce it (as minimally and precisely as possible)?

1. I pointed the file containing the creds for registry in /etc/crio/crio.conf
global_auth_file="/etc/crio/config.json"
2. I created a single credential entry with pass
3. I mentioned the name of the storage with credsStore variable in /etc/crio/config.json
{"auths": {"some.registry:5000":{}},"credsStore":"pass"}
4. I restarted the crio and tried to pull an image

Anything else we need to know?

No response

CRI-O and Kubernetes version

crio version 1.29.2
Version:        1.29.2
GitCommit:      d317b5dc918bbfbc78481072a0d93e572aa8d0e8
GitCommitDate:  2024-02-22T19:23:38Z
GitTreeState:   clean
BuildDate:      1970-01-01T00:00:00Z
GoVersion:      go1.21.1
Compiler:       gc
Platform:       linux/amd64
Linkmode:       static
BuildTags:
  static
  netgo
  osusergo
  exclude_graphdriver_btrfs
  exclude_graphdriver_devicemapper
  seccomp
  apparmor
  selinux
LDFlags:          unknown
SeccompEnabled:   true
AppArmorEnabled:  true

OS version

# On Linux:
$ cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.6 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.6 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
$ uname -a
Linux inst-dbtmz-inst-pool-general-new-minimus 5.15.0-1049-oracle #55~20.04.1-Ubuntu SMP Wed Nov 22 15:15:07 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Additional environment details (AWS, VirtualBox, physical, etc.)

[saschagrunert]

@neverd1m thank you for the report. I don't think we stupport credsStore, but Credential Helpers (credHelpers) should work. Do you mind giving them a try?

[neverd1m]

thanks for the reply!

I tried with credHelpers as well but i get this error now:

level=info msg="Trying to access \"some.registry:5000/helpers/whoami:1.5.0\"" file="docker/docker_image_src.go:87"
level=debug msg="Looking up in credential helper pass based on credHelpers entry in /etc/crio/config.json" file="config/config.go:807"
level=debug msg="Not logged in to some.registry:5000 with credential helper pass" file="config/config.go:754"
level=debug msg="No credentials for some.registry:5000/helpers/whoami found" file="config/config.go:272"

• I tried with docker client to compare and docker login is successful with this config.
  {"auths": {"some.registry:5000":{}},"credHelpers": {"some.registry:5000":"pass"}}

• pass show docker-credential-helpers/{BASE64}/user output is correct and there are no passphrase

[saschagrunert]

@neverd1m the logs at least indicate that the credential helper is being used. Is the user (root) running CRI-O able to see the same password using pass show? I feel we're using different storage paths here.

[neverd1m]

@saschagrunert i suspect the route being the issue as well, but all the commands are executed under the root user. pass show command has correct output for it when run from command line. There are no other logs on cri-o on this step so i'm about to trace the cri-o process to see how it calls the credential helper. Any advice how to proceed with that?

[bitoku]

/assign

[bitoku]

@neverd1m
I can use credHelpers in cri-o.
When I first tried, I failed because I used a different storage path.
After I created a password storage on the right path, it worked.

cri-o just runs docker-credential-pass command as the same user who runs cri-o and checks the credential.

cri-o/vendor/github.com/containers/image/v5/pkg/docker/config/config.go

Lines 748 to 758 in 5f4f3e9

	func getCredsFromCredHelper(credHelper, registry string) (types.DockerAuthConfig, error) {
	helperName := fmt.Sprintf("docker-credential-%s", credHelper)
	p := helperclient.NewShellProgramFunc(helperName)
	creds, err := helperclient.Get(p, registry)
	if err != nil {
	if credentials.IsErrCredentialsNotFoundMessage(err.Error()) {
	logrus.Debugf("Not logged in to %s with credential helper %s", registry, credHelper)
	err = nil
	}
	return types.DockerAuthConfig{}, err
	}

So you might want to check with your console if echo "some.registry:5000" | docker-credential-pass get returns the correct password.

If you run cri-o as root, /root/.password-store should be used as a password storage.
It's also good to check if pass command uses that directory.

Not logged in ... log appears when docker-credential-pass returns credentials not found in native keychain error.
If you want to know when it returns the error, you should see docker-credential-pass code.
https://github.com/docker/docker-credential-helpers/blob/master/pass/pass.go#L149

[github-actions]

A friendly reminder that this issue had no activity for 30 days.