feat: support for reading auth credentials from docker credential helpers

[mdelapenya]

What does this PR do?

This PR exposes a new method:

• AuthFromDockerConfig(registry string): retrieves the auth config type for a registry, obtained from the docker config

To implement it, we will internally check, in this particular order:

1. the DOCKER_AUTH_CONFIG environment variable, unmarshalling the string value from its JSON representation and using it as the Docker config.
2. the DOCKER_CONFIG environment variable, as an alternative path to the Docker config file.
3. else it will load the default Docker config file, which lives in the user's home, e.g. ~/.docker/config.json
4. it will use the right Docker credential helper to retrieve the authentication (user, password and base64 representation) for the given registry.

For that, we are using https://github.com/cpuguy83/dockercfg, which already handles the location of the Docker config file using the DOCKER_CONFIG environment variable.

Finally, now that we are able to read the Docker Auth configs, we are deprecating all previous manners to add them to the container definition (the container request, the buildFromDockerfile struct, and the reaper). Instead of using those fields, we are going to use the aforementioned extraction of the Docker Auth configs to populate them when needed.

In the case of building from a Dockerfile including a FROM clause consuming an image living in a private registry, the code will parse the Dockerfile alongside the build args, to extract all Docker images in it, trying to locate the Docker auth configs for each image. The build args will be interpolated in the case the image contains them.

Docs page has been updated with how Testcontainers for Go will autodiscover the Docker auth configs.

Why is it important?

It will allow consumers of the library to discover the auth credentials when working with images living in private registries.

We decided to provide that capability instead of automatically discovering it, but I'd not be opposed to do it if desired.

Related issues

• Closes Support for authentication when pulling Docker images everywhere #609
• Relates [Feature]: RegistryCred multiple private registry credentials support #532
• Relates How to control user account that launches images.  #324

[netlify]

✅ Deploy Preview for testcontainers-go ready!

Name	Link
🔨 Latest commit	402471b
🔍 Latest deploy log	https://app.netlify.com/sites/testcontainers-go/deploys/6407721572b5fa00086a20f5
😎 Deploy Preview	https://deploy-preview-869--testcontainers-go.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site settings.

[mdelapenya]

@HofmeisterAn came with the idea of reading DOCKER_AUTH_CONFIG as a possible input for the JSON value of the Docker config. I think it could be added to this PR with easy, although not sure if that should be added to the original library: https://github.com/cpuguy83/dockercfg

Thoughts?

[HofmeisterAn]

👏

[kiview]

I was wondering the same thing as mentioned by @HofmeisterAn, can't testcontainers-go load the correct credentials automatically without the user explicitly configuring it?

[HofmeisterAn]

I have two questions:

1. Does the implementation cover the PAT? I am not sure if .NET is outdated, but we have something like this, that sets the IdentityToken property.
2. Don't you need to execute the creds store somewhere to receive the credentials? Or is that covered by the dependency?

[mdelapenya]

I have two questions:

1. Does the implementation cover the PAT? I am not sure if .NET is outdated, but we have something like this, that sets the IdentityToken property.

PAT == Personal Access Token, right? There are two scenarios:

1. building from dockerfile: we receive the docker auth configs from the credentials helper or the config file, and we build a types.AuthConfig struct (a Docker type) with what comes from the config. That object is entirely passed to the ImageBuildOptions docker type, so whatever Docker client's ImageBuild method does internally with the PAT is transparent for us. If it's there, it will be passed.
2. pulling an image: we receive the docker auth configs from the credentials helper or the config file, and we JSON encode that struct, finally passing it to the ImagePullOptions Docker type. The options are directly passed to the Docker client's ImagePull method.

BTW, the identity token, when obtained from the cred-helper (see https://github.com/cpuguy83/dockercfg/blob/main/auth.go#L67 and https://github.com/cpuguy83/dockercfg/blob/main/auth.go#L161-L164) is returned as a password, pruning the user field.

in any case I'm going to give it a try tomorrow with a local test and a personal token, and check how it goes 🤔 Thanks for raising this concern!

2. Don't you need to execute the creds store somewhere to receive the credentials? Or is that covered by the dependency?

Indeed, it's the dependency the responsible of calling the credential helpers. See https://github.com/cpuguy83/dockercfg/blob/main/auth.go#L51

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

[mdelapenya]

I've tested this PR using private registries and private images on Docker Hub, and all our tests pass. Merging it

[srabraham]

Hey @mdelapenya, we're users of this package, and thanks for your work on it. We'd been relying on setting Docker credentials via RegistryCred, and after some mangling, I now have our code setting the DOCKER_AUTHCONFIG environment variable instead. One piece of feedback though: this PR claims to have "deprecated" RegistryCred, when in fact it made that feature stop doing anything at all. That presents quite confusing behavior for consumers. If you'd deleted those fields entirely, that at least we would've gotten a compilation error. As it it, there's an option (RegistryCred, and maybe a few others too) that exist but do nothing now

[mdelapenya]

@srabraham oh I'm sorry if this PR broke you (and all the users). I really thought we covered the deprecation paths for the fields, but it seems not. We could release a patch release bringing back the value of the deprecated fields if needed. How would you see that?

I'd like to thank you for your honesty in this feedback, and would like to also recognize that deprecating things are not that easy.

At the same time, we are trying to move to a much more stable state of the code and APIs to release a 1.0.0 version, and that could come with some breaking changes (unfortunately not this time 😥). I wish we'd communicate those changes in the best possible way through the release notes.

[srabraham]

@mdelapenya Thanks for that! We did figure out how to get it working; it just took a little time (EDIT: see code snippets here for a fix). The solution for us was to replace this, which we were using to set RegistryCred

with this, prior to making a GenericContainerRequest

as users of Bazel, our bazel test was disallowed from accessing $HOME, so the initial problem manifested as an error like "Failed to get image auth for https://index.docker.io/v1/. Setting empty credentials for the image: videoamp/{{redacted}}. Error is:error looking up user home dir: $HOME is not defined"

the one additional thing I'd add is that it'd be nice if this piece would fail if DOCKER_AUTH_CONFIG is set, but fails JSON unmarshalling. It's otherwise kind of confusing. e.g. I initially was confused after mistakenly passing in a base64-encoded string (rather than just bytes)

[mdelapenya]

@srabraham we could include that code snippet in the release notes for 0.19.0 as a workaround

[mdelapenya]

@srabraham I went ahead and updated the release notes for 0.19.0, tagging this PR as a breaking change. Again thanks for sharing your feedback in a positive way. We much appreciate it, and again, my apologies for the non-properly communicated breaking change.

[srabraham]

@srabraham I went ahead and updated the release notes for 0.19.0, tagging this PR as a breaking change. Again thanks for sharing your feedback in a positive way. We much appreciate it, and again, my apologies for the non-properly communicated breaking change.

Thanks for all your assistance with this! Yep, users of pre-v1 software need to be cautious, and that's ok! If you or anyone else would like my above snippets in code form (rather than screenshots), here they are: https://gist.github.com/srabraham/52e65ea37db5580a091334e3d065fed9