✨ Make secret name optional

[sagikazarmark]

What are you trying to do?

I have a function that creates a secret: https://daggerverse.dev/mod/github.com/sagikazarmark/daggerverse/registry-config@b45dbd7448bb967aca4a538af9ce7f042abf0316#RegistryConfig.secret

The current secret API takes a name at the moment: dag.SetSecret("name", "value").

I'm not sure, but my guess is that name is supposed to be unique (ie. to make sure there is no interference between calls to the same function).

Why is this important to you?

Right now I'm working around the issue mentioned above by exposing an optional secret name argument in the API. I'd rather not do that.

How are you currently working around this?

I'm planning to generate a secret name based on the content: sagikazarmark/daggerverse#104

That will make sure it's unique enough and won't require me to expose options in the API for the secret name.

[shykes]

FYI @jedevc @vito @helderco @aluzzardi I think we may need to revisit the design decisions that led to this point. Since then, we launched functions, modules, and didn't revisit the initial plan of adding "secrets providers". Is the current design still relevant? Do we still need the secret name?

[helderco]

One use case we have today is when adding secrets to a docker build. The secret name will be turned into the secret's id in the Dockerfile:

RUN --mount=type=secret,id=secret-file curl

One problem today is that if you're passing a secret via the CLI, the secret name is an hash. So you currently need to work around that by either creating a new secret inside your function, with the plaintext of the Secret argument and the name that you use in the Dockerfile, or get the secret's name and pass it to the build as a build argument.

Build arg example

Fix for https://github.com/MikaelElkiaer/dagger-examples/blob/ae8a49103ea086f09eb50a423d4bed4871cb0c12/dockerfile-secrets/dagger/main.go:

func (m *DockerfileSecrets) Build(ctx context.Context, buildContext *Directory, secretFile *Secret) (*Container, error) {
	secretName, err := secretFile.Name(ctx)
	if err != nil {
		return nil, err
	}
	return buildContext.
		DockerBuild(dagger.DirectoryDockerBuildOpts{
			BuildArgs: []BuildArg{
				{Name: "SECRET_ID", Value: secretName},
			},
			Secrets: []*Secret{secretFile},
		}), nil
}

In the docker build, you pass the secrets as Secrets: []*Secret{...}. If it were a map, we could use the map’s keys to be the IDs in the Dockerfile. But GraphQL doesn’t support maps so we’d need a new type with name: String! and value: SecretID! fields.

Suggestion: add a withName field

However, since Secret already has a name, I think the simplest solution is adding a withName field to change the secret’s name. So the collapsed example above becomes much simpler:

func (m *DockerfileSecrets) Build(buildContext *Directory, secretFile *Secret) *Container {
	return buildContext.
		DockerBuild(dagger.DirectoryDockerBuildOpts{
			Secrets: []*Secret{secretFile.WithName("secret-file")},
		})
}

If you then make the name in SetSecret optional too (or create a new one without the name), then it won’t bother you if you don’t need to set it, but will also allow to lazily set it when needed.

Not sure about the secret drivers use case as I haven’t thought much about it. Curious to know what @sipsma has to say.

[helderco]

Forgot that we have a dag.Secret("<name>") and that secrets have been isolated in modules:

• Isolate secrets to each module #6601

\cc @jedevc

[jedevc]

Honestly, I'm not opposed to removing dag.Secret. It's incredibly fiddly as an API, and has global state, so would happily see it gone (though it still needs to exist, it could do so just internally).

[shykes]

2 questions:

1. What happens if we remove the concept of secret name altogether? It seems like it may be getting in the way and not actually helping

2. Do we still want a "pluggable secret provider" feature? If so what would that look like? The current design exists in part to keep the door open to such a design in the future. If we no longer feel the need for that, it's important to know. (cc @aluzzardi )

[helderco]

1. What happens if we remove the concept of secret name altogether? It seems like it may be getting in the way and not actually helping

How would it work in a docker build as explained in #7358 (comment) ?

Could be an input type like BuildArg:

+ input SecretArg {
+   name: String!
+   value: SecretID!
+ }

 type Container {
   build(
     context: DirectoryID!
     dockerfile: String = "Dockerfile"
     buildArgs: [BuildArg!] = []
-     secrets: [SecretID!] = []
+     secrets: [SecretArg!] = []
     target: String = ""
   ): Container!
 }

This way I'd agree with removing the name as I can't think of anything else where a name is needed.

[helderco]

2. Do we still want a "pluggable secret provider" feature? If so what would that look like? The current design exists in part to keep the door open to such a design in the future. If we no longer feel the need for that, it's important to know.

I'm struggling to find a need for it when a normal module seems to do the trick.

[shykes]

@helderco yeah agreed on your idea for solving docker build. If the only reason for keeping a name is docker build... Better to solve the problem for docker build, and remove the secret name.

How does the docker CLI do it? Is it actually a build arg, or a distinct concept of build secret?

[TomChv]

How does the docker CLI do it? Is it actually a build arg, or a distinct concept of build secret?

It's a different concept based on the official doc

RUN --mount=type=secret,id=mytoken
TOKEN=$(cat /run/secrets/mytoken) ...

docker build --secret id=mytoken,src=$HOME/.aws/credentials .

Which has a unique identifier (mytoken in this example).