Skip to content
This repository has been archived by the owner on May 3, 2022. It is now read-only.

WIP: Push pull to image registry #442

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

WIP: Push pull to image registry #442

wants to merge 5 commits into from

Conversation

simonferquel
Copy link
Contributor

Based on #428
Introduced duffle bundle push/pull to work with a registry, leveraging genuinetools/reg.

For now we create a Manifest with a config mediatype fixed to docker image, because registries only accept a limited number of config types in their validation logic. Once they accept custom config types, we will be able to change that.

Also, cross-repository references cannot be modelled in a registry manifest, which makes it impossible to create images references at the manifest level. We need to dig into that with docker/distribution team.

…references

This aims at having signed bundles truly immutable. That means that all referenced images (invocationImages or images) without a digest are modified into digested references:
- if an image is available in the docker engine, and it has already a RepoDigest, we use it
- if an image is not available in the docker engine, we pull it and get its RepoDigest
- if an image is available in the docker engine, but has no RepoDigest, we consider that it has been built locally and has not yet been pushed. In that case we fail, and tell the user to push it.

Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
Also fix an issue with key generation on Windows

Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
Introduced `duffle bundle push/pull` to work with a registry,
leveraging genuinetools/reg.

For now we create a Manifest with a config mediatype fixed to docker
image, because registries only accept a limited number of config types
in their validation logic. Once they accept custom config types, we will
be able to change that.

Also, cross-repository references cannot be modelled in a registry
manifest, which makes it impossible to create images references at the
manifest level. We need to dig into that with docker/distribution team.

Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
@@ -89,6 +95,13 @@ func (bs *bundleSignCmd) signBundle(bundleFile, keyring string) error {
return err
}

if err := b.FixupContainerImages(containerImageResolver); err != nil {
if ok, image := image.IsErrImageLocalOnly(err); ok {
fmt.Fprintf(os.Stderr, "Image %q is only available locally. Please push it to the registry\n", image)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@simonferquel simonferquel changed the title Push pull to image registry WIP: Push pull to image registry Nov 16, 2018
@jeremyrickard
Copy link
Member

Hey @simonferquel we were having a more general discussion today abotu what the right thing to do w/r/t digests are and I wanted to pull on that thread.

We've been discussing using the Image ID vs the Repo Digest as a way of "validating" these things. I agree 100% with what you've done here and I think this is really the right way for validating these things when a registry is involved.

We also have a use case for "thick" bundles that we envision doing import/export with, like for disconnected edge device scenarios. My understanding with save/load (what we're using for that capability) is that digests aren't preserved:

moby/moby#22011

In this PR, you're pushing to the registry to grab the digest after the push, as they aren't calculated on build. How do we handle the import/export validation with digests? Does utilizing Image ID make more sense in that scenario/in general?

cc: @michelleN @carolynvs @technosophos @squillace

I think there is some concern about pushing to registries as part of a signing / duffle build inner dev loop. Any thoughts on that? For a "local dev" loop on a bundle, should we skip signing and not handle any of this? Currently the duffle build flow does sign things.

@simonferquel
Copy link
Contributor Author

The more I think about it, the more I think signing bundles sustematically is not such a good idea:

  • for the dev lifecycle, working with immutable bundles is very difficult. And with the idea of having signed bundles completely immutable down to having immutable image references, it is even worse.
  • for the registry flow, to guarantee resilience against referenced repository deletion, we need to copy all referenced images and invocation images manifests in the same repo as the bundle, wire everything with manifests list, and modify the bundle itself to change image references so that they point to the new manifests (the image layers/configs themselves wont be copied, only the manifests). This means modification of the bundle on push.

IMO, thin bundles should never be signed but should be digest-adressable from a registry + signable in notary, while pgp based signatures should be used for thick bundles integrity checks only

@simonferquel
Copy link
Contributor Author

We could also simplify thick bundle signature/validation quite a lot by signing the whole tarball instead of validating each individual image and putting just the bundle in a pgp enveloppe.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants