Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HIP: images lock file #281

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open

Conversation

paleloser
Copy link

In this HIP we're proposing a standard way of tracking a Helm Chart's container images through a new images-lock.yaml file, making easier to migrate whole Charts through different registries and generating a bill of materials, among other use cases.

Find complete details on the improvement document.

Pablo Caraballo Llorente and others added 6 commits December 16, 2022 09:22
Signed-off-by: Pablo Caraballo Llorente <pcaraballo@vmware.com>
Adding some typo corrections, and punctuation marks. Also defining the image-lock

Signed-off-by: mpermar <mpermar@gmail.com>
Signed-off-by: Pablo Caraballo Llorente <pcaraballo@vmware.com>
Signed-off-by: Pablo Caraballo Llorente <pcaraballo@vmware.com>
Signed-off-by: Javier J. Salmerón-García <jsalmeron@vmware.com>
Signed-off-by: Pablo Caraballo Llorente <pcaraballo@vmware.com>
Signed-off-by: Pablo Caraballo Llorente <pcaraballo@vmware.com>
Signed-off-by: Pablo Caraballo Llorente <pcaraballo@vmware.com>
Signed-off-by: Pablo Caraballo Llorente <pcaraballo@vmware.com>
hips/hip-0019.md Outdated Show resolved Hide resolved
* An `images-lock.yaml` (or similarly named) would be backed up by a spec. plus it could contain other useful information such as the images' digests that can be used by supply chains to enforce additional good security practices.
* Helm could internally generate this file out of the `annotations` proposed in `HIP-0015`.
* Helm could apply an existing `images-lock.yaml` file to the _post-rendered_ manifests to enforce a BOM (just like others such as `npm` do with their `packages-lock.yml`).
* By parsing this file, Helm could help users to migrate Charts between registries, either by providing new commands or through new features in the existing commands.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could also be developed as a plugin initially.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is a good point @joejulian . It might be worth actually to start this as a plugin.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took a while due to other priorities but we ended up following your advice @joejulian https://github.com/vmware-labs/distribution-tooling-for-helm


It may be less relevant to the end users how is the `images-lock.yaml` packaged alongside the Helm Chart, whether if it is pushed to a different OCI blob, or how is it named in the registry. These points should be mentioned somewhere as well, but it's likely that with less emphasis.

In addition to the points made above, given the size of the improvement, promotion through conference talks, presentations or any other public conversation would be strongly suggested.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe something in here about the linter and chart-testing


## Motivation

Container images are stored separately from Helm Charts. On one hand, this makes the Helm Chart really small in size. But on the other hand, this brings many challenges in terms of distribution and network access to images, which could be avoided if the application was self-contained.
Copy link

@gjenkins8 gjenkins8 Jan 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my experience, there are two differing ways helm charts are used:

  1. The chart "floats" around the application. In that helm chart version X, can be used to install a range of application versions Y... e.g. https://artifacthub.io/packages/helm/kubegemsapp/nginx#nginx-parameters has the image.tag property. Which effectively corresponds to the version of nginx being deployed (and which has little correspondence to the helm chart version there)

  2. The chart version is synonymous with the application version. ie. helm chart version X is used to deploy application version X. For which the chart must not allow parameterization of image tags. I don't know of an example in the wild, but we follow this pattern internally at my company for delivering software.

I think for this HIP to practically succeed, there needs to be a way to account for the first scenario. But don't get me wrong, also enhancing/formalizing a methodology for the second scenario would be fantastic. Very glad to see this HIP!

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great feedback @gjenkins8. I don't see much impediment in this proposal for 1. or 2. Basically what we are asking for Helm ecosystem is to have a way to generate a bill of materials at some agreed lifecycle command (e.g. helm dependency update). That command for example generates today a Chart.lock but in the future it could further decorate that lock or generate a different listing the container image dependencies.

Those container image annotations don't need to point to an specific version. e.g. a chart using nginx container could declare de dependency to 1.23.3, 1.23, 1, or not declare it at all and use latest. The lock file just states what was available at the given tag in dependency update time. As a Helm Chart is just a recipe, an user should always be able to use --set tag.image=1.12 and get an nginx deployment with nginx 1.12.

Now, lets say some new OCI/relocation/deployment/verification tooling exists built on top of this HIP and an user wants to use the latest nginx Helm Chart with nginx 1.12. That user, or automation, could simply change the annotation and pin that Helm Chart to that specific container image, or it could manually update the images lock (discouraged). Perhaps there are even more ways to do this.

Co-authored-by: Joe Julian <me@joejulian.name>
Signed-off-by: Pablo Caraballo Llorente <paleloser.code@gmail.com>
@mpermar
Copy link

mpermar commented Aug 24, 2023

For everyone falling into this PR. We have implemented a plugin that puts in practice the idea of a lock described in this PR. Hopefully this will serve as an adoption gauge as well. We are pretty excited about the possibilities that having this type of lock opens. It has already removed quite a few hassles for us on our internal content production pipelines.

Check out the plugin at https://github.com/vmware-labs/distribution-tooling-for-helm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants