feat: add support for pushing chart packages to oci

[dmccaffery]

Add a command, such as push similar to upload that pushes chart packages to an oci registry.

cr push <registry-url>

The cr cli can rely on the push api within helm to perform the work.

OCI support has reached general availability as of helm v3.8.1.

[dirien]

Hi all,

I would like to work on this task as I need this too.

Questions would be for me:

• Should it be an extra command, like @dmccaffery suggested with cr push ...
• Or, should it a flag on the existing cr upload command?

Looking forward for you feedback.

[dmccaffery]

I hacked a solution in this PR: fluxcd-community/helm-charts#94

[dirien]

yeah @dmccaffery, that looks cool! Thought the same, but want it to be part of cr.

[dmccaffery]

Me too! 😁

[dirien]

Do you prefer the cr push command as stated in your initial post?

[monotek]

Is somebody activly working on this?
We would like to use this in prometehus community (prometheus-community/helm-charts#2454).

[cpanato]

I can plan to work on this but if you have time @dirien go for it

i would implement the cr push

[dmccaffery]

I'm also happy to work on this now that agreement to implement a separate command (push) is there. I will have time this coming weekend. If anyone else wants to tackle it before then, let me know so we don't duplicate effort.

Thanks!

[dirien]

As @dmccaffery did the initial issue, it's fair that he has the honour to implement it! Let me know if I can help though! ☺️

[Ornias1993]

Would there also likely be an execution speed improvement?

[kingdonb]

What I found digging through the ends of this thread, you can do this today and it's not through cr but doesn't compromise all that much:

https://github.com/fluxcd-community/helm-charts/pull/94/files#diff-87db21a973eed4fef5f32b267aa60fcee5cbdf03c67fafdc2a9b553bb0b15f34
(from: #183 (comment)) bumping this up for visibility a bit

If you are using chart-releaser-action today, and you just want to add OCI (keeping the original index.yaml-based Helm Repository in place) you can add that 'push charts to GHCR' shim, that re-publishes the same chart files on the OCI repo.

I'd like to see the cr push action make some progress as well, but for what I need today this information might help unblock me a bit sooner and get my OCI repos kickstarted 🎉

I think this dual-publish approach should work for most basic uses, but where I think it will have trouble anyone publishing an umbrella chart. Those will need to either make the full switch (new charts, parent and child, umbrella and what goes under it, all published on OCI, with all references in OCI) or publish different charts in either source, else the users may wind up with charts from both sources, or references in the OCI chart that point back to the old helm repository – and they won't get the benefits as they will still end up downloading and parsing index.yaml – from the action interface for chart-releaser-action, does it make any sense to support publishing to both repos? Or would it make sense to gate with a flag (eg. legacy_publish: true or oci_publish: true) and change the flags defaults when the major version for cr bumps?

@Ornias1993 OCI represents all kind of speed improvements, the index.yaml is the bottleneck in Helm's traditional repo and it rears its head when you have hundreds or thousands of releases for as many different charts as your repo hosts, I'm sure that's what you mean.

OCI uses a tag index instead, where the tag is a semver (same as Helm has always used) and tag indexes are just way cheaper to parse than index.yaml with metadata for every release. If you're experiencing speed problems, that is most likely where they came from?

Just coming up to speed on this issue, I'm interested to see it finished too 👍

Large chart hosts like bitnami, etc. have the most to gain from this upgrade, since anyone that automates upgrades with Helm will be downloading their index.yaml again in full, for each chart upgrade in any chart in the whole repo, even if they are caching as efficiently as possible, this index file is a very wasteful process in retrospect (and OCI fixes it!)

My knowledge about the chart-releaser project is a bit lacking, does the chart-releaser handle umbrella charts already?

[kingdonb]

Follow-on to my last post, it seems there are some simplifications we can make and some of my assumptions were too cautious

• https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack - We should try to add support here first, this is a great project that already uses chart-releaser and is an umbrella chart, this also has external dependencies
• This raises the question of "what happens if my upstream dependencies aren't on OCI, but I want to publish my umbrella chart as OCI-only"

Turns out I guess that's not as big of an issue as I imagined, since the dependencies will be packaged together in the OCI artifact, the dependency paths only get invoked when you want to run helm dependency build or helm dep update

Is anybody making progress on this, can we make progress? I was prepared to update the github actions workflow, but I was a bit daunted by the go project underneath (and I could try diving in tomorrow, but if it turned out somebody already has...)

[Ornias1993]

There is another issue here, and I think that might also hit prometheus-community @kingdonb
If you release your containers on OCI, I don't think you can also release your helm chart on the same OCI using the same name as well...

[kingdonb]

@Ornias1993 That's true-ish... there is a media type on each image layer, there are configurations which you can use to bundle image layers of different types, but a straight helm push into the same image repo as docker build+push is inadvisable. I'm more familiar with this issue from the Flux side, where it's flux push and the media type is configurable.

If you want to release your charts as OCI images, I recommend following the pattern that you see here:

https://github.com/stefanprodan?tab=packages&repo_name=podinfo

Create a repo of the same name, and push app images there (in this case podinfo)

Then, in your organization (or user) create an image repo with charts/podinfo for example. You can create these arbitrary structures on GHCR without creating a corresponding charts git repository, by doing helm push or flux push on-demand, and associate the new OCI image storage repo with the Git repository whose actions to push from via GitHub Actions permissions UI. (This is under-documented, but we should ideally not document all this for GitHub, they should have docs to support this... I haven't found them yet.) Anyway, this approach works well for organizations with many charts, and then the main problem is to ensure all the dependencies also adopt OCI, which may require some string substitution or maybe not... haven't tested it all yet from end-to-end.

I plan to try re-publishing Hephy Workflow as OCI charts in a day or two and we'll see how it goes there.

Pushing a mix of image types to the same repo is inadvisable because you can no longer take advantage of basic semver tags and pattern matching to decide which tag is the latest in your series, you need a way to distinguish between tags of the different types (and scanning the images to read their metadata is expensive and does not scale to thousands of tags, as Flux project knows from our history and experience...)

[jkroepke]

• prometheus-community/helm-charts@main/charts/kube-prometheus-stack - We should try to add support here first, this is a great project that already uses chart-releaser and is an umbrella chart, this also has external dependencies

We are using OCI push for months now and it just works great.