dependsOn support across different resource kinds

[aclevername]

Slack discussion: https://cloud-native.slack.com/archives/CLAJ40HV3/p1625149645282400
Previous related issue: fluxcd/kustomize-controller#242

Hello 👋

Feature request

I would like to utilises flux's awesome dependsOn feature for resources of different kinds (in particular a HelmRelease resource depending on a Kustomization)

Context

HelmRelease and Kustomization have support for specifying a dependsOn field in there spec that determine what resources must be running before deploying the resource. For example from the flux docs:

apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: backend
  namespace: default
spec:
  interval: 5m
  chart:
    spec:
      chart: podinfo
      ...
---
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: frontend
  namespace: default
spec:
  interval: 5m
  chart:
    spec:
      chart: podinfo
      ...
  dependsOn:
    - name: backend

This above configuration ensures that the backend HelmRelease resource is deployed and running before attempting to depoy the frontend.

Currently this cannot be done for resources of different kinds, as dependsOn only takes a name and namespace, and assumes the resource is of the same kind. There is a workaround, which is to wrap every HelmRelease resource in a Kustomization resource, and add a healthCheck to the Kustomization. Example. While this does work, it adds a lot of extra complexity and files to your git repository.

Proposal

Introduce a kind and apiVersion to the dependsOn field, matching the pattern of how sourceRef works.

apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: Kustomization
metadata:
  name: backend
  namespace: default
spec:
      ...
---
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: frontend
  namespace: default
spec:
  interval: 5m
  chart:
    spec:
      chart: podinfo
      ...
  dependsOn:
    - name: backend
      kind: Kustomization
      apiVersion: source.toolkit.fluxcd.io/v1beta1

Technical discussion

Hiddeco mentioned a desire not to have the source-controller and helm-controller API be coupled together, which makes sense. This functionality could follow how healthChecks in Kustomization works and use the kstatus standard for determining if a resource is ready, without having to hardcore in specific types.

Thoughts? The eksctl/pitch-black-team would be happy to work on PRs to implement this 😄

[hiddeco]

Based on a past conversation on Slack, you know I am generally positive / in support of this idea.

Couple of things that need to be discussed and/or taken into account:

• Limitations of kstatus, and what happens if something has not been "acknowledged" by a controller at present, which may happen if a cluster is really busy: Kustomization marked as ready when health check resource is not ready kustomize-controller#387
• Impact on fluxcd/pkg/runtime and the generic API elements in fluxcd/pkg/apis, more specifically:
  • https://github.com/fluxcd/pkg/blob/main/apis/meta/dependencies.go#L23
  • https://github.com/fluxcd/pkg/blob/main/runtime/dependency/sort.go#L29-L34

[davinkevin]

Hello,

Any news on this "feature request"? I'm in the same situation with a simple case with CertManager installed with an HelmRelease and a Kustomization to install some ClusterIssuer.

Thank you all 🙏

[uhari03]

@davinkevin Did you find any work around for this? Or do I need a "dummy" kustomization to wrap the HelmRelease?

[davinkevin]

The best workaround for me was to:

• locally generate the chart using helm template
• commit it
• use a simple flux.io/Kustomization to install it

It brings additional advantages to the GitOps logic:

• we are able to analyse chart content before going to prod
• we don't have drift problem with charts
• we are sure an update won't be triggered by an external operation (non-immutable tag in the helm registry for example)
• we don't suffer from charts using generated values which could be applied multiple times a day
• we can apply standard kustomization above them with ease
• we have CRDs directly into the generated manifest, so no upgrade problem
• we have CRDs available, our favorite IDEA is able to use them to provide auto-completion for those resources
• we don't need a connection to an external (helm) repository, everything is in our git repository

And after some discussions with multiple companies during events, I can say this pattern is more widely used than I though (even outside of Flux).

[tbondarchuk]

@davinkevin That's actually what ArgoCD does under the hood. But Argo converts helm hooks to its own so it runs pre/post install/upgrade jobs in the correct order. Now, with Flux and helm chart flattened to plain manifests there is no way to ensure "apply order", as far as I know (unless I've missed some magical annotation?). Cert manager chart does not have any such hooks I think, but for example ingress-nginx does. Were you able to overcome this somehow?

[davinkevin]

If you are dealing with two kustomization, you can use the dependsOn described in the original comment because you don't mix types.

I had a similar problem with the mastodon chart, and I chose to just split the result of the helm template. It was more work, but again, I control the data I push and removed an extra layer of indirection. It's an opinionated vision, but in my case, it's ok. Of course if you have hundreds of case, this solution doesn't scale.

[dverzolla]

Hi,

A dependsOn with different kinds is still a good choice, all other workarounds take a long path to solve this.

[Forbzy]

This is exactly what I've been waiting for.

[dverzolla]

You mean 'working for' or 'waiting for' ?

[Forbzy]

Sorry yes waiting for. I updated my comment.