Flux in clusters without a CNI

[maxpain]

Hi.

Is it possible to install Flux in a cluster without a CNI? (for example using hostNetwork: true)
I need Cilium to be initially installed (and updated in the future) by Flux from HelmRelease manifests in the git repository using GitOps.

[stefanprodan]

Flux needs a CNI to access the source-controller.svc service. You can install a CNI plugin at cluster bootstrap with the Helm CLI, then Flux will take over the release if you define a HelmRelease with the same values.

[maxpain]

Yes, the problem here is that I have two places where cilium helm values exist. I want to do the initial installation of Cilium from the HelmRelease file. Can I achieve this using some Job with flux CLI?

We have a lot of clusters, and I need to install everything declaratively.
I use Talos OS, and it can install Kubernetes manifests on bootstrap time, so I wanted to install Flux CD first and then it should fetch our infra git repo and install Cilium.

Can Flux work with hostNetwork: true?

[maxpain]

Or imagine a situation when you for some reason broke your CNI plugin on all clusters by pushing a commit to the infra flux repo and have to manually deploy a fix.

[stefanprodan]

It should be possible to use the host network, you'll need to set source-controller.flux-system.svc.cluster.local. in the kustomize-controller resolver to point to the source-controller host IP.

[maxpain]

Yes, I did it using Kustomize: https://github.com/maxpain/flux-without-cni/blob/main/kustomization.yaml

[disi]

I did deploy flannel CNI via helm to the cluster and created a Helmrelease for it, same name and same namespace. However, the kustomization.yaml seems to be ignored for the Helmrelease?
If I change the version value in the file flannel-cni.yaml, Flux does deploy a different version.
flannel-cni.yaml:

---
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: HelmRepository
metadata:
  name: flannel
  namespace: flux-system
spec:
  interval: 24h
  url: https://flannel-io.github.io/flannel/
---
apiVersion: helm.toolkit.fluxcd.io/v2beta1 # always helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease # always HelmRelease
metadata:
  name: flannel # name in kubernetes
  namespace: flux-system # namespace in kubernetes
spec:
  interval: 30m # check for updates
  timeout: 5m # timeout for kubernetes operations (default 5m)
  install:
    remediation:
      retries: -1 # always try to fix
  releaseName: flannel # best use upstream chart name
  chart:
    spec:
      chart: flannel  # Must be the same as the upstream chart name
      version: '*' # use latest version
      interval: 12h
      sourceRef:
        kind: HelmRepository # <HelmRepository|GitRepository|Bucket>
        name: flannel # References the HelmRepository usually upstream chart name
        namespace: flux-system

kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

# This setting has currently no effect
# If you want to change the flannel version not to be the latest release, do so in flannel-cni.yaml directly
# patch the release version
# * = latest release
# <1.0.0 = release lower than 1.0.0
# 0.23.x = latest subversion in release 0.23
# =0.23.0 = fixed release 0.13.0
patches:
- target:
    kind: HelmRelease
    name: flannel
    namespace: flux-system
  patch: |-
    - op: replace
      path: /spec/chart/spec/version
      value: "*"
resources:
  - flannel-cni.yaml

[stefanprodan]

I have created a Flux distribution that works on clusters without a CNI. Please see https://github.com/stefanprodan/flux-aio.

[kvaps]

I just raised a question of flux-community/helm-chart needed this feature.
I'd like to implement it fluxcd-community/helm-charts#216