Support non-git kustomize yamls

[keikumata]

Issue

We have thousands of clusters using Flux right now and for us, it's not feasible for every cluster to hit our git repository due to git server load constraints & throttling. To solve this issue, we replicate our git servers to a gitsrv pod on each of the clusters and have Flux hit the local git endpoint. However, this solution hits a limitation where if we want different services to have different release cadences, then we need to create a local git mirror per service, which is less than ideal.

Suggested solution

Kustomize controller, like Flux, has the above limitation as it assumes the source data is a git repository (ignoring Helm as we don't use it). Would it be possible to support a source that is an external link to just the kustomize yaml? This way we can push our kustomize yamls into different (regional) storage accounts and just reference the URL from our clusters (the Kustomization resource could reference it directly even).

I know this strays away from "pure GitOps" (centralize everything in git) but was hoping to get your guys' thoughts on this. I believe other production scenarios will have similar constraints if there are many clusters being run. If supporting a Kustomize source seems reasonable, I'm happy to contribute the code!

[stefanprodan]

Kustomize controller, like Flux, has the above limitation as it assumes the source data is a git repository

The kustomize-controller makes no such assumption, what it needs is an artifact with manifests that source-controller provides.

Would it be possible to support a source that is an external link to just the kustomize yaml?

I guess you are referring to the Kubernetes manifests not just a kustomization yaml. We could implement other sources in source-controller using the S3 protocol. Instead of a Git repository, the manifests could be fetched from a bucket: Minio, AWS S3., GCP bucket, Azure blob??

[stefanprodan]

We've implement Bucket source type for S3 compatible storage in source-controller and we've added support for Buckets in kustomize-controller. Ref #242

@keikumata is Azure Bucket compatible with the S3 API?