Replies: 3 comments 3 replies
-
|
I have a few questions and comments...
kubectl has imperative features. Not everything about it is declarative. People who want to do declarative work just avoid those features. The same idea applies to Helm. The point where being declarative matter is when it's sent to k8s. You declare to Kubernetes what you want and it works, through a reconciliation loop, to make that happen. How the material is generated to pass to Kuberentes doesn't need to be declarative. Not for k8s. Why does Helm having non-declarative features matter in this situation?
What is required in terms of transparency and for what? My first guess is that it's container images being used including those the operator uses that don't show up in the Kubernetes manifests. One of the reasons this is important is for the case of air gapped environments or those environments where a company vendors in the dependency and does things like security scanning. For this case there needs to be the ability to set all container images. Even those used by the operator. These can also be documented. Even using the artifact hub syntax for images. Is there some other reason for the transparency?
You're pointing out that your installation process is imperative. You can't declare everything to k8s and have the reconciliation loop work things out. This is common and highlights how the imperative ideal model doesn't always work. You're not the only one to encounter this.
Is there an easier way? The top level wrapper will benefit from being a Helm chart. What's the most straight forward way to do things inside that wrapper? Building some ordering system on top of hooks sounds new and brittle. |
Beta Was this translation helpful? Give feedback.
-
Right, I didn't write that clearly. What I keep hearing, is that an operator might want to audit all the manifests, before they're applied to the cluster. So they could check for service accounts, roles and such. I'm not sure this kind of pre-apply declarativeness is achievable with complex installs, especially when those contain dependencies that we don't develop ourselves, like minio and linkerd. I guess one should use kubewarden instead, to enforce such policies (as no privileged containers), instead of auditing a huge manifest manually.
Yes, there were some comments from projects that already do that, e.g. the rancher-istio chart uses a post-install hook to run an installer, but with the amount of dependencies Epinio has, it seems brittle, indeed. @mattfarina regarding "Helms existing logic around updates, waiting for resources" and "idempotency" in the installer. I can't judge how beneficial a switch to more Helm charts would be, compared to applying manifests from code directly. What's your epinion? |
Beta Was this translation helpful? Give feedback.
-
If I think of this outside of Epinio and how it would lay out in something like Rancher, having different Helm charts would be useful if things are shared system services (like cert manager). If everything uses a known system namespace we can install cert manager just once and reuse. Rancher itself has some capabilities around this. So, I would look at it from the integration standpoint with the larger system. |
Beta Was this translation helpful? Give feedback.
-
|
I see helmfile has support for ordering: https://github.com/roboll/helmfile#dag-aware-installationdeletion-ordering-with-needs Do you think it could be used to implement the everything-charts solution? |
Beta Was this translation helpful? Give feedback.
-
wait... that's what the Epinio installer currently does (btw, the internal registry is a helm chart: ) |
Beta Was this translation helpful? Give feedback.
-
Nice, so every component is in a helm chart already. |
Beta Was this translation helpful? Give feedback.
-
An idea that comes up regularly, is to use have the install logic in Helm, instead of the Epinio installer.
Possible advantages:
Problems:
The current idea is to use Helm hooks to construct a hierarchy of helm charts. The top level chart would run these charts in a certain order. If something has to wait for something, e.g. the Epinio server needs a private CA built by a cluster issuer resource, that's used to configure cert-manager, than that would have to be split into several charts. Most likely three charts for this example (cert-manager, cluster-issuers and certificate resource, Epinio).
It might be beneficial to build this hierarchy, which probably comes out at 20 Helm charts, some of them being upstream charts, with Hypper. Hypper can skip the installation of charts, if there is already a matching version in the cluster.
However, we would need to wrap Hypper in another Helm chart, so Rancher's Marketplace can use it.
Beta Was this translation helpful? Give feedback.
All reactions