Optional components - single helm chart

[jimmykarily]

Epinio needs various components for its provided functionality. Up to now, these components should be there otherwise Epinio won't work. This puts a lower limit to the installation complexity even when Epinio is just installed for quick demo locally. To make the process easier we had to come up with a custom installer. This installer is a maintenance burden has limited functionality and forces us to publish two different helm charts.

A better approach would be for Epinio to use the components when they are available but not fail otherwise. This is how it's possible (per component):

Linkerd

It is used to encrypt communication between components (e.g. Epinio server talking to Minio). It's enabled through annotations on namespaces. It can be made optional by putting the annotations in place and making sure it's going to work if likerd is installed (check through tests). If linkerd is not there, communication will simply not be encrypted, done.

Traefik

This is the ingress controller which implements our Ingresses for the epinio api server, the internal registry and the applications. We don't have complex requirements from the Ingress and we could as well live with nginx or other ingress if the user has one installed. We can make Traefik optional by putting the needed annotations for the various ingresses. Whichever is there (or is the default) will work. Relevant issues: #1178, #382

See also here regarding the usage of each component: https://docs.epinio.io/installation/install_epinio_manual.html

Kubed

This is needed to copy the registry secrets to every new Epinio namespace (and possibly other secret copying I'm currently missing). It's a very lightweight component and easy to install. We can simply make this a subchart of the Epinio chart. No need to make it optional.

https://github.com/epinio/helm-charts/issues/91

Cert manager

Needed to create TLS certificates for anything that has an Ingress. We can detect its presence by checking if the relevant CRDs exist. If not, we simply skip the creation of Certificates and let all communication be non-TLS.

#1192

Tekton

gone

S3 storage

Will be gone with this: #1157

OCI registry

Use to store the staging artifacts (images) and as a replacement for S3 (see issue above). This can't be made optional but it can be external or a subchart in the Epinio chart.

[jimmykarily]

With the above plan, we can get rid of the second (installer) helm chart. We will only ship one helm chart that can be used for any type of installation.

This should unblock this story: #1182

[ksondere]

Trying to understand this better. As I understand epinio it's almost an open source distro if you will of the CI/CD process. So when I see the statement that Tekton is gone I'm confused. I thought that was at the heart of this. Also, I'm actually searching to understand how GitOps particularly Rancher Fleet fits into the CD picture of this system.

[ksondere]

Just as Rancher is a "distro" of Kubernetes isn't epinio a CI/CD "distro". And yes where does fleet fit into this distro as it appears to be central to much of what Rancher is doing.

[jimmykarily]

Any PaaS can fit a definition of a CI/CD pipeline and that's because CI/CD can mean so much. Take Heroku for example, you give it your code and it gives you a URL to access your app. You don't know if it's using Tekton or any other tool behind the scenes because that's an implementation detail, you are not supposed to know or use it even if it's there. Having said that, PaaS solves problems which usually don't fit in CI/CD responsibilities. E.g. Generating TLS certificates for your app, mounting secrets, fetching logs etc

Tekton in Epinio has been an implementation detail. There are some features that could be implemented by exposing Tekton to the user (e.g. customizable application deployments) but we chose to implement those in other ways (e.g. with helm charts). So Tekton was replaced with a Kubernetes Job and nobody should care, because Tekton was not a feature of Epinio, it was an implementation detail.

Regarding Fleet, it's not currently used in Epinio. It can be used alongside Epinio to implement a GitOps development flow in Epinio but it't not integrated. At some point, Epinio could integrate Fleet to do the same natively.

I hope this clarifies things.

[ksondere]

Going to the link , I wonder if documentation can make things a little more clear on how epinio works alongside fleet. The documentation basically says if your not using fleet then install gitjob.

[enrichman]

I'm still a bit puzzled about the "check, check, check". I think that if we want to simplify the installation we should not even check for existing stuff, but just give the minimum viable product and eventually give instructions on how to spice it up.

So basically the single basic chart should be only Epinio. 🙂

i.e.:

Mandatory components

Epinio

run this chart and you got Epinio up&running (and that's it).

OCI Registry

Then you need an OCI registry where to store the images, if you have one just set the env var like this, or you can install an internal one like this.

Ingress

If you want to reach Epinio you need to setup an ingress like this, or if you already have Traefik or an Nginx add this labels/services etc.

Optional components (security, production configuration)

Cert Manager

You should secure your traffic, so you should install cert-manager, to do so you have to blablabla. But if you already have you just need to add this issuers and do this.

Linkerd/Istio

Also the internal traffic should be encrypted, then you could use Linkerd, Istio or something else, just add this labels like this blabla.

[jimmykarily]

There are things Epinio doesn't have to know if they are there (e.g. Linkerd) and things it needs to (e.g. CertManager). Creating a TLS certificate for a newly deployed application happens when the user pushes the application. Epinio has to know if CertManager is installed in order to decide whether to create a Certificate resource in Kubernetes or not.

To support linkerd, Epinio just has to add the relevant annotation on the namespaces it creates, but it doesn't have to care if Linkerd is actually running or not.

Then there are components without which, Epinio can't even work, like the registry. There are various approaches to Epinio installation and we have tried a bunch of them. What you describe could also work and that's what we describe in this document: https://docs.epinio.io/installation/install_epinio_manual.html

But people don't generally follow a step by step guide with 10 steps just to try out a project they haven't tried before. In the current state of Epinio, it's important that we have an easy way to install it, even if that installation is not suitable for production.

[enrichman]

Yes, I meant that Epinio doesn't need to know about their existence during the installation, or fail if they're missing, but as you said they will be used if provided. So if there is the cert-manager then Epinio will provide TLS for free, while not having it then it will result in unencrypted traffic.

[agracey]

The problem is that if you try to create a CertificateRequest object and that CRD doesn't exist due to a missing cert-manager, the create will fail. This gets exacerbated when we change to using helm since the whole "install" will rollback.

[enrichman]

I'm playing with the most simple Epinio installation, to see what can be improved or simplified.

So at the moment I have cut down the chart to these Epinio components:

• Epinio Deployment
• Epinio and Epinio-Staging Namespace
• Epinio Service
• Epinio App CRD

there are also the components about the Epinio "configuration", such as the ClusterRole, ClusterRoleBinding, Role, RoleBinding and ServiceAccount.

With only these components Epinio is reachable and up&running. Furthermore the only needed variable in the values seems to be the server.sessionKey for the cookies.

Then I've added manually the Ingress (I wanted to test both the Traefik and Nginx but since I needed to install the Nginx controller withouth Traefik that was taking time and it was also something that @manno was trying I gave up).:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: epinio
  namespace: epinio
  annotations:
    kubernetes.io/ingress.class: traefik
  labels:
    app.kubernetes.io/name: epinio
spec:
  rules:
  - host: epinio.172.18.0.3.omg.howdoi.website
    http:
      paths:
      - path: /
        pathType: ImplementationSpecific
        backend:
          service:
            name:  epinio-server
            port:
              number: 80

After the Ingress I've tried to deploy an app, and I've seen that the first problem was obviously about my configuration.

Trying to do an epinio config update was failing for the missing user, so I've added one like this:

apiVersion: v1
kind: Secret
type: BasicAuth
metadata:
  labels:
    epinio.suse.org/api-user-credentials: "true"
  name: default-epinio-user
  namespace: epinio
stringData:
  password: password
  username: admin

Tried again in order to get the config but it hangs here. Basically it tells me that it's waiting for the server certs, but it should not be so "mandatory", because after I edited manually the config file with the correct username and password then the cli works, also because we provide the --skip-ssl-verification flag.

Proposal: can we skip the cert if not available?

After that I've tried a push with epinio push --skip-ssl-verification -n samplejava -p . and it almost worked, but it failed because of the missing workspace namespace.

This part is probably related to a huge topic that we will need to address, about multi-tenancy, user permissions and so on.
I agree that the operator should create the needed namespace (so this is something that needs to be done during the installation, or later). The epinio config update should probably be more "smart", and fetch a specific user, for a specific namespace. But as I said, that's a more complex topic.