Services in Epinio

[jimmykarily]

It makes sense for any PaaS to allow the users to provision services in a self-service manner. The way this works should separate the concerns between Ops and Developer teams:

• Ops: This team decides what can be provisioned, where and how. Maybe also "who" can provision (though it won't be relevant to Epinio until we have roles).
• Developers: This team should be able to list, create and delete services enabled by the Ops team. They shouldn't have to worry about the "how" and "where" as much as they care about how to get connection details for the service they provisioned.

In the past, we used the Open Service Broker API to enable this but that project didn't seem get adoption in the community and we removed it from Epinio.

We also investigated the idea of integrating with CrossPlane. While this is possible, there doesn't seem to be an existing marketplace of XRDs we could use to populate a marketplace in Epinio. We would need to do most of the work ourselves (the Epinio team) or leave it to our users.

I want to suggest an idea here which may allow us to integrate both with Crossplane but also simple helm charts (e.g. https://artifacthub.io/packages/helm/bitnami/mysql) or any other solution that works with resources that are created on a Kubernetes cluster.

Epinio needs a way to discover service made available by the Ops team. All Epinio needs to know is:

• A name and description of the service
• Supported configuration (things that can be configured, e.g. database size)
• Where the details of the service will be stored after it's provisioned (probably a Secret name)

This contract is partially fulfilled by Crossplane but the "Supported configuration" is not strictly defined on the XRD and would need us to invent some convention. Also, although there is a provider for helm charts, it works on "managed resources" (Release) not XRDs. This means, we can't use it directly by making Epinio XRD-aware.

So, what if instead of Crossplane XRDs, we created our own custom resource which defines:

• The helm chart (repository, URL, anything needed to find and install a chart)
• The values the user is allowed to set as a map
• The secret where the connection details can be found after provisioning (most of the time this depends on the name of the release so this might not be the the actual secret name but a value like "release-name", tbd)

There are existing projects that can install helm charts using a controller (e.g. https://fluxcd.io/docs/guides/helmreleases) but they don't have a custom resource like XRDs which defines what can be provisioned, without actually provisioning it.

If an Ops team wants to work with Crossplane, they could simply set it up on the cluster and create a Helm chart with the required XRs in it. They can make it available in Epinio by simply creating the Epinio "Service" resource which points to that helm chart.

Similarly to what we are doing for applications we will just use Helm charts as the contract between Epinio and the Ops teams.

I haven't given this idea much thought, this is more like notes on what I have in mind. It may be ignoring existing projects that already do this, or have logic gaps I didn't think of. I wanted to open it for discussion as early as possible. This will allow us to reject it faster if it's not good.

[agracey]

I think this is great idea!

I wonder if we can reuse any of the marketplace CRDs that Rancher includes? Maybe see if we can add a way to filter a bit based on RBAC or labels...

[enrichman]

The idea seems cool, so basically a developer could provision a Postgres database, if the resource was defined by the ops team, and this will trigger the deployment of a database, with some predefined values, and/or thresholds?

[jimmykarily]

Yes exactly. That's the same concept with service brokers and crossplane just implemented using helm charts.

[enrichman]

Nice, we should really consider the idea of renaming the actual services then. 😂

[jimmykarily]

renaming is done: #1287

[jimmykarily]

spike story here: #1281