Open internal docker registry for customers

[pbochynski]

Description
Provide a way to use in-cluster docker registry for hosting customer's images.
As the registry is accessible with the node-port it should be possible to push the images from the developer's machine or from CI/CD agent using port forwarding. Provide a command in the kyma CLI to push images.

AC

• The registry backend could be also replaced with durable highly available storage
  • NFS volume
  • Configure s3-compliant object store as a backend via DockerRegistry CR  docker-registry#25
• Allow configuration for internal docker registry volume  docker-registry#46
• Allow access to internal docker registry via docker CLI docker-registry#38

Reasons
Pushing images directly to the in-cluster registry simplifies the development, testing and even production releases.
Users should be able to easily deploy their code and run their tests without the need for additional BTP or hyperscaler services.
Currently a problem arises when users want to deploy their code to the cluster, due to the fact that there is no SAP registry available. Unlike with CF where a cf push does all the work necessary to deploy the code.

Related Issues
Enable Registry Cache Extension

[kwiatekus]

Allow serverless module users to configure custom HA store as serverless docker registry backend.

• Configure s3-compliant object store as a backend via DockerRegistry CR  docker-registry#25

[kwiatekus]

@pbochynski we see more and more reasons to extract docker registry into separate module. Any objections?
We would like to try it out and do it in a way that serverless (untill its still dependant on internal docker registry) ux doesnt suffer
kyma-project/docker-registry#19

[pbochynski]

@pbochynski we see more and more reasons to extract docker registry into separate module. Any objections?

The only problem I see is that the serverless module will not work out of the box. You would introduce the dependency to registry module or to external registry. But maybe it is another reason to consider changing serverless module in the way that it doesn't have to build a container. Just load function code from CR and cache dependencies on persistent volume.
What do you think?

[kwiatekus]

Thanks @pbochynski . That would mean that serverless runs pre-built containers as function runtimes with code dependencies mounted. dependencies resolving would need to be moved from built time into runtime ( start time ). There is a consequence in function scaling scenario: i.e when a new replica starts it would need to resolve dependencies ( i.e npm install ) into a mounted volume. In theory, replicas could have different versions of dependencies. But if we consider serverless being dev module only that should be ok.

[kwiatekus]

Opening internal docker registry should be considered as extending toolset for developers to run/test their applications on kyma only in develoment mode. We should keep recommending full-fledged HA external docker registry for production grade installations of customer applications.

[kwiatekus]

We aim to separate docker registry reconciliation to a dedicated operator, which could be used both in and out of sereverles context.

• Reconcile internal docker registry via dedicated operator docker-registry#19
• POC: Serverless uses separated internal docker registry module  serverless#849

[kwiatekus]

• Separate reconciler for docker registry has already a first working version developed in the dedicated repository https://github.com/kyma-project/docker-registry

The team is also looking into using the latest version of distribution (a.k.a registry) from cncf ->

• Use latest distribution image as internal docker registry docker-registry#18 [

[pPrecel]

@pbochynski we see more and more reasons to extract docker registry into separate module. Any objections?

The only problem I see is that the serverless module will not work out of the box. You would introduce the dependency to registry module or to external registry. But maybe it is another reason to consider changing serverless module in the way that it doesn't have to build a container. Just load function code from CR and cache dependencies on persistent volume. What do you think?

I think, in this case, it would be nice to have one PVC for all functions, not to reserve space for every function. The problem is that almost all types of PVC are namespace-scoped so it would be impossible to use simple PVC. The only solution I see right now is to use nfs (network-file-system) that allows to share volume between namespaces. It's something to investigate

[kwiatekus]

Following recent feedback we will look into allowing using the module via docker CLI
kyma-project/docker-registry#38