Provide tooling for automated Kyma lifecycle and subscription management

[varbanv]

Description

Users should be able to fully automate Kyma related tasks in their CI/CD pipelines. This starts with Kyma instance provisioning, includes deployment of necessary artifacts and test executions, and ends with Kyma instance removal and associated service instance cleanup. All of this should not require much additional effort from the end user and should be as easy as possible to configure.

Context

Problem

Right now, users can automate the Kyma instance creation using the new Terraform provider, the cloud orchestrator tools, or the btpcli directly, however, the next step of accessing the cluster requires user interaction.

And finally, there are a number of scenarios that could prevent a cluster from being deleted and would require user interaction to complete.
While the second and third problems can be solved with some coding and additional resources, we want to provide ease-of-use in order to improve the perception and adoption of Kyma.

Benefits

For customers:

• speed up development and release cycles
• implement best practices around CI/CD
• reduce configuration drift
• reduce dependency on long-living Kyma instances
• reduce cost by only using Kyma instances when needed
  Example usecase scenario that should be possible to run in automated way

For us:

• increase adoption
• enable customers to treat Kyma instances as disposable assets and reduce risk
• enable users to reduce cost

Proposed solution

Design and implement a new set of kyma CLI commands that helps developers who use managed kyma runtimes within BTP ecosystem with development of their CAP applications. CLI commands should be designed with automation flow in mind (No user context should be required),

The commands should cover:

• automated lifecycle management of kyma runtime lifecycle (provisioning, deprovisioning)
• getting access to freshly created cluster
• attaching BTP service instances to kyma runtimes via service manager
• eliminating manual actions (i.e mapping hana instance, establishing trust for custom IAS tenant, etc..)

Acceptance criteria

• Users can create a Kyma instance and access it without any user interaction.
  • Support additional OIDC configuration with shoot-oidc-service extension #18305
  • POC: Command to access btp kyma runtime with workflow OIDC token cli#2093
  • Allow passing json configuration when calling provision command cli#2115
• Users can completely remove an existing Kyma instance without any user interaction, even if there are modules "stuck" in deletion.
  • Set force-deletion flag when creating shoot-cluster infrastructure-manager#103
• Provided solutions satisfy all security and compliance SAP standards (for example: no non-expiring access).
• Documentation and guides that cover the full end-to-end CI/CD scenario

[Disper]

What would be the exact/examples of scenarios around provisioning?

[kwiatekus]

Similar request to log into kyma in headless mode
https://github.tools.sap/kyma/backlog/issues/2518

[kwiatekus]

https://github.tools.sap/kyma/backlog/issues/2660#issuecomment-2104608

[pbochynski]

One possible solution is: #18305

[kwiatekus]

What would be the exact/examples of scenarios around provisioning?

@Disper
Additional config decribing a "system issuer" should be collected from user and sent to KEB. Further on, provisioner uses this data to deploy the OICD config object and enable the shoot-oidc-service extension.
https://github.tools.sap/kyma/backlog/issues/2660#issuecomment-3658944

Provisioner should add cluster role binding to principal represented by tokens issued by the system issuer

[kwiatekus]

Provisioning kyma runtime via BTP CLI
https://blogs.sap.com/2022/02/24/creating-sap-btp-kyma-runtime-via-the-sap-btp-cli/

Cloud orchestrator:
https://pages.github.tools.sap/cloud-orchestration/

[kwiatekus]

For users to be able to deploy their code w/o additional subscription (in the DEV mode) we aim to separate deocker-regisrty into separate DEV module to be used also outside of serverless usage
#18555 (comment)

[kwiatekus]

As of today, the new prototype commands (developed in https://github.com/kyma-project/cli/tree/v3) allow to

• provision managed kyma runtime within SAP BTP subaccount
• create hana db instance in an automated way (incl. binding and mapping to kyma).
• create references to shared instances

Getting access to new kyma runtime vi kyma CLI (in 100% automated scenario) will be possible after implementig #18305

[kwiatekus]

Customizable OIDC configurations (required for this epic) are being discussed within
kyma-project/infrastructure-manager#134

[kwiatekus]

We started working on cli part:

• kyma cli command that generates kubeconfig for CI/CD workflow
  POC: Command to access btp kyma runtime with workflow OIDC token cli#2093
• enhancing cli provisioning command to onclude extra oidc config
  Allow passing json configuration when calling provision command cli#2115 (dependency to KEB Extend Kyma Provisioning Parameters to allow multiple OIDC definitions kyma-environment-broker#423)
• kyma cli command that helps creating a serviceaccount based kubeconfig
  Produce token based kubeconfig originating from ServiceAccount via kyma cli cli#2036

ETA - on track