Gardener Extension for Katacontainers

Project Gardener implements the automated management and operation of Kubernetes clusters as a service. Its main principle is to leverage Kubernetes concepts for all of its tasks.

This controller implements Gardener's extension contract for the kata ContainerRuntime. The latest release's ControllerRegistration resource that can be used to register this controller to Gardener can be found here.

Please find more information regarding the extensibility concepts and a detailed proposal here.

Table of Contents

• Gardener Extension for Katacontainers
• What does this package provide?
• Wait! How does this differ from kata-deploy?
• Current limitations
• How to...
  • Enable the extension as a gardener operator
  • Use it as end user

What does this package provide?

Generally, this extension should enable using katacontainers as container runtime within Shoot clusters. Thus, you can isolate your workload easily into qemu / firecracker / cloudhypervisor VMs on the nodes, when this extension is installed.

Wait! How does this differ from kata-deploy?

Actually, it doesn't. Internally, the extension is using the same mechanism as provided by kata-deploy. However, when it comes to using katacontainers in combination with firecracker VMs, the machine image needs to be tweaked a bit. See also this how-to to get an idea of what needs to be configured for the usage of katacontainers with firecracker. This configuration overhead is handled by this extension. For this, the extension controller modifies the OperatingSystemConfiguration via a mutatingWebhookConfiguration so that the preparation of the nodes is performed. More precisely, an additional script for the devmapper setup is provided and executed by an additional systemd service unit. Moreover, the controller deploys some helm-charts containing the configuration and the daemonset provided by kata-deploy.

Current limitations

Generally, the limitations are two-fold. First, the cloud service provider your Shoot is running on needs to support nested virtualization for katacontainers to work properly. On the Azure cloud, nested virtualization is enabled for various machine types. As our development and testing environment was limited to Shoots hosted on Azure, the extension controller will only do something meaningful, when Shoots are spawned on Azure. Second, the implementation was only tested for Ubuntu as machine image. Thus, the controller will idle, when other machine images are selected. Of course, the support for other machine images can be implemented. However, this is a future topic, and this package should be considered as proof of concept.

How to...

Enable the extension as a gardener operator

Of course, you need to apply the controller-registration resources to the garden cluster first. You can find the corresponding yaml-files in our releases. Moreover, you will need to modify your Azure-cloudprofile so that it contains the following configuration:

...
  machineImages:
  - name: ubuntu
    versions:
    - cri:
      - containerRuntimes:
        - type: kata
        name: containerd
      - name: docker
      version: 18.4.20210415
...

Use it as end user

If you have access rights to the garden cluster, go ahead and follow the instructions in Enable the extension as a gardener operator. Otherwise, ask your gardener operator to do so. Once the extension is installed, you should be able to select kata as additional runtime from the Gardener dashboard during shoot creation or simply specify the corresponding container runtime in your Shoot.Spec. To run some example workload in your Shoot cluster you can use the sample workloads from kata-deploy.