Skip to content

Network Design

Jump to bottom
Fabian Deutsch edited this page Jul 17, 2018 · 1 revision

Goals

  1. Allow us to seamlessly tie in with all Kubernetes networking which is intended for usual container workloads, regardless if it's a node level feature (Istio et al) or cluster level (Ingress, Service, NetworkPolicy)

  2. Allow us to provide additional network interfaces to the cluster network (via mechanisms like multus) or to non-cluster networks (via i.e. device plugins)

  3. Allow 1. and 2. to be used in parallel

Use-cases

  1. Compatible
    1. Allow usage of Istio
    2. Allow usage of multus
  2. Advanced
    1. Allow to provide L2 connectivity to the VM
  3. Both
    1. Allow usage of Istio for the cluster network in addition to L2 connectivity for the VM

Requirements

  1. Be compatible with how Kubernetes does networking in order to be able to leverage all tooling around networkin in the Kubernetes landscape without tension and conflicts

  2. Provide advanced networking bypassing Kubernetes networking model in order to get into a situation which is closer to how traditional virtualization management work

Foundation

The foundation for our networking is to have an API which allows to

  • specify multiple interfaces per VM
  • let the user choose different backends and connection methods for each interface

A backend could be i.e. a device plugin or multus, a connection method could be passthrough, bridged, or even proxied. Obviously the connection methods depend on backends.

Approaches

1. Kubernetes Compatible (100%)

The technical requirement to be 100% compatible with Kubernetes assumption about networking is, to let the VM (from a networking perspective) behave like a process. Thus all traffic is going in and out of TCP or UDP sockets. The approach especially needs to avoid to rely on any specific Kernel configurations (like iptables, ebtables, tc, …) in order to not conflict with other Kubernetes networking tools like Istio or multus.

The currently in focus approach is to use SLRIP networking. But due to it's shortcomings (unmaintained, unsafe, not performing well), this is just an intermediate solution.

Research for alternatives is going on.

1. Kubernetes Compatible (less than 100%)

As 1., just with a relaxed constraint on Kernel configuration. Thus in this approach we would allow certain Kernel configurations which give us more freedom to find a solution, but do bear the risk of conflicting with other networking tools (Istio, …). The mitigation is to handle known conflicts explicitly.

2. Advanced (DP)

Device plugins five us a great freedom in how we create a network interface in a Kubernetes pod. Currently it is a swiss army knife and we'll be using it to provide L2 connectivity to a VM in a simple fashion (bridged with vlans).

DP provided interfaces can be connected to the VM interfaces using the VM API.

2. Advanced (multus)

Multus implements support for the de-facto standard Network CRD. A network backend can be written which can consume multus (or Network CRDs in general, not yet clear how exactly).

Multus provided interfaces can be connected to the VM interfaces using the VM API.

Clone this wiki locally