Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

⚠️ ComponentConfig Implementation #891

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 0 additions & 1 deletion .golangci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ linters:
- unparam
- ineffassign
- nakedret
- interfacer
- gocyclo
- lll
- dupl
Expand Down
8 changes: 8 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ TOOLS_DIR := hack/tools
TOOLS_BIN_DIR := $(TOOLS_DIR)/bin
GOLANGCI_LINT := $(abspath $(TOOLS_BIN_DIR)/golangci-lint)
GO_APIDIFF := $(TOOLS_BIN_DIR)/go-apidiff
CONTROLLER_GEN := $(TOOLS_BIN_DIR)/controller-gen

# The help will print out all targets with their descriptions organized bellow their categories. The categories are represented by `##@` and the target descriptions by `##`.
# The awk commands is responsible to read the entire set of makefiles included in this invocation, looking for lines of the file as xyz: ## something, and then pretty-format the target and help. Then, if there's a line with ##@ something, that gets pretty-printed as a category.
Expand Down Expand Up @@ -66,6 +67,9 @@ $(GOLANGCI_LINT): $(TOOLS_DIR)/go.mod # Build golangci-lint from tools folder.
$(GO_APIDIFF): $(TOOLS_DIR)/go.mod # Build go-apidiff from tools folder.
cd $(TOOLS_DIR) && go build -tags=tools -o bin/go-apidiff github.com/joelanford/go-apidiff

$(CONTROLLER_GEN): $(TOOLS_DIR)/go.mod # Build controller-gen from tools folder.
cd $(TOOLS_DIR) && go build -tags=tools -o bin/controller-gen sigs.k8s.io/controller-tools/cmd/controller-gen

## --------------------------------------
## Linting
## --------------------------------------
Expand All @@ -83,6 +87,10 @@ modules: ## Runs go mod to ensure modules are up to date.
go mod tidy
cd $(TOOLS_DIR); go mod tidy

.PHONY: generate
generate: $(CONTROLLER_GEN) ## Runs controller-gen for internal types for config file
$(CONTROLLER_GEN) object paths="./pkg/config/v1alpha1/...;./examples/configfile/custom/v1alpha1/..."

## --------------------------------------
## Cleanup / Verification
## --------------------------------------
Expand Down
40 changes: 20 additions & 20 deletions designs/component-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ This change is important because:

- Provide an interface for pulling configuration data out of exposed `ComponentConfig` types (see below for implementation)
- Provide a new `ctrl.NewFromComponentConfig()` function for initializing a manager
- Provide an embeddable `ControllerConfiguration` type for easily authoring `ComponentConfig` types
- Provide an embeddable `ControllerManagerConfiguration` type for easily authoring `ComponentConfig` types
- Provide an `DefaultControllerConfig` to make the switching easier for clients

### Non-Goals/Future Work
Expand Down Expand Up @@ -137,8 +137,8 @@ import (
configv1alpha1 "k8s.io/component-base/config/v1alpha1"
)

// ControllerConfiguration defines the embedded RuntimeConfiguration for controller-runtime clients.
type ControllerConfiguration struct {
// ControllerManagerConfiguration defines the embedded RuntimeConfiguration for controller-runtime clients.
type ControllerManagerConfiguration struct {
Namespace string `json:"namespace,omitempty"`

SyncPeriod *time.Duration `json:"syncPeriod,omitempty"`
Expand All @@ -147,16 +147,16 @@ type ControllerConfiguration struct {

MetricsBindAddress string `json:"metricsBindAddress,omitempty"`

Health ControllerConfigurationHealth `json:"health,omitempty"`
Health ControllerManagerConfigurationHealth `json:"health,omitempty"`

Port *int `json:"port,omitempty"`
Host string `json:"host,omitempty"`

CertDir string `json:"certDir,omitempty"`
}

// ControllerConfigurationHealth defines the health configs
type ControllerConfigurationHealth struct {
// ControllerManagerConfigurationHealth defines the health configs
type ControllerManagerConfigurationHealth struct {
HealthProbeBindAddress string `json:"healthProbeBindAddress,omitempty"`

ReadinessEndpointName string `json:"readinessEndpointName,omitempty"`
Expand All @@ -181,11 +181,11 @@ import (
configv1alpha1 "sigs.k8s.io/controller-runtime/pkg/apis/config/v1alpha1"
)

// DefaultControllerConfiguration is the Schema for the DefaultControllerConfigurations API
type DefaultControllerConfiguration struct {
// DefaultControllerManagerConfiguration is the Schema for the DefaultControllerManagerConfigurations API
type DefaultControllerManagerConfiguration struct {
metav1.TypeMeta `json:",inline"`

Spec configv1alpha1.ControllerConfiguration `json:"spec,omitempty"`
Spec configv1alpha1.ControllerManagerConfiguration `json:"spec,omitempty"`
}
```

Expand All @@ -194,19 +194,19 @@ This would allow a controller author to use this struct with any config that sup
```yaml
# config.yaml
apiVersion: config.somedomain.io/v1alpha1
kind: FoobarControllerConfiguration
kind: FoobarControllerManagerConfiguration
spec:
port: 9443
metricsBindAddress: ":8080"
leaderElection:
leaderElect: false
```

Given the following config and `DefaultControllerConfiguration` we'd be able to initialize the controller using the following.
Given the following config and `DefaultControllerManagerConfiguration` we'd be able to initialize the controller using the following.


```golang
mgr, err := ctrl.NewManagerFromComponentConfig(ctrl.GetConfigOrDie(), scheme, configname, &defaultv1alpha1.DefaultControllerConfiguration{})
mgr, err := ctrl.NewManagerFromComponentConfig(ctrl.GetConfigOrDie(), scheme, configname, &defaultv1alpha1.DefaultControllerManagerConfiguration{})
if err != nil {
// ...
}
Expand All @@ -222,8 +222,8 @@ Since this design still requires setting up the initial `ComponentConfig` type a
```golang
leaderElect := true

config := &defaultv1alpha1.DefaultControllerConfiguration{
Spec: configv1alpha1.ControllerConfiguration{
config := &defaultv1alpha1.DefaultControllerManagerConfiguration{
Spec: configv1alpha1.ControllerManagerConfiguration{
LeaderElection: configv1alpha1.LeaderElectionConfiguration{
LeaderElect: &leaderElect,
},
Expand All @@ -250,7 +250,7 @@ import (
)

type ControllerNameConfigurationSpec struct {
configv1alpha1.ControllerConfiguration `json:",inline"`
configv1alpha1.ControllerManagerConfiguration `json:",inline"`
}

type ControllerNameConfiguration struct {
Expand All @@ -260,20 +260,20 @@ type ControllerNameConfiguration struct {
}
```

Usage of this custom `ComponentConfig` type would require then changing the `ctrl.NewFromComponentConfig()` to use the new struct vs the `DefaultControllerConfiguration`.
Usage of this custom `ComponentConfig` type would require then changing the `ctrl.NewFromComponentConfig()` to use the new struct vs the `DefaultControllerManagerConfiguration`.

## User Stories

### Controller Author with `controller-runtime` and default type

- Mount `ConfigMap`
- Initialize `ctrl.Manager` with `NewFromComponentConfig` with config name and `DefaultControllerConfiguration` type
- Initialize `ctrl.Manager` with `NewFromComponentConfig` with config name and `DefaultControllerManagerConfiguration` type
- Build custom controller as usual

### Controller Author with `controller-runtime` and custom type

- Implement `ComponentConfig` type
- Embed `ControllerConfiguration` type
- Embed `ControllerManagerConfiguration` type
- Mount `ConfigMap`
- Initialize `ctrl.Manager` with `NewFromComponentConfig` with config name and `ComponentConfig` type
- Build custom controller as usual
Expand Down Expand Up @@ -311,9 +311,9 @@ _Provided that the controller provides manifests_

- [x] 02/19/2020: Proposed idea in an issue or [community meeting]
- [x] 02/24/2020: Proposal submitted to `controller-runtime`
- [x] 03/02/2020: Updated with default `DefaultControllerConfiguration`
- [x] 03/02/2020: Updated with default `DefaultControllerManagerConfiguration`
- [x] 03/04/2020: Updated with embeddable `RuntimeConfig`
- [x] 03/10/2020: Updated embeddable name to `ControllerConfiguration`
- [x] 03/10/2020: Updated embeddable name to `ControllerManagerConfiguration`


<!-- Links -->
Expand Down
7 changes: 7 additions & 0 deletions examples/configfile/builtin/config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
apiVersion: controller-runtime.sigs.k8s.io/v1alpha1
kind: ControllerManagerConfiguration
cacheNamespace: default
metrics:
bindAddress: :9091
leaderElection:
leaderElect: false
74 changes: 74 additions & 0 deletions examples/configfile/builtin/controller.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
/*
Copyright 2018 The Kubernetes Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package main

import (
"context"
"fmt"

appsv1 "k8s.io/api/apps/v1"
"k8s.io/apimachinery/pkg/api/errors"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/log"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
)

// reconcileReplicaSet reconciles ReplicaSets
type reconcileReplicaSet struct {
// client can be used to retrieve objects from the APIServer.
client client.Client
}

// Implement reconcile.Reconciler so the controller can reconcile objects
var _ reconcile.Reconciler = &reconcileReplicaSet{}

func (r *reconcileReplicaSet) Reconcile(ctx context.Context, request reconcile.Request) (reconcile.Result, error) {
// set up a convenient log object so we don't have to type request over and over again
log := log.FromContext(ctx)

// Fetch the ReplicaSet from the cache
rs := &appsv1.ReplicaSet{}
err := r.client.Get(context.TODO(), request.NamespacedName, rs)
if errors.IsNotFound(err) {
log.Error(nil, "Could not find ReplicaSet")
return reconcile.Result{}, nil
}

if err != nil {
return reconcile.Result{}, fmt.Errorf("could not fetch ReplicaSet: %+v", err)
}

// Print the ReplicaSet
log.Info("Reconciling ReplicaSet", "container name", rs.Spec.Template.Spec.Containers[0].Name)

// Set the label if it is missing
if rs.Labels == nil {
rs.Labels = map[string]string{}
}
if rs.Labels["hello"] == "world" {
return reconcile.Result{}, nil
}

// Update the ReplicaSet
rs.Labels["hello"] = "world"
err = r.client.Update(context.TODO(), rs)
if err != nil {
return reconcile.Result{}, fmt.Errorf("could not write ReplicaSet: %+v", err)
}

return reconcile.Result{}, nil
}
72 changes: 72 additions & 0 deletions examples/configfile/builtin/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
/*
Copyright 2020 The Kubernetes Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package main

import (
"os"

appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/runtime"
clientgoscheme "k8s.io/client-go/kubernetes/scheme"
_ "k8s.io/client-go/plugin/pkg/client/auth/gcp"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client/config"
cfg "sigs.k8s.io/controller-runtime/pkg/config"
"sigs.k8s.io/controller-runtime/pkg/log"
"sigs.k8s.io/controller-runtime/pkg/log/zap"
"sigs.k8s.io/controller-runtime/pkg/manager/signals"
)

var scheme = runtime.NewScheme()

func init() {
log.SetLogger(zap.New())
clientgoscheme.AddToScheme(scheme)
}

func main() {
entryLog := log.Log.WithName("entrypoint")

// Setup a Manager
entryLog.Info("setting up manager")
mgr, err := ctrl.NewManager(config.GetConfigOrDie(), ctrl.Options{
Scheme: scheme,
}.AndFromOrDie(cfg.File()))
if err != nil {
entryLog.Error(err, "unable to set up overall controller manager")
os.Exit(1)
}

// Setup a new controller to reconcile ReplicaSets
err = ctrl.NewControllerManagedBy(mgr).
For(&appsv1.ReplicaSet{}).
Owns(&corev1.Pod{}).
Complete(&reconcileReplicaSet{
client: mgr.GetClient(),
})
if err != nil {
entryLog.Error(err, "unable to create controller")
os.Exit(1)
}

entryLog.Info("starting manager")
if err := mgr.Start(signals.SetupSignalHandler()); err != nil {
entryLog.Error(err, "unable to run manager")
os.Exit(1)
}
}
8 changes: 8 additions & 0 deletions examples/configfile/custom/config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
apiVersion: examples.x-k8s.io/v1alpha1
kind: CustomControllerManagerConfiguration
clusterName: example-test
cacheNamespace: default
metrics:
bindAddress: :8081
leaderElection:
leaderElect: false