⚠️ Discontinue Kube RBAC Proxy in Default Kubebuilder Scaffolding #3899

camilamacedo86 · 2024-05-06T15:50:06Z

Remove the Kube RBAC Proxy and update the docs accordingly so that we can have a latest release in v3x to warning users about the changes. More info: https://github.com/kubernetes-sigs/kubebuilder/blob/master/designs/discontinue_usage_of_kube_rbac_proxy.md

Closes: kube-rbac-proxy warn about deprecation and future breaking changes #3524
Closes: RFC: The kube-rbac-proxy is too opinionated to be opt-out. #3482
Partially: Migrate away from google.com gcp project kubebuilder #3230
Partially: [META - Phases]: Discontinue Kube RBAC Proxy in Default Kubebuilder Scaffolding #3871
Partially: Migrate away from google.com gcp project kubebuilder kubernetes/k8s.io#2647
Part of: [META - Phases]: Discontinue Kube RBAC Proxy in Default Kubebuilder Scaffolding #3871

**To understand why it is required see **: https://github.com/kubernetes-sigs/kubebuilder/blob/master/designs/discontinue_usage_of_kube_rbac_proxy.md

PS.: After the release we will add a discussion in this repo, prepare a guidance, communicate in the mailing list and channels.

More info: (Slack Threads)

To confirm the ETA: https://kubernetes.slack.com/archives/CCK68P2Q2/p1712622102206909
To discuss solutions: https://kubernetes.slack.com/archives/CCK68P2Q2/p1711913605487359

camilamacedo86 · 2024-05-06T15:57:13Z

docs/book/src/getting-started/testdata/project/config/default/kustomization.yaml

- path: manager_auth_proxy_patch.yaml
+# [METRICS] The following patch will enable the metrics endpoint. Ensure that you also protect this endpoint.
+# If you want to expose the metric endpoint of your controller-manager uncomment the following line.
+#- path: manager_metrics_patch.yaml


Metrics should not be enable by default as it was before.

docs/book/src/getting-started/testdata/project/config/prometheus/monitor.yaml

erikgb

I did a quick scan, and I think there is more to be cleaned up.

...ook/src/component-config-tutorial/testdata/project/config/default/manager_metrics_patch.yaml

docs/book/src/component-config-tutorial/testdata/project/config/rbac/metrics_role.yaml

docs/book/src/component-config-tutorial/testdata/project/config/rbac/metrics_role_binding.yaml

docs/book/src/component-config-tutorial/testdata/project/config/rbac/kustomization.yaml

camilamacedo86 · 2024-05-06T22:22:39Z

Hi @erikgb

Thank you a lot for your time and help.
It was amazing !!!

All points are addressed and I supplement the docs to let the users know how to protected the endpoint.
I hope that is good enough to fly now !!!

erikgb · 2024-05-07T05:30:24Z

...ook/src/component-config-tutorial/testdata/project/config/default/manager_metrics_patch.yaml

+      containers:
+      - name: manager
+        args:
+        - "--metrics-bind-address=0.0.0.0:8080"


I don't think this patch is needed to enable metrics. We have our metrics scraped by Prometheus without this arg set. Ref. controller-runtime argument help.

-metrics-bind-address string
The address the metric endpoint binds to. (default ":8080")

So if you want to have a patch to enable the metrics endpoint, you have to disable it by default by setting it to 0, ref. https://github.com/kubernetes-sigs/controller-runtime/blob/479b723944e34ae42c9911fe01228ff34eb5ca81/pkg/metrics/server/server.go#L120-L122

Oh. thank you a lot for check and share it.
I think ideally we should not enable by default.
Not everybody want to use it and by enable the metrics is required to protect the endpoint
So, it would be better if that is a conscious decision.

Done the changes and also added a test to ensure that the metrics endpoint will not be exposed in this case

It("should generate a runnable project without metrics exposed", func() { kbc.IsRestricted = false GenerateV4WithoutMetrics(kbc) Run(kbc, true, false, false) })

In this case, we are using the curl pod to ensure that the connection will be refused such as:

$ kubectl logs curl -n e2e-vbau-system % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 10.96.136.215:8080... * connect to 10.96.136.215 port 8080 failed: Connection refused * Failed to connect to e2e-vbau-controller-manager-metrics-service.e2e-vbau-system.svc.cluster.local port 8080 after 3 ms: Connection refused 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 * Closing connection 0 curl: (7) Failed to connect to e2e-vbau-controller-manager-metrics-service.e2e-vbau-system.svc.cluster.local port 8080 after 3 ms: Connection refused

Thank you a lot.

camilamacedo86 · 2024-05-07T08:54:39Z

testdata/project-v4-multigroup/config/manager/manager.yaml

-        - --leader-elect
+          - --leader-elect
+          - --health-probe-bind-address=:8081
+          - --metrics-bind-address=0


--metrics-bind-address=0 -> Used here to disable it by default

docs/book/src/reference/metrics.md

varshaprasad96

Thanks for putting this through @camilamacedo86!

/lgtm
/approve

Just a few nits, can be handled later too!

varshaprasad96 · 2024-05-14T23:40:20Z

docs/book/src/reference/metrics.md

+we now require the use of shared infrastructure. But as [`kube-rbac-proxy`](https://github.com/brancz/kube-rbac-proxy) is a process
+to be part of but not yet, sadly we cannot build and promote these images using the new


Suggested change

we now require the use of shared infrastructure. But as [`kube-rbac-proxy`](https://github.com/brancz/kube-rbac-proxy) is a process

to be part of but not yet, sadly we cannot build and promote these images using the new

we now require the use of shared infrastructure. But as [`kube-rbac-proxy`](https://github.com/brancz/kube-rbac-proxy) is in a process

to be a part of it, but not yet, sadly we cannot build and promote these images using the new

varshaprasad96 · 2024-05-14T23:50:58Z

pkg/plugins/common/kustomize/v2/scaffolds/api.go

 			fmt.Sprintf("\n- %[1]s_editor_role.yaml\n- %[1]s_viewer_role.yaml", crdName))
 		if err != nil {
 			log.Errorf("Unable to add Editor and Viewer roles in the file "+
 				"%s.", rbacKustomizeFilePath)
 		}
+		// Add an empty line at the end of the file
+		err = pluginutil.AppendCodeIfNotExist(rbacKustomizeFilePath,


Just curious, why is appending an empty line at the end of this brace file necessary?

To avoid breaking changes. So, it can work with scaffolds that has the kube-rbac-proxy or not.

k8s-ci-robot · 2024-05-15T00:01:23Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: camilamacedo86, varshaprasad96

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [camilamacedo86,varshaprasad96]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot · 2024-05-15T07:39:31Z

New changes are detected. LGTM label has been removed.

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels May 6, 2024

k8s-ci-robot requested review from Kavinjsir and varshaprasad96 May 6, 2024 15:50

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 6, 2024

camilamacedo86 force-pushed the discontinue-rbac branch from aa27c06 to 93a7dbf Compare May 6, 2024 15:55

camilamacedo86 commented May 6, 2024

View reviewed changes

docs/book/src/getting-started/testdata/project/config/prometheus/monitor.yaml Outdated Show resolved Hide resolved

camilamacedo86 added the release-blocker label May 6, 2024

camilamacedo86 mentioned this pull request May 6, 2024

WIP:✨ Phase 1: Add Network Policy #3853

Open

camilamacedo86 requested review from erikgb and brancz May 6, 2024 16:04

camilamacedo86 mentioned this pull request May 6, 2024

[META - Phases]: Discontinue Kube RBAC Proxy in Default Kubebuilder Scaffolding #3871

Open

5 tasks

camilamacedo86 changed the title ~~⚠️ Discontinue Kube RBAC Proxy in Default Kubebuilder Scaffolding~~ WIP: ⚠️ Discontinue Kube RBAC Proxy in Default Kubebuilder Scaffolding May 6, 2024

k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 6, 2024

camilamacedo86 force-pushed the discontinue-rbac branch 2 times, most recently from 92b585f to 00f5ecf Compare May 6, 2024 17:56

camilamacedo86 changed the title ~~WIP: ⚠️ Discontinue Kube RBAC Proxy in Default Kubebuilder Scaffolding~~ ⚠️ Discontinue Kube RBAC Proxy in Default Kubebuilder Scaffolding May 6, 2024

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 6, 2024

erikgb suggested changes May 6, 2024

View reviewed changes

camilamacedo86 force-pushed the discontinue-rbac branch 3 times, most recently from aba519b to cc85d34 Compare May 6, 2024 22:20

camilamacedo86 requested a review from erikgb May 6, 2024 22:22

erikgb reviewed May 7, 2024

View reviewed changes

camilamacedo86 force-pushed the discontinue-rbac branch from cc85d34 to 1aee283 Compare May 7, 2024 08:52

camilamacedo86 requested a review from erikgb May 7, 2024 08:53

camilamacedo86 commented May 7, 2024

View reviewed changes

camilamacedo86 force-pushed the discontinue-rbac branch from 1aee283 to f740067 Compare May 7, 2024 08:59

camilamacedo86 force-pushed the discontinue-rbac branch 6 times, most recently from ab83bd8 to 5d38a92 Compare May 7, 2024 18:13

camilamacedo86 mentioned this pull request May 14, 2024

Add support for go 1.22 #3792

Closed

camilamacedo86 commented May 14, 2024

View reviewed changes

docs/book/src/reference/metrics.md Outdated Show resolved Hide resolved

camilamacedo86 commented May 14, 2024

View reviewed changes

docs/book/src/reference/metrics.md Outdated Show resolved Hide resolved

camilamacedo86 commented May 14, 2024

View reviewed changes

docs/book/src/reference/metrics.md Outdated Show resolved Hide resolved

camilamacedo86 commented May 14, 2024

View reviewed changes

docs/book/src/reference/metrics.md Outdated Show resolved Hide resolved

camilamacedo86 force-pushed the discontinue-rbac branch 2 times, most recently from 9efea29 to b332e82 Compare May 14, 2024 23:56

varshaprasad96 approved these changes May 15, 2024

View reviewed changes

k8s-ci-robot assigned varshaprasad96 May 15, 2024

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 15, 2024

camilamacedo86 force-pushed the discontinue-rbac branch from b332e82 to bbb155e Compare May 15, 2024 07:39

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 15, 2024

camilamacedo86 force-pushed the discontinue-rbac branch 4 times, most recently from 8d39b62 to 6ddf84b Compare May 15, 2024 08:07

Discontinue Kube RBAC Proxy in Default Kubebuilder Scaffolding

973a600

camilamacedo86 force-pushed the discontinue-rbac branch from 6ddf84b to 973a600 Compare May 15, 2024 08:09

camilamacedo86 merged commit c01af8f into kubernetes-sigs:master May 15, 2024
16 of 18 checks passed

camilamacedo86 deleted the discontinue-rbac branch May 15, 2024 10:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

⚠️ Discontinue Kube RBAC Proxy in Default Kubebuilder Scaffolding #3899

⚠️ Discontinue Kube RBAC Proxy in Default Kubebuilder Scaffolding #3899

camilamacedo86 commented May 6, 2024 •

edited

camilamacedo86 May 6, 2024

erikgb left a comment

camilamacedo86 commented May 6, 2024

erikgb May 7, 2024

camilamacedo86 May 7, 2024

camilamacedo86 May 7, 2024 •

edited

camilamacedo86 May 7, 2024

varshaprasad96 left a comment

varshaprasad96 May 14, 2024

camilamacedo86 May 15, 2024

varshaprasad96 May 14, 2024

camilamacedo86 May 15, 2024

k8s-ci-robot commented May 15, 2024

k8s-ci-robot commented May 15, 2024

		we now require the use of shared infrastructure. But as [`kube-rbac-proxy`](https://github.com/brancz/kube-rbac-proxy) is a process
		to be part of but not yet, sadly we cannot build and promote these images using the new

⚠️ Discontinue Kube RBAC Proxy in Default Kubebuilder Scaffolding #3899

⚠️ Discontinue Kube RBAC Proxy in Default Kubebuilder Scaffolding #3899

Conversation

camilamacedo86 commented May 6, 2024 • edited

Choose a reason for hiding this comment

erikgb left a comment

Choose a reason for hiding this comment

camilamacedo86 commented May 6, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

camilamacedo86 May 7, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

varshaprasad96 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

k8s-ci-robot commented May 15, 2024

k8s-ci-robot commented May 15, 2024

camilamacedo86 commented May 6, 2024 •

edited

camilamacedo86 May 7, 2024 •

edited