⚠️ (go/v3 and config-gen alpha command) Update controller-runtime dep from v0.11.2 to v0.12.1, controller-tools from v0.8.0 to v0.9.0 and k8s deps from 1.23 to 1.24.1 #2679

NikhilSharmaWe · 2022-05-11T19:46:48Z

The new release is bumped with k8s 1.24 and go 1.18.
Users that might do the upgrade will need to explicitly set configmapsleases when configuring leader election, more info: ⚠ leaderelection: use 'leases' as default resource lock object controller-runtime#1773 if you are (migrating from v2 to v3).
Check out https://github.com/kubernetes-sigs/controller-runtime/releases/tag/v0.12.0 for more info about the new release.

Closes: #2681

camilamacedo86

Thank you for the contribution 🥇
See the comment: https://github.com/kubernetes-sigs/kubebuilder/pull/2679/files#r871211532

NikhilSharmaWe · 2022-05-12T04:52:32Z

@camilamacedo86 Unit tests are failing
Is it due to the versions of k8s.io/cli-runtime@v0.23.0 and k8s.io/kubectl@v0.22.2.

All error logs are like:

Error: cannot use oapi (type *"github.com/google/gnostic/openapiv2".Document) as type *"github.com/googleapis/gnostic/openapiv2".Document in argument to supportsDryRun

go.mod

camilamacedo86 · 2022-05-20T07:30:14Z

This PR is blocked by upgrading the declarative project: kubernetes-sigs/kubebuilder-declarative-pattern#217.

PS.: The first version of go/v3 uses controller-runtime v0.7.2 ( see: https://github.com/kubernetes-sigs/kubebuilder/blob/v3.0.0/testdata/project-v3/go.mod#L12 ), so the changes described in kubernetes-sigs/controller-runtime#1903 is fine. However, we will need to update the migration guide doc from v2 to v3 with: kubernetes-sigs/controller-runtime#1903

everettraven · 2022-06-03T13:48:18Z

Unit tests are failing Is it due to the versions of k8s.io/cli-runtime@v0.23.0 and k8s.io/kubectl@v0.22.2.

All error logs are like:
Error: cannot use oapi (type *"github.com/google/gnostic/openapiv2".Document) as type *"github.com/googleapis/gnostic/openapiv2".Document in argument to supportsDryRun

@NikhilSharmaWe these errors seem to be related to the kubebuilder-declarative-pattern dependency, so since your PR to bump it to support k8s 1.24 was merged, I think this line needs to be modified:

kubebuilder/pkg/plugins/golang/declarative/v1/api.go

Line 35 in c7d57aa

kbDeclarativePatternForV3 = "d0f104b6a96e152043e9c2d76229373a981ac96a"

to be:

kbDeclarativePatternForV3 = "fe5be9431eae158f86f9de23000a9a2ec06745fc"

everettraven · 2022-06-03T18:48:27Z

@NikhilSharmaWe I think I may have found the problem that is causing the tests to fail with the proto file issue.

Problem

The problem seems to be because the go.mod dependencies are being populated with indirect dependencies that map to the k8s packages with a version of v0.24.0. When the dependencies for the v1beta1 support are modified, the indirect dependencies aren't properly updated without removing all the indirect dependencies before running go mod tidy.

The exact indirect dependency that is causing the issue is github.com/google/gnostic v0.5.7-v3refs // indirect because client-go v0.24.0 and controller-runtime v0.12.1 have this dependency causing the scaffolded project to have it as an indirect dependency. client-go v0.21.2 and controller-runtime v0.9.2 use github.com/googleapis/gnostic v0.5.5 // indirect which is a different package path entirely.

Solution

The solution I found is to modify the pkg/plugins/golang/v3/commons.go file to remove all the indirect dependencies before running go mod tidy. This logic looks like:

// Due to some indirect dependency changes with a bump in k8s packages from 0.23.x --> 0.24.x we need to
// clear out all indirect dependencies before we run `go mod tidy` so that the above changes get resolved correctly
if err := util.ReplaceRegexInFile("go.mod", `(require \(\n(\t.* \/\/ indirect\n)+\))`, ""); err != nil {
	log.Warnf("unable to update the go.mod indirect dependencies: %s", err)
}

I will also leave a PR review comment around the area in the file it should be added.

everettraven · 2022-06-03T18:50:45Z

pkg/plugins/golang/v3/commons.go

@@ -145,7 +145,7 @@ manifests: controller-gen`
 		}

 		if err := util.ReplaceInFile("go.mod",
-			"k8s.io/klog/v2 v2.30.0",
+			"k8s.io/klog/v2 v2.60.1",
 			"k8s.io/klog/v2 v2.9.0"); err != nil {
 			log.Warnf("unable to update the go.mod with k8s.io/klog/v2 v2.9.0: %s", err)
 		}


After this line we should add:

// Due to some indirect dependency changes with a bump in k8s packages from 0.23.x --> 0.24.x we need to // clear out all indirect dependencies before we run `go mod tidy` so that the above changes get resolved correctly if err := util.ReplaceRegexInFile("go.mod", `(require \(\n(\t.* \/\/ indirect\n)+\))`, ""); err != nil { log.Warnf("unable to update the go.mod indirect dependencies: %s", err) }

The reasoning for this change can be seen here: #2679 (comment)

everettraven · 2022-06-03T18:52:52Z

go.mod

-	sigs.k8s.io/controller-runtime v0.11.2
+	golang.org/x/tools v0.1.10-0.20220218145154-897bd77cd717
+	k8s.io/apimachinery v0.24.0 // for `kubebuilder alpha config-gen`
+	sigs.k8s.io/controller-runtime v0.12.1
 	sigs.k8s.io/controller-tools v0.8.0 // for `kubebuilder alpha config-gen`


controller-tools should be upgraded to v0.9.0 since it doesn't support k8s 1.24 until that release. More info on the v0.9.0 release can be found here: https://github.com/kubernetes-sigs/controller-tools/releases/tag/v0.9.0

Suggested change

sigs.k8s.io/controller-tools v0.8.0 // for `kubebuilder alpha config-gen`

sigs.k8s.io/controller-tools v0.9.0 // for `kubebuilder alpha config-gen`

@NikhilSharmaWe,
We need also add this info in the title and description
I am doing tha

everettraven · 2022-06-03T20:51:32Z

@NikhilSharmaWe it looks like the new unit test failures are related to an update made in controller-tools v0.9.0 that no longer writes the status field in the CRD. The PR that made this change is here: kubernetes-sigs/controller-tools#630

Due to this we need to update the expected files to match the new format. To do this you need to:

uncomment this line:

kubebuilder/pkg/cli/alpha/config-gen/configgen_test.go

Line 30 in 019c185

// UpdateExpectedFromActual: true,
In the pkg/cli/alpha/config-gen directory, run go test
comment the changed line again

After that the unit tests should pass again.

…om v0.11.2 to v0.12.1

everettraven

Looks good to me, great job @NikhilSharmaWe !

everettraven · 2022-06-06T13:23:52Z

/cc @joelanford

Could you take a look at this so we can try and get this in ASAP? Thanks!

jmrodri · 2022-06-06T18:55:48Z

What does this error mean?

sigs.k8s.io/kubebuilder/v3/pkg/plugins/golang/v3/scaffolds
  Incompatible changes:
  - ControllerRuntimeVersion: value changed from "v0.11.2" to "v0.12.1"

varshaprasad96 · 2022-06-06T19:17:19Z

What does this error mean?

sigs.k8s.io/kubebuilder/v3/pkg/plugins/golang/v3/scaffolds
  Incompatible changes:
  - ControllerRuntimeVersion: value changed from "v0.11.2" to "v0.12.1"

That seems to be because of an APIDiff check - modifying an exported field. If someone with permissions could override that check, we can merge it.

everettraven · 2022-06-06T19:23:42Z

What does this error mean?

sigs.k8s.io/kubebuilder/v3/pkg/plugins/golang/v3/scaffolds
  Incompatible changes:
  - ControllerRuntimeVersion: value changed from "v0.11.2" to "v0.12.1"

@jmrodri from what I can tell it seems to just be stating that the bump to v0.12.1 is a breaking change from v0.11.2 which we already knew. As @varshaprasad96 mentioned, I think this can be safely overridden.

jmrodri

/lgtm

k8s-ci-robot · 2022-06-06T21:29:08Z

@jmrodri: changing LGTM is restricted to collaborators

In response to this:

/lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

jmrodri · 2022-06-06T21:29:14Z

Looks pretty straight forward to me.

camilamacedo86 · 2022-06-07T11:07:37Z

pkg/cli/alpha/config-gen/testdata/componentconfig/expected.yaml

-    kind: ""
-    plural: ""
-  conditions: []
-  storedVersions: []


why it was removed?
Do we know that?

This is due to this PR: kubernetes-sigs/controller-tools#630

It changed the generation of a CRD to no longer include the status fields. This PR was included in the v0.9.0 release of controller-tools. Here is the link to the release: https://github.com/kubernetes-sigs/controller-tools/releases/tag/v0.9.0

camilamacedo86 · 2022-06-07T11:12:11Z

pkg/plugins/golang/v3/commons.go

+		// clear out all indirect dependencies before we run `go mod tidy` so that the above changes get resolved correctly
+		if err := util.ReplaceRegexInFile("go.mod", `(require \(\n(\t.* \/\/ indirect\n)+\))`, ""); err != nil {
+			log.Warnf("unable to update the go.mod indirect dependencies: %s", err)
+		}


On top we are been very precisely about what dep we should downgrade
So, why do we need to add this one?
I think is better we just update what we scaffold by default instead of all it might change things that were not done in the default scaffold.

This was a change that I recommended. I think it is fine if we want to change this to only include changing the dependency that was causing an issue, however I will explain my reasoning for the recommendation below.

Reasoning for recommended change

The indirect dependencies in the go.mod seem to be getting populated with the indirect dependencies that correlate to the base scaffolding (e.g. k8s 1.24 dependencies) and are not properly updated when the direct dependency versions are downgraded. When I tested this locally, running go mod tidy did not update the indirect dependencies when downgrading direct dependencies.

As a result of the above, the indirect dependencies that are being used may be incompatible with the versions that the direct dependencies are using. I think we should be ensuring that the indirect dependencies that are being used are the same as those that are used when manually creating a new go.mod that contains the same direct dependencies. The best way I found to do this with Kubebuilder's scaffolding is to remove all the indirect dependencies and then run go mod tidy to repopulate them. This also helps to ensure that all the indirect dependencies will be compatible with the expectations of the downgraded direct dependencies.

@camilamacedo86 WDYT? If this should be considered out of the scope of this PR and this is something we should still change, I am happy to create an issue to track this and just have this PR fix the dependencies it needs to fix.

@everettraven I am ok with we better address it as you suggest in a follow up since it is only used with the deprecated flag --crd-version=v1beta1 which no longer works with k8s >= 1.22.

camilamacedo86

/lgtm
/approved

k8s-ci-robot · 2022-06-07T13:48:15Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: camilamacedo86, everettraven, jmrodri, NikhilSharmaWe

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [camilamacedo86]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels May 11, 2022

k8s-ci-robot requested review from camilamacedo86 and estroz May 11, 2022 19:47

NikhilSharmaWe force-pushed the newReleaseUpdate branch from 61a2f55 to 140a2b0 Compare May 11, 2022 19:51

camilamacedo86 requested changes May 12, 2022

View reviewed changes

NikhilSharmaWe force-pushed the newReleaseUpdate branch from 140a2b0 to b17731d Compare May 12, 2022 04:05

camilamacedo86 reviewed May 12, 2022

View reviewed changes

go.mod Show resolved Hide resolved

NikhilSharmaWe mentioned this pull request May 17, 2022

✨ (declarative/v1) : Update the sigs.k8s.io/kubebuilder-declarative-pattern dep used for projects scaffolded with go/v3 plugin from fea7e5cc701290589ec20ef4d9c0629d08b5307d to d0f104b6a96e152043e9c2d76229373a981ac96a #2682

Merged

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 18, 2022

everettraven mentioned this pull request Jun 1, 2022

meta issue for k8s 1.24 bump operator-framework/operator-sdk#5820

Closed

13 tasks

NikhilSharmaWe force-pushed the newReleaseUpdate branch from b17731d to 9214963 Compare June 3, 2022 14:53

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 3, 2022

NikhilSharmaWe force-pushed the newReleaseUpdate branch 3 times, most recently from aba99f8 to 5528cde Compare June 3, 2022 15:28

NikhilSharmaWe changed the title ~~⚠️ (go/v3 and config-gen alpha command) Update controller-runtime dep from v0.11.2 to v0.12.0.~~ ⚠️ (go/v3 and config-gen alpha command) Update controller-runtime dep from v0.11.2 to v0.12.1. Jun 3, 2022

NikhilSharmaWe force-pushed the newReleaseUpdate branch from 5528cde to 521985a Compare June 3, 2022 15:30

everettraven suggested changes Jun 3, 2022

View reviewed changes

NikhilSharmaWe force-pushed the newReleaseUpdate branch from 521985a to 1421ff9 Compare June 3, 2022 19:12

k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 3, 2022

NikhilSharmaWe force-pushed the newReleaseUpdate branch from 1421ff9 to 4b37fe7 Compare June 4, 2022 04:39

(go/v3 and config-gen alpha command) Update controller-runtime dep fr…

98e8df7

…om v0.11.2 to v0.12.1

NikhilSharmaWe force-pushed the newReleaseUpdate branch from 4b37fe7 to 98e8df7 Compare June 4, 2022 04:50

everettraven approved these changes Jun 6, 2022

View reviewed changes

k8s-ci-robot requested a review from joelanford June 6, 2022 13:23

jmrodri approved these changes Jun 6, 2022

View reviewed changes

camilamacedo86 reviewed Jun 7, 2022

View reviewed changes

camilamacedo86 approved these changes Jun 7, 2022

View reviewed changes

k8s-ci-robot assigned camilamacedo86 Jun 7, 2022

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 7, 2022

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 7, 2022

camilamacedo86 merged commit f19b01d into kubernetes-sigs:master Jun 7, 2022

everettraven mentioned this pull request Jun 7, 2022

Improve the implementation of the v1beta1 CRD dependency downgrade logic #2710

Closed

camilamacedo86 mentioned this pull request Jun 8, 2022

⚠️ : (go/v3) upgrade controller-gen from 0.8.0 to 0.9.0 #2715

Merged

everettraven mentioned this pull request Jun 24, 2022

REQUEST: New membership for @everettraven kubernetes/org#3522

Closed

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

⚠️ (go/v3 and config-gen alpha command) Update controller-runtime dep from v0.11.2 to v0.12.1, controller-tools from v0.8.0 to v0.9.0 and k8s deps from 1.23 to 1.24.1 #2679

⚠️ (go/v3 and config-gen alpha command) Update controller-runtime dep from v0.11.2 to v0.12.1, controller-tools from v0.8.0 to v0.9.0 and k8s deps from 1.23 to 1.24.1 #2679

NikhilSharmaWe commented May 11, 2022 •

edited by camilamacedo86

camilamacedo86 left a comment •

edited

NikhilSharmaWe commented May 12, 2022 •

edited

camilamacedo86 commented May 20, 2022 •

edited

everettraven commented Jun 3, 2022

everettraven commented Jun 3, 2022

everettraven Jun 3, 2022

everettraven Jun 3, 2022

camilamacedo86 Jun 7, 2022

everettraven commented Jun 3, 2022

everettraven left a comment

everettraven commented Jun 6, 2022

jmrodri commented Jun 6, 2022

varshaprasad96 commented Jun 6, 2022 •

edited

everettraven commented Jun 6, 2022

jmrodri left a comment

k8s-ci-robot commented Jun 6, 2022

jmrodri commented Jun 6, 2022

camilamacedo86 Jun 7, 2022

everettraven Jun 7, 2022

camilamacedo86 Jun 7, 2022 •

edited

everettraven Jun 7, 2022 •

edited

camilamacedo86 Jun 7, 2022

camilamacedo86 left a comment

k8s-ci-robot commented Jun 7, 2022

	sigs.k8s.io/controller-tools v0.8.0 // for `kubebuilder alpha config-gen`
	sigs.k8s.io/controller-tools v0.9.0 // for `kubebuilder alpha config-gen`

⚠️ (go/v3 and config-gen alpha command) Update controller-runtime dep from v0.11.2 to v0.12.1, controller-tools from v0.8.0 to v0.9.0 and k8s deps from 1.23 to 1.24.1 #2679

⚠️ (go/v3 and config-gen alpha command) Update controller-runtime dep from v0.11.2 to v0.12.1, controller-tools from v0.8.0 to v0.9.0 and k8s deps from 1.23 to 1.24.1 #2679

Conversation

NikhilSharmaWe commented May 11, 2022 • edited by camilamacedo86

camilamacedo86 left a comment • edited

Choose a reason for hiding this comment

NikhilSharmaWe commented May 12, 2022 • edited

camilamacedo86 commented May 20, 2022 • edited

everettraven commented Jun 3, 2022

everettraven commented Jun 3, 2022

Problem

Solution

everettraven Jun 3, 2022

Choose a reason for hiding this comment

everettraven Jun 3, 2022

Choose a reason for hiding this comment

camilamacedo86 Jun 7, 2022

Choose a reason for hiding this comment

everettraven commented Jun 3, 2022

everettraven left a comment

Choose a reason for hiding this comment

everettraven commented Jun 6, 2022

jmrodri commented Jun 6, 2022

varshaprasad96 commented Jun 6, 2022 • edited

everettraven commented Jun 6, 2022

jmrodri left a comment

Choose a reason for hiding this comment

k8s-ci-robot commented Jun 6, 2022

jmrodri commented Jun 6, 2022

camilamacedo86 Jun 7, 2022

Choose a reason for hiding this comment

everettraven Jun 7, 2022

Choose a reason for hiding this comment

camilamacedo86 Jun 7, 2022 • edited

Choose a reason for hiding this comment

everettraven Jun 7, 2022 • edited

Choose a reason for hiding this comment

camilamacedo86 Jun 7, 2022

Choose a reason for hiding this comment

camilamacedo86 left a comment

Choose a reason for hiding this comment

k8s-ci-robot commented Jun 7, 2022

NikhilSharmaWe commented May 11, 2022 •

edited by camilamacedo86

camilamacedo86 left a comment •

edited

NikhilSharmaWe commented May 12, 2022 •

edited

camilamacedo86 commented May 20, 2022 •

edited

varshaprasad96 commented Jun 6, 2022 •

edited

camilamacedo86 Jun 7, 2022 •

edited

everettraven Jun 7, 2022 •

edited