Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ssa: apply hpa drift detection fix to autoscaling/v2 #449

Merged
merged 1 commit into from Jan 25, 2023

Conversation

somtochiama
Copy link
Member

@somtochiama somtochiama commented Jan 11, 2023

Fixes: #437
Fixes: fluxcd/flux2#3277

The drift detection fix introduced here: #207 was only applied to autoscaling/v2beta2.
This PR adds the autoscaling/v2 apiVersion since the same issue still persists and updates client-go to 1.26

Signed-off-by: Somtochi Onyekwere somtochionyekwere@gmail.com

ssa/utils.go Outdated
@@ -448,6 +448,7 @@ func fixHorizontalPodAutoscaler(object *unstructured.Unstructured) error {
if object.GetKind() == "HorizontalPodAutoscaler" {
switch object.GetAPIVersion() {
case "autoscaling/v2beta2":
case "autoscaling/v2":
var d hpav2beta2.HorizontalPodAutoscaler
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to continue to use the hpav2beta2 here instead of moving to hpav2?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! I think we can do hpav2beta2 for autoscaling/v2beta2 and then hpav2 for autoscaling/v2. It would mean duplicating the code in the two switch cases. wdyt?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds OK to me.

@hiddeco hiddeco added area/server-side-apply SSA related issues and pull requests bug Something isn't working labels Jan 11, 2023
Makefile Outdated
@@ -17,7 +17,7 @@ GO_TEST_ARGS ?= -race
ENVTEST_ARCH ?= amd64

# Kubernetes versions to use envtest with
ENVTEST_KUBERNETES_VERSION?=1.25
ENVTEST_KUBERNETES_VERSION?=latest
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may break our attempt to have hermetic builds. fluxcd/flux2#3323 has previous discussion around it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will pin it to 1.26.0

ssa/utils.go Outdated
@@ -21,6 +21,7 @@ import (
"encoding/json"
"fmt"
"io"
hpav2 "k8s.io/api/autoscaling/v2"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be moved below with the other third party dependencies.

@@ -427,6 +428,17 @@ func SetNativeKindsDefaults(objects []*unstructured.Unstructured) error {
return fmt.Errorf("%s validation error: %w", FmtUnstructured(u), err)
}
u.Object = out
case "autoscaling/v2":
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Third case with almost the same code. Maybe it's time to write a common function for it, unless it has any known performance impact?
Something like

func setHPADefaults(u *unstructured.Unstructured, obj interface{}) error

and pass the typed variable d as an empty interface.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The loop uses a specific type

var metrics []hpav2.MetricSpec # we use hpav2beta2 for autoscaling/v2beta2
```.
Should we default all to []hpav2.MetricSpec?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's for fixHorizontalPodAutoscaler(). The above suggestion is for SetNativeKindsDefaults().
I think something similar can be done for fixHorizontalPodAutoscaler() as well, but may not be pretty 😄 Maybe give it a try, multiple common functions that return empty interface or some generic type. Already sounds a bit complicated. But there may be some simple way to do it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, sorry I missed that.
I attempted for fixHorizontalPodAutoscaler. It wasn't pretty but I will give it another shot.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although I am not sure what default is being set for HPA in that function. The HPA part seems to just convert to and from the typed struct while the rest set a missing default. The tests pass when I take out the switch case for the HPA in SetNativeKindsDefaults

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the tests continue to pass, maybe we are missing some test case for it?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what should be tested, since the function isn't setting/unsetting any values for HPA kinds. @/stefanprodan is probably the best person to say

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous workaround for HPA bug in #207 isn't related to SetNativeKindsDefaults(). And the first time these defaults were added for HPA was in #160 without any comment or test for HPA itself. An update to it was in #165 which was due to some incompatibility in the HPA API, no tests or any explicitly documented reason around it.

If the v2beta2 API and v2 API are compatible, I think adding a new case for v2 HPA in SetNativeKindsDefaults() may not be needed. The actual fix for the issue that this PR is addressing may be enough with the change in fixHorizontalPodAutoscaler().

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The missing int to string conversion is not specific to PodSpec, that's why I used the conversion for HPA too.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can figure out if this is still the case for HPA or it was a Kubernetes 1.19 bug only, @somtochiama has created an issue here #454 so I'll merge this as it is and we can remove code in another PR.

Copy link
Member

@stefanprodan stefanprodan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Thanks @somtochiama 🎖️

Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/server-side-apply SSA related issues and pull requests bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ssa: HPA drift detection is broken on Kubernetes 1.26 Constant Drift in HPA Resource
4 participants