Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added autoscaler options to worker groups #9245

Merged
merged 5 commits into from
Feb 28, 2024

Conversation

aaronfern
Copy link
Contributor

@aaronfern aaronfern commented Feb 26, 2024

How to categorize this PR?

/area auto-scaling
/kind api-change

What this PR does / why we need it:
This PR adds some autoscaler options to workers.
With this change, users can configure certain worker specific autoscaler options instead of one single value for all worker groups

Which issue(s) this PR fixes:
Fixes partially gardener/autoscaler#240

Special notes for your reviewer:
Autoscaler will read these values from annotations in the machineDeployment (ref: gardener/autoscaler#257), so the values from spec.provider.worker.autoscaler are added as annotations to the machineDeployment

Release note:

It is now possibleto specify some `cluster-autoscaler` related options per worker pool `in Shoot`s via `spec.provider.workers[].clusterAutoscaler`. Read more about it [here](https://github.com/gardener/gardener/blob/master/docs/usage/shoot_autoscaling.md).

@gardener-prow gardener-prow bot added area/auto-scaling Auto-scaling (CA/HPA/VPA/HVPA, predominantly control plane, but also otherwise) related kind/api-change API change with impact on API users labels Feb 26, 2024
@gardener-prow gardener-prow bot added cla: yes Indicates the PR's author has signed the cla-assistant.io CLA. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Feb 26, 2024
@rfranzke
Copy link
Member

/assign

@aaronfern
Copy link
Contributor Author

/retest

example/90-shoot.yaml Outdated Show resolved Hide resolved
example/90-shoot.yaml Outdated Show resolved Hide resolved
pkg/apis/core/types_shoot.go Outdated Show resolved Hide resolved
pkg/apis/core/types_shoot.go Outdated Show resolved Hide resolved
pkg/apis/core/v1beta1/types_shoot.go Outdated Show resolved Hide resolved
pkg/component/extensions/worker/worker.go Outdated Show resolved Hide resolved
pkg/provider-local/controller/worker/machines.go Outdated Show resolved Hide resolved
extensions/pkg/controller/worker/machines.go Outdated Show resolved Hide resolved
pkg/provider-local/controller/worker/machines.go Outdated Show resolved Hide resolved
@aaronfern
Copy link
Contributor Author

/retest

docs/extensions/worker.md Outdated Show resolved Hide resolved
docs/usage/shoot_autoscaling.md Outdated Show resolved Hide resolved
docs/usage/shoot_autoscaling.md Outdated Show resolved Hide resolved
docs/usage/shoot_autoscaling.md Outdated Show resolved Hide resolved
docs/usage/shoot_autoscaling.md Outdated Show resolved Hide resolved
pkg/apis/extensions/v1alpha1/types_worker.go Outdated Show resolved Hide resolved
pkg/apis/extensions/v1alpha1/types_worker.go Outdated Show resolved Hide resolved
pkg/component/extensions/worker/worker.go Outdated Show resolved Hide resolved
pkg/component/extensions/worker/worker.go Outdated Show resolved Hide resolved
@aaronfern
Copy link
Contributor Author

/test pull-gardener-e2e-kind-ha-multi-zone-upgrade

Copy link
Member

@rfranzke rfranzke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Just a few more nits before we are ready :)

@@ -115,6 +121,8 @@ The `spec.pools[].nodeTemplate.capacity` field contains the resource information

The `spec.pools[].machineControllerManager` field allows to configure the settings for machine-controller-manager component. Providers must populate these settings on worker-pool to the related [fields](https://github.com/gardener/machine-controller-manager/blob/master/kubernetes/machine_objects/machine-deployment.yaml#L30-L34) in MachineDeployment.

The `spec.pools[].clusterAutoscaler` field contains `cluster-autoscaler` settings that are to be applied only to specific worker group. `Cluster-autoscaler` expects to find these settings as annotations on the `MachineDeployment`, and so providers must pass these values to the corresponding `MachineDeployment` via annotations. The keys for these annotations can be found [here](https://github.com/gardener/gardener/blob/master/pkg/apis/extensions/v1alpha1/types_worker.go) and the values for the corresponding annotations should be the same as what is passed into the field.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The `spec.pools[].clusterAutoscaler` field contains `cluster-autoscaler` settings that are to be applied only to specific worker group. `Cluster-autoscaler` expects to find these settings as annotations on the `MachineDeployment`, and so providers must pass these values to the corresponding `MachineDeployment` via annotations. The keys for these annotations can be found [here](https://github.com/gardener/gardener/blob/master/pkg/apis/extensions/v1alpha1/types_worker.go) and the values for the corresponding annotations should be the same as what is passed into the field.
The `spec.pools[].clusterAutoscaler` field contains `cluster-autoscaler` settings that are to be applied only to specific worker group. `cluster-autoscaler` expects to find these settings as annotations on the `MachineDeployment`, and so providers must pass these values to the corresponding `MachineDeployment` via annotations. The keys for these annotations can be found [here](https://github.com/gardener/gardener/blob/master/pkg/apis/extensions/v1alpha1/types_worker.go) and the values for the corresponding annotations should be the same as what is passed into the field.

Do you want to refer to your helper function (extensionsv1alpha1helper.GetMachineDeploymentClusterAutoscalerAnnotations)?

@@ -1388,6 +1388,64 @@ var _ = Describe("Shoot Validation Tests", func() {
Expect(errorList).To(BeEmpty())
})
})

Describe("clusterAutoscaler options validation", func() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Describe("clusterAutoscaler options validation", func() {
Describe("ClusterAutoscaler options validation", func() {

Comment on lines 74 to 75
annotations := map[string]string{}
if caOptions != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
annotations := map[string]string{}
if caOptions != nil {
var annotations map[string]string
if caOptions != nil {
annotations := map[string]string{}

@@ -68,3 +68,27 @@ func FilePathsFrom(files []extensionsv1alpha1.File) []string {

return out
}

// GetMachineDeploymentClusterAutoscalerAnnotations returns a map of annotations with values intended to be used as cluster autoscaler options for the worker group
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// GetMachineDeploymentClusterAutoscalerAnnotations returns a map of annotations with values intended to be used as cluster autoscaler options for the worker group
// GetMachineDeploymentClusterAutoscalerAnnotations returns a map of annotations with values intended to be used as cluster-autoscaler options for the worker group.

Copy link
Member

@rfranzke rfranzke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

@gardener-prow gardener-prow bot added the lgtm Indicates that a PR is ready to be merged. label Feb 28, 2024
Copy link
Contributor

gardener-prow bot commented Feb 28, 2024

LGTM label has been added.

Git tree hash: e768e1a87c8bb033a7a9e8eadaa37fafa3049e33

Copy link
Contributor

gardener-prow bot commented Feb 28, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: rfranzke

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/auto-scaling Auto-scaling (CA/HPA/VPA/HVPA, predominantly control plane, but also otherwise) related cla: yes Indicates the PR's author has signed the cla-assistant.io CLA. kind/api-change API change with impact on API users lgtm Indicates that a PR is ready to be merged. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants