Add cpu target percentile parameter to VPA options in Shoot spec

[voelzmo]

How to categorize this PR?

/area auto-scaling
/kind enhancement

What this PR does / why we need it:
This PR exposes the targetCPUPercentile parameter of the VPA recommender in the Shoot spec. With this parameter, Shoot owners or gardener operators can fine-tune the recommender behavior regarding recommendations for CPU.

The default is 0.9, which means the recommender will issue a recommendation that is fulfilling 90% of the observed CPU measurements. As CPU is a compressible resource, Shoot owner may want to set this to a smaller value, depending on the headroom on their nodes and the requirements for their workloads.

Release note:

In order to fine-tune VPA CPU recommendations, the CPU target percentile parameter for `vpa-recommender` can now be configured in the `Shoot` specification via the `.spec.kubernetes.verticalPodAutoscaler.targetCPUPercentile` field.

[voelzmo]

/cc @vlerenc @andrerun

[voelzmo]

/hold running some experiments to verify this has the desired effect

[voelzmo]

/retest
/hold cancel
Tests have been successful: when modifying the parameter, we see that his has a direct impact on the cpu recommendations. Lower values lead to lower recommendations overall.

[rfranzke]

PR looks good, but adaptation in https://github.com/gardener/gardener/blob/master/docs/usage/shoot_autoscaling.md#vertical-pod-auto-scaling is still missing.

[voelzmo]

PR looks good, but adaptation in https://github.com/gardener/gardener/blob/master/docs/usage/shoot_autoscaling.md#vertical-pod-auto-scaling is still missing.

Ah, sorry, thanks for catching that I missed that.
On a related note: I just find myself copy-pasting the same things that I already wrote in the API docs for those properties. Should we maybe just link to the API docs instead of replicating that information here?

[rfranzke]

Should we maybe just link to the API docs instead of replicating that information here?

We could. OTOH, the API docs typically do not contain too much information. Maybe this works in case of the VPA fields - if so, feel free to just link the generated docs (maybe in a separate PR).

/lgtm
/approve

[gardener-prow]

LGTM label has been added.

Git tree hash: 8e12fdab4f4492df3f9ce1935384df1fe9774ac6

[gardener-prow]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: rfranzke

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [rfranzke]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[voelzmo]

/retest