✨ Add rest client request duration and size metrics

[sbueringer]

Signed-off-by: Stefan Büringer buringerst@vmware.com

This PR adds the following metrics:

• rest_client_request_duration_seconds
• rest_client_request_size_bytes
• rest_client_response_size_bytes

As they are defined in component-base upstream: https://github.com/kubernetes/component-base/blob/master/metrics/prometheus/restclient/metrics.go

I think the cardinality should be fine as the host label intentionally only contains the "template url".

Example:

rest_client_request_duration_seconds_sum{host="https://172.18.0.3:6443/api/v1/namespaces/%7Bnamespace%7D/pods/%7Bname%7D?timeout=10s",verb="GET"} 0.002900856
rest_client_request_duration_seconds_count{host="https://172.18.0.3:6443/api/v1/namespaces/%7Bnamespace%7D/pods/%7Bname%7D?timeout=10s",verb="GET"} 1

More examples can be found under: https://storage.googleapis.com/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cluster-api/8207/pull-cluster-api-e2e-main/1630798093588893696/artifacts/clusters/bootstrap/metrics/capi-system/capi-controller-manager/capi-controller-manager-9db79449b-vnfh8/metrics.txt
(please note that this file contains more rest_client metrics than I added in this PR)

[sbueringer]

Just aligned this to the way the metrics are defined just with name instead of subsystem + name here: https://github.com/kubernetes/component-base/blob/master/metrics/prometheus/restclient/metrics.go#L78

[sbueringer]

I wonder if we should drop RequestLatency in l.41-62 (in a separate PR) given:

	// Deprecated: This metric is deprecated for removal in a future release: using the URL as a
	// dimension results in cardinality explosion for some consumers. It was deprecated upstream
	// in k8s v1.14 and hidden in v1.17 via https://github.com/kubernetes/kubernetes/pull/83836.

[sbueringer]

/assign @vincepri @alvaroaleman

[vincepri]

/approve
/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sbueringer, vincepri

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [sbueringer,vincepri]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[sbueringer]

@vincepri Would it be possible to cherry-pick this PR into release-0.14?

No problem if we don't want to do it, would be just nice to be able to pick it up earlier

[vincepri]

/cherry-pick release-0.14

[k8s-infra-cherrypick-robot]

@vincepri: new pull request created: #2218

In response to this:

/cherry-pick release-0.14

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.