[kube-prometheus-stack] Allow to not skip the TLS verification in kubelet servicemonitor

[matofeder]

Is your feature request related to a problem ?

By default the kubelet serving certificate deployed by kubeadm is self-signed. This means a connection from external services like the metrics-server/kube-api or Prometheus to a kubelet cannot be secured with TLS, see the docs

Probably because of the above reason the kube-promtheus-stack skips the TLS verification in kubelet service monitor and this setting is hardcoded.

Describe the solution you'd like.

kubelet could obtain properly signed serving certificates (signed by the Kubernetes CA) via the serverTLSBootstrap: true field. This enables the bootstrap of kubelet server certificates by requesting them from the certificates.k8s.io API. Read the related docs

In the above case, the Kubelet metrics endpoints are properly secured by the server certificate issued by Kubernetes CA.

Then the kubelet servicemonitor should be configured to not skip the TLS verification.

Describe alternatives you've considered.

NONE

Additional context.

Related notes can be found here: SovereignCloudStack/issues#495 (comment)

[jkroepke]

Do you want to create a PR for this?

[matofeder]

Do you want to create a PR for this?

yes, if you/community agree that it is a good idea to do so