log error about cert expired - does not show important info

[KlavsKlavsen]

our kube-apiserver just started failing to start.. The log from containerd only says:
2023-12-16T07:35:01.426019865+01:00 stderr F }. Err: connection error: desc = "transport: authentication handshake failed: x509: certificate has expired or is not yet valid: current time 2023-12-16T06:35:01Z is after 2023-12-03T03:37:19Z"

so the cert appearently should have expired on the 3rd of december (and yet it only fails now) - and also - if I check the cert its using :

# openssl x509 -in /etc/kubernetes/pki/apiserver.crt -noout -text | grep fter
            Not After : Apr 13 14:08:43 2024 GMT
07:34:04 root@htzhel1-ax41na:~

its not actually run out..

The errorlog could be much more helpful -if it shared exactly which certificate its checking.. (file path - or whatever its got open) - AND Serial number from cert to uniquely identify the one its looking at.. :(

its k8s 1.26.4

[KlavsKlavsen]

In our case - it turned out to be the etcd server it was trying to connect to - that had the bad cert.. REALLY would have been helpful if it had shared that info

[KlavsKlavsen]

kubeadm certs check-expiration has the info - and shows this etcd cert being expired.. odd thing is normal k8s upgrades didn't re-issue these - but did re-issue the rest (which last until apr 2024).. seems a bug as well

[neolit123]

i don't know if any of the apisever maintainers watch this repository. better close this ticket and open a new one in kubernetes/kubernetes.

[neolit123]

kubeadm certs check-expiration has the info - and shows this etcd cert being expired.. odd thing is normal k8s upgrades didn't re-issue these - but did re-issue the rest (which last until apr 2024).. seems a bug as well

if upgrade was done by kubeadm this can happen if the etcd version did not change between minor k8s versions. it is by design because etcd upgrade is skipped and with it certs renewal of etcd. you can open a ticket in kubernetes/kubeadm for discussion if you want. i.e. maybe the maintainers team will agree to change it.

the etcd version does change between patch and minor versions of k8s. so as long as you update more often the etcd certs will renew.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.