New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PersistentVolumeLabel admission plugin provides wrong Azure region #124525
Comments
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the The Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
/sig storage |
/sig storage |
@andyzhangx it would be great if Azure cloud provider did not provide any labels when it's actually not configured (i.e. the external one is used). Is there an easy way to detect that in the admission plugin in the API server?
Or
Note that it affects 1.29 and older that still have the cloud provider. I am fixing 1.30 differently in #124505 |
/priority important-soon The PersistentVolumeLabel admission plugin + CSI migration on azure are basically unusable when used together. I guess it affects a small nr. of clusters, but there is no workaround. |
@jsafrane the pv lable issue only exists on in-tree disk pv, right? since in-tree disk volume is already deprecated long time ago, I think we could live with that issue. |
In-tree PVs are deprecated, but still supported. If someone upgrades a fully working cluster to a version that has the external cloud provider instead of the in-tree one, they cannot use their StorageClass that was working for many releases. |
BTW, I would prefer removing the whole PersistentVolumeLabel admission plugin instead, it's not very useful, but that's for v1.31. See #124504. I need 1.29 and older working somehow even with the admission enabled. |
I filed #124528 to fix it as simply as possible, PTAL. Can there be a real Azure VM with |
this empty value won't make sense, it should always be |
What happened?
On Kubernetes 1.29 with PersistentVolumeLabel admission plugin enabled and no in-tree cloud provider configured in the kube-apiserver, the admission plugins adds label
topology.kubernetes.io/region: ""
to in-tree Azure Disk PVs.The label is completely wrong - the Azure cloud provider is not initialized and this just provides empty string. This makes any pod that uses these PVs unschedulable.
As result, all in-tree PVs dynamically provisioned using CSI migration are not usable, because they have a wrong region.
What did you expect to happen?
When there is no cloud config provided, PersistentVolumeLabel does not add any labels to PVs. PVs created by the CSI driver + CSI migration already have correct
nodeAffinity
, the labels are useless.How can we reproduce it (as minimally and precisely as possible)?
Enable PersistentVolumeLabel admission in your cluster and disable any cloud providers. It does not need access to Azure, you can run it anywhere.
Example for
local-up-cluster.sh
:Create in-tree Azure Disk PV. Again, we're testing just the admission, we're not going to use the PV in a pod, so no access to Azure is needed. UUIDs were sanitized ;-)
Actual result: wrong region label on the PV:
Expected result: no labels on the PV.
Anything else we need to know?
No response
Kubernetes version
Cloud provider
OS version
No response
Install tools
Container runtime (CRI) and version (if applicable)
Related plugins (CNI, CSI, ...) and versions (if applicable)
The text was updated successfully, but these errors were encountered: