Resizing does not work with NetApp Trident

[holtakj]

Describe the bug
Resizing the prometheus PV managed by NetApp trident does not work.

pvc-autoresizer is triggered, the requested size in the PVC is adapted but the NetApp volume is not resized.
Resizing manually by changing the request size in the PVC works fine, Trident resizes the volume on the SAN.
NetApp Trident supports resizing.
https://docs.netapp.com/us-en/trident/trident-use/vol-expansion.html

Events:
kube-monitoring 0s Normal Resized persistentvolumeclaim/prometheus-kube-prometheus-stack-helm-prometheus-db-prometheus-kube-prometheus-stack-helm-prometheus-1 PVC volume is resized to 2Gi
kube-monitoring 0s Normal Resizing persistentvolumeclaim/prometheus-kube-prometheus-stack-helm-prometheus-db-prometheus-kube-prometheus-stack-helm-prometheus-1 External resizer is resizing volume pvc-5d996e1a-935a-4c17-a9e6-b05b12bed55b
kube-monitoring 0s Warning ExternalExpanding persistentvolumeclaim/prometheus-kube-prometheus-stack-helm-prometheus-db-prometheus-kube-prometheus-stack-helm-prometheus-1 waiting for an external controller to expand this PVC
kube-monitoring 0s Warning VolumeResizeFailed persistentvolumeclaim/prometheus-kube-prometheus-stack-helm-prometheus-db-prometheus-kube-prometheus-stack-helm-prometheus-1 resize volume "pvc-5d996e1a-935a-4c17-a9e6-b05b12bed55b" by resizer "csi.trident.netapp.io" failed: rpc error: code = Unknown desc = unable to resize the volume: volume resize failed
kube-monitoring 0s Normal Resizing persistentvolumeclaim/prometheus-kube-prometheus-stack-helm-prometheus-db-prometheus-kube-prometheus-stack-helm-prometheus-1 External resizer is resizing volume pvc-5d996e1a-935a-4c17-a9e6-b05b12bed55b

Logs:
{"level":"info","ts":"2024-05-16T09:49:05Z","logger":"pvc-autoresizer.resize","msg":"waiting for resizing...","namespace":"kube-monitoring","name":"prometheus-kube-prometheus-stack-helm-prometheus-db-prometheus-kube-prometheus-stack-helm-prometheus-1","capacity":1020702720}
{"level":"info","ts":"2024-05-16T09:49:15Z","logger":"pvc-autoresizer.resize","msg":"waiting for resizing...","namespace":"kube-monitoring","name":"prometheus-kube-prometheus-stack-helm-prometheus-db-prometheus-kube-prometheus-stack-helm-prometheus-1","capacity":1020702720}
{"level":"info","ts":"2024-05-16T09:49:25Z","logger":"pvc-autoresizer.resize","msg":"waiting for resizing...","namespace":"kube-monitoring","name":"prometheus-kube-prometheus-stack-helm-prometheus-db-prometheus-kube-prometheus-stack-helm-prometheus-1","capacity":1020702720}

Environments

• Version:
  rke2 1.28.9
  NetApp Trident 24.02
• OS:
  Ubuntu 22.04

Expected behavior
PV is resized.

[cupnes]

@holtakj
There is a log ‘csi.trident.netapp.io’ failed: rpc error in Events:. Is there something wrong on the Trident side?