vai: log errors on add

[moio]

client-go unfortunately "swallows" them

Partly addresses rancher/rancher#48384

[joshmeranda]

Should get a review from people with more Vai knowledge (@ericpromislow?)

[tomleb]

Does client-go "swallow" them because we don't set a klog log level high enough? Or are they really ignored?

In any case, I feel like we should log for all cases anyway (add, update and delete)?

[moio]

Does client-go "swallow" them because we don't set a klog log level high enough? Or are they really ignored?

Really ignored, unfortunately:

https://github.com/kubernetes/client-go/blob/ab443a50c6203a0d90ec3d369640b8a2b2981312/tools/cache/controller.go#L196-L203

(error is either ignored or pop is retried, but in no case logged)

In any case, I feel like we should log for all cases anyway (add, update and delete)?

Update is transitively covered by Add, Delete I have just added.

Thanks

[richard-cox]

Eagerly looking forward to this to help debug an issue indexing storage.k8s.io.storageclass metadata.annotations."storageclass.kubernetes.io/is-default-class"

[tomleb]

That will definitely be useful, surprised those errors are ignored like that.

[ericpromislow]

There might be some double-logging on these errors -- I count five calls from steve to ….[Ss]tore.Delete() at there might be others, where the method's receiver variable isn't called "store". All the calls I did fine do no logging, so better to put up with some double-logging and then unlogged errors.