Creating a bucket with RGW multisite on a rook cluster having a non-master zone makes the credentials in the generated secret wrong

[lgyurci]

Is this a bug report or feature request?

• Bug Report

Deviation from expected behavior:
The credentials in the generated secret (which has the same name as the ObjectBucketClaim) are wrong
Expected behavior:
The credentials in the generated secret (which has the same name as the ObjectBucketClaim) are not wrong
How to reproduce it (minimal and precise):

1. Create an RGW multisite setup
2. Create a bucket on the secondary cluster
3. Etiher try to use the credentials in the generated secret, or compare them with the ACCESS_KEY/SECRET_KEY stored in the RGWs brain (adosgw-admin user info --rgw-realm=$REALM --rgw-zonegroup=$ZONEGROUP --rgw-zone=$SECONDARY-ZONE --uid=$UID)

Environment:

• Rook version (use rook version inside of a Rook Pod): v1.12.0-alpha.0.474.g5230fcd60
• Storage backend version (e.g. for ceph do ceph -v): ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)
• Kubernetes version (use kubectl version): v1.26.11+rke2r1
• Kubernetes cluster type (e.g. Tectonic, GKE, OpenShift): RKE2

[lgyurci]

I believe the operation is sent to the primary RGW for processing (according to ceph docs, bucket operations are forwarded to the master zone), and this somehow disturbs rook

[thotz]

@alimaredia worked to fix the keys for users created in the secondary zone. I thought that change was merged

[lgyurci]

So... Was it merged?

[thotz]

This was the PR which should fix the issue ceph/ceph#48731 got backported till pacific as well