error provisioning bucket: Provision: can't create ceph user: no user name provided

[krbcmt]

Is this a bug report or feature request?

• Bug Report

Deviation from expected behavior:
After upgrading Rook from 1.5.6 to 1.6.8, #7924 made it so if cephObjectStore.gateway.securePort is set then the Operator communicates with the RGW admin API over TLS. It appears that the hostname that's used for the RGW is derived from {cephObjectStore.name}.{cephObjectStore.namespace}.svc. In my circumstance I'm not able to create cluster-wide, verifiable, valid certificates for a domain like that, only for hostnames ending with the kubernetes clusterDomain defined in the kubelet config, typically it's cluster.local.

Expected behavior:
The admin API should have the ability to function with any certificate configured for the RGW.

How to reproduce it (minimal and precise):

1. Create cephObjectStore.gateway.sslCertificateRef with SAN for:
   {cephObjectStore.name}.{cephObjectStore.namespace}.svc.cluster.local
   e.g. some-object-store.rook-ceph.svc.cluster.local
2. Create an ObjectBucketClaim
3. See error in the operator logs saying:

2021-08-25 20:27:35.072315 I | op-bucket-prov: Provision: creating bucket "some-bucket" for OBC "some-obc"
E0825 20:27:35.076805 6 controller.go:199] error syncing 'default/some-obc': error provisioning bucket: Provision: can't create ceph user: no user name provided and unable to generate a unique name: failed to get ceph user "ceph-user-PktI0RS1": Get "https://some-object-store.rook-ceph.svc:443/admin/user?format=json&uid=ceph-user-PktI0RS1": x509: certificate is valid for some-object-store.rook-ceph.svc.cluster.local, not some-object-store.rook-ceph.svc, requeuing

[thotz]

IMO only way to do add some-object-store.rook-ceph.svc to DNS of ur cert. Please for internal testing in CI we use https://github.com/rook/rook/blob/master/tests/scripts/generate-tls-config.sh to generate certs. In the script we have defined for possible DNS for rgw. Because all of the valid endpoint access RGW service in k8s environments.

cat <<EOF >"${DIR}"/csr.conf
[req]
req_extensions = v3_req
distinguished_name = req_distinguished_name
[req_distinguished_name]
[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
[alt_names]
DNS.1 = ${SERVICE}
DNS.2 = ${SERVICE}.${NAMESPACE}
DNS.3 = ${SERVICE}.${NAMESPACE}.svc
DNS.4 = ${SERVICE}.${NAMESPACE}.svc.cluster.local
IP.1  = ${IP}
EOF

[leseb]

Will be fixed in #9020