osd: cleanup cluster using the root UID

We need to use the root UID to run the cleanup job since ceph-volume
does not support running as non-root in all the versions. We have this
ceph tracker to allow this https://tracker.ceph.com/issues/53511 but
it's not in all ceph's versions. Implementing a version check is tricky
since at this stage the cluster is being deleted and it's hard to catch
it.

Closes: #9375
Signed-off-by: Sébastien Han <seb@redhat.com>

pkg/operator/ceph/cluster/cleanup.go

@@ -123,10 +123,18 @@ func (c *ClusterController) cleanUpJobContainer(cluster *cephv1.CephCluster, mon
 		}...)
 	}
 
+	// Run a UID 0 since ceph-volume does not support running non-root
+	// See https://tracker.ceph.com/issues/53511
+	// Also, it's hard to catch the ceph version since the cluster is being deleted so not
+	// implementing a version check and simply always run this as root
+	rootUserID := int64(0)
+	securityContext := osd.PrivilegedContext()
+	securityContext.RunAsUser = &rootUserID
+
 	return v1.Container{
 		Name:            "host-cleanup",
 		Image:           c.rookImage,
-		SecurityContext: osd.PrivilegedContext(),
+		SecurityContext: securityContext,
 		VolumeMounts:    volumeMounts,
 		Env:             envVars,
 		Args:            []string{"ceph", "clean"},