csi: no longer install the volumereplication crds from rook

The volume replication CRDs are an external component, not owned by Rook.
Therefore, they should be installed as any other independent component
in case the admin will install other consumers of the volumereplication CRDs
in the future in addition to Rook and the CSI driver.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit c420f23)

# Conflicts:
#	build/crds/build-crds.sh
#	build/crds/crds.go
#	cluster/charts/rook-ceph/templates/resources.yaml
#	cluster/examples/kubernetes/ceph/crds.yaml
#	go.mod
#	go.sum
#	tests/framework/installer/ceph_installer.go
#	tests/integration/ceph_object_test.go
#	tests/integration/ceph_smoke_test.go

Documentation/ceph-csi-drivers.md

@@ -77,8 +77,18 @@ PVC will be updated to new size.
 ## RBD Mirroring
 
 To support RBD Mirroring, the [Volume Replication Operator](https://github.com/csi-addons/volume-replication-operator/blob/main/README.md) will be started in the RBD provisioner pod.
-Volume Replication Operator is a kubernetes operator that provides common and reusable APIs for storage disaster recovery. It is based on [csi-addons/spec](https://github.com/csi-addons/spec) specification and can be used by any storage provider.
-It follows controller pattern and provides extended APIs for storage disaster recovery. The extended APIs are provided via Custom Resource Definition (CRD).
-To enable volume replication:
-- For Helm deployments see the [helm settings](helm-operator.md#configuration).
-- For non-Helm deployments set `CSI_ENABLE_VOLUME_REPLICATION: "true"` in the operator.yaml
+The Volume Replication Operator is a kubernetes operator that provides common and reusable APIs for storage disaster recovery. It is based on [csi-addons/spec](https://github.com/csi-addons/spec) specification and can be used by any storage provider.
+It follows the controller pattern and provides extended APIs for storage disaster recovery. The extended APIs are provided via Custom Resource Definitions (CRDs).
+
+### Enable volume replication
+
+1. Install the volume replication CRDs:
+
+```console
+kubectl create -f https://raw.githubusercontent.com/csi-addons/volume-replication-operator/v0.1.0/config/crd/bases/replication.storage.openshift.io_volumereplications.yaml
+kubectl create -f https://raw.githubusercontent.com/csi-addons/volume-replication-operator/v0.1.0/config/crd/bases/replication.storage.openshift.io_volumereplicationclasses.yaml
+```
+
+2. Enable the volume replication controller:
+   - For Helm deployments see the [csi.volumeReplication.enabled setting](helm-operator.md#configuration).
+   - For non-Helm deployments set `CSI_ENABLE_VOLUME_REPLICATION: "true"` in operator.yaml

build/crds/build-crds.sh

@@ -53,6 +53,7 @@ generating_crds_v1() {
   $YQ_BIN_PATH w -i "${OLM_CATALOG_DIR}"/ceph.rook.io_cephclusters.yaml spec.versions[0].schema.openAPIV3Schema.properties.spec.properties.storage.properties.storageClassDeviceSets.items.properties.volumeClaimTemplates.items.properties.metadata.properties.annotations.x-kubernetes-preserve-unknown-fields true
 }
 
+<<<<<<< HEAD
 generating_crds_v1alpha2() {
   "$CONTROLLER_GEN_BIN_PATH" "$CRD_OPTIONS" paths="./pkg/apis/rook.io/v1alpha2" output:crd:artifacts:config="$OLM_CATALOG_DIR"
   # TODO: revisit later
@@ -67,6 +68,8 @@ generate_vol_rep_crds() {
   "$CONTROLLER_GEN_BIN_PATH" "$CRD_OPTIONS" paths="github.com/csi-addons/volume-replication-operator/api/v1alpha1" output:crd:artifacts:config="$OLM_CATALOG_DIR"
 }
 
+=======
+>>>>>>> c420f2309 (csi: no longer install the volumereplication crds from rook)
 generating_main_crd() {
   true > "$CEPH_CRDS_FILE_PATH"
   true > "$CEPH_HELM_CRDS_FILE_PATH"
@@ -111,8 +114,6 @@ if [ -z "$NO_OB_OBC_VOL_GEN" ]; then
   generating_crds_v1alpha2
 fi
 
-generate_vol_rep_crds
-
 generating_main_crd
 
 for crd in "$OLM_CATALOG_DIR/"*.yaml; do

cluster/charts/rook-ceph/templates/resources.yaml

@@ -8945,6 +8945,7 @@ spec:
               x-kubernetes-preserve-unknown-fields: true
       subresources:
         status: {}
+<<<<<<< HEAD
 ---
 apiVersion: apiextensions.k8s.io/v1
 kind: CustomResourceDefinition
@@ -9784,6 +9785,8 @@ spec:
                       type: boolean
   subresources:
     status: {}
+=======
+>>>>>>> c420f2309 (csi: no longer install the volumereplication crds from rook)
 
 ---
 apiVersion: apiextensions.k8s.io/v1beta1

cluster/charts/rook-ceph/values.yaml

@@ -288,7 +288,9 @@ csi:
   #cephfsPodLabels: "key1=value1,key2=value2"
   # Labels to add to the CSI RBD Deployments and DaemonSets Pods.
   #rbdPodLabels: "key1=value1,key2=value2"
-  # Enable volume replication controller
+  # Enable the volume replication controller.
+  # Before enabling, ensure the Volume Replication CRDs are created.
+  # See https://rook.io/docs/rook/latest/ceph-csi-drivers.html#rbd-mirroring
   volumeReplication:
     enabled: false
     #image: "quay.io/csiaddons/volumereplication-operator:v0.1.0"

cluster/examples/kubernetes/ceph/crds.yaml

@@ -8932,6 +8932,7 @@ spec:
               x-kubernetes-preserve-unknown-fields: true
       subresources:
         status: {}
+<<<<<<< HEAD
 ---
 apiVersion: apiextensions.k8s.io/v1
 kind: CustomResourceDefinition
@@ -9219,3 +9220,5 @@ status:
     plural: ""
   conditions: []
   storedVersions: []
+=======
+>>>>>>> c420f2309 (csi: no longer install the volumereplication crds from rook)

cluster/examples/kubernetes/ceph/operator-openshift.yaml

@@ -403,7 +403,9 @@ data:
   # Whether to start the discovery daemon to watch for raw storage devices on nodes in the cluster.
   # This daemon does not need to run if you are only going to create your OSDs based on StorageClassDeviceSets with PVCs.
   ROOK_ENABLE_DISCOVERY_DAEMON: "false"
-  # Enable volume replication controller
+  # Enable the volume replication controller
+  # Before enabling, ensure the Volume Replication CRDs are created.
+  # See https://rook.io/docs/rook/latest/ceph-csi-drivers.html#rbd-mirroring
   CSI_ENABLE_VOLUME_REPLICATION: "false"
   # The timeout value (in seconds) of Ceph commands. It should be >= 1. If this variable is not set or is an invalid value, it's default to 15.
   ROOK_CEPH_COMMANDS_TIMEOUT_SECONDS: "15"

cluster/examples/kubernetes/ceph/operator.yaml

@@ -329,7 +329,9 @@ data:
   ROOK_ENABLE_DISCOVERY_DAEMON: "false"
   # The timeout value (in seconds) of Ceph commands. It should be >= 1. If this variable is not set or is an invalid value, it's default to 15.
   ROOK_CEPH_COMMANDS_TIMEOUT_SECONDS: "15"
-  # Enable volume replication controller
+  # Enable the volume replication controller.
+  # Before enabling, ensure the Volume Replication CRDs are created.
+  # See https://rook.io/docs/rook/latest/ceph-csi-drivers.html#rbd-mirroring
   CSI_ENABLE_VOLUME_REPLICATION: "false"
   # CSI_VOLUME_REPLICATION_IMAGE: "quay.io/csiaddons/volumereplication-operator:v0.1.0"
 

go.mod

@@ -7,8 +7,11 @@ require (
 	github.com/banzaicloud/k8s-objectmatcher v1.1.0
 	github.com/ceph/go-ceph v0.11.0
 	github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f
+<<<<<<< HEAD
 	github.com/csi-addons/volume-replication-operator v0.1.1-0.20210525040814-ab575a2879fb
 	github.com/ghodss/yaml v1.0.1-0.20190212211648-25d852aebe32
+=======
+>>>>>>> c420f2309 (csi: no longer install the volumereplication crds from rook)
 	github.com/go-ini/ini v1.51.1
 	github.com/google/go-cmp v0.5.5
 	github.com/google/uuid v1.1.2