mds: create EC pool as secondary pool

When creating EC fs, create replicated pool as primary
pool and ec pool as secondary pool, creating ec pool
as primary is not encouraged and it will lead to failure.

Also, changing the pool name in storageclass-ec file.

Closes: #8210
Signed-off-by: subhamkrai <srai@redhat.com>

Documentation/ceph-filesystem-crd.md

@@ -86,6 +86,8 @@ spec:
     replicated:
       size: 3
   dataPools:
+    - replicated:
+        size: 3
     - erasureCoded:
         dataChunks: 2
         codingChunks: 1
@@ -94,8 +96,7 @@ spec:
     activeStandby: true
 ```
 
-(These definitions can also be found in the [`filesystem-ec.yaml`](https://github.com/rook/rook/blob/{{ branchName }}/cluster/examples/kubernetes/ceph/filesystem-ec.yaml) file.
-Also see an example in the [`storageclass-ec.yaml`](https://github.com/rook/rook/blob/{{ branchName }}/cluster/examples/kubernetes/ceph/csi/cephfs/storageclass-ec.yaml) for how to configure the volume.)
+**IMPORTANT**: For erasure coded pools, we have to create a replicated pool as the default data pool and an erasure-coded pool as a secondary pool.
 
 ### Mirroring
 

cluster/examples/kubernetes/ceph/csi/cephfs/storageclass-ec.yaml

@@ -14,7 +14,7 @@ parameters:
 
   # Ceph pool into which the volume shall be created
   # Required for provisionVolume: "true"
-  pool: myfs-ec-data0
+  pool: myfs-ec-data1
 
   # The secrets contain Ceph admin credentials. These are generated automatically by the operator
   # in the same namespace as the cluster.

cluster/examples/kubernetes/ceph/filesystem-ec.yaml

@@ -17,6 +17,8 @@ spec:
       size: 3
   # The list of data pool specs
   dataPools:
+    - replicated:
+        size: 3
     # You need at least three `bluestore` OSDs on different nodes for this config to work
     - erasureCoded:
         dataChunks: 2