Skip to content

Commit

Permalink
file: allow to create CephFS data pools with predefined names
Browse files Browse the repository at this point in the history 
Add an ability to create data pools for CephFS with predefined
names.

Related-Issue: #9295
Signed-off-by: Denis Egorenko <degorenko@mirantis.com>
  • Loading branch information
@degorenko
degorenko committed Dec 7, 2021
1 parent 5f44b1c commit 16c2741
Show file tree
Hide file tree
Showing 13 changed files with 583 additions and 89 deletions.
37 changes: 23 additions & 14 deletions Documentation/ceph-filesystem-crd.md
View file
Expand Up @@ -30,10 +30,12 @@ spec:
failureDomain: host
replicated:
size: 3
dataPools:
- failureDomain: host
replicated:
size: 3
namedDataPools:
- name: replicated-pool
poolSpec:
failureDomain: host
replicated:
size: 3
preserveFilesystemOnDelete: true
metadataServer:
activeCount: 1
Expand Down Expand Up @@ -69,7 +71,7 @@ spec:

### Erasure Coded

Erasure coded pools require the OSDs to use `bluestore` for the configured [`storeType`](ceph-cluster-crd.md#osd-configuration-settings). Additionally, erasure coded pools can only be used with `dataPools`. The `metadataPool` must use a replicated pool.
Erasure coded pools require the OSDs to use `bluestore` for the configured [`storeType`](ceph-cluster-crd.md#osd-configuration-settings). Additionally, erasure coded pools can only be used with `namedDataPools`. The `metadataPool` must use a replicated pool.

> **NOTE**: This sample requires *at least 3 bluestore OSDs*, with each OSD located on a *different node*.
Expand All @@ -85,10 +87,12 @@ spec:
metadataPool:
replicated:
size: 3
dataPools:
- erasureCoded:
dataChunks: 2
codingChunks: 1
namedDataPools:
- name: ec-pool
poolSpec:
erasureCoded:
dataChunks: 2
codingChunks: 1
metadataServer:
activeCount: 1
activeStandby: true
Expand Down Expand Up @@ -117,10 +121,12 @@ spec:
failureDomain: host
replicated:
size: 3
dataPools:
- failureDomain: host
replicated:
size: 3
namedDataPools:
- name: replicated-pool
poolSpec:
failureDomain: host
replicated:
size: 3
preserveFilesystemOnDelete: true
metadataServer:
activeCount: 1
Expand Down Expand Up @@ -183,7 +189,10 @@ See the official cephfs mirror documentation on [how to add a bootstrap peer](ht
The pools allow all of the settings defined in the Pool CRD spec. For more details, see the [Pool CRD](ceph-pool-crd.md) settings. In the example above, there must be at least three hosts (size 3) and at least eight devices (6 data + 2 coding chunks) in the cluster.

* `metadataPool`: The settings used to create the filesystem metadata pool. Must use replication.
* `dataPools`: The settings to create the filesystem data pools. If multiple pools are specified, Rook will add the pools to the filesystem. Assigning users or files to a pool is left as an exercise for the reader with the [CephFS documentation](http://docs.ceph.com/docs/master/cephfs/file-layouts/). The data pools can use replication or erasure coding. If erasure coding pools are specified, the cluster must be running with bluestore enabled on the OSDs.
* `namedDataPools`: The settings to create the filesystem data pools with predefined name. If multiple pools are specified, Rook will add the pools to the filesystem. Assigning users or files to a pool is left as an exercise for the reader with the [CephFS documentation](http://docs.ceph.com/docs/master/cephfs/file-layouts/). The data pools can use replication or erasure coding. If erasure coding pools are specified, the cluster must be running with bluestore enabled on the OSDs. Each element consist of:
* `name`: Name for pool
* `poolSpec`: Pool spec (ceph-pool-crd.md#spec)
* `dataPools`: (deprecated in favor of `namedDataPools`) The settings to create the filesystem data pools. If specified will be transitioned in runtime to `namedDataPools` with default name and index, thus pool name will be saved for any already present there pools.
* `preserveFilesystemOnDelete`: If it is set to 'true' the filesystem will remain when the
CephFilesystem resource is deleted. This is a security measure to avoid loss of data if the
CephFilesystem resource is deleted accidentally. The default value is 'false'. This option
Expand Down
10 changes: 6 additions & 4 deletions Documentation/ceph-filesystem.md
View file
Expand Up @@ -35,9 +35,11 @@ spec:
metadataPool:
replicated:
size: 3
dataPools:
- replicated:
size: 3
namedDataPools:
- name: replicated-pool
poolSpec:
replicated:
size: 3
preserveFilesystemOnDelete: true
metadataServer:
activeCount: 1
Expand Down Expand Up @@ -98,7 +100,7 @@ parameters:

# Ceph pool into which the volume shall be created
# Required for provisionVolume: "true"
pool: myfs-data0
pool: replicated-pool

# The secrets contain Ceph admin credentials. These are generated automatically by the operator
# in the same namespace as the cluster.
Expand Down
174 changes: 171 additions & 3 deletions deploy/charts/rook-ceph/templates/resources.yaml
View file
Expand Up @@ -4808,7 +4808,7 @@ spec:
description: FilesystemSpec represents the spec of a file system
properties:
dataPools:
description: The data pool settings
description: (Deprecated in favor of namedDataPools) The data pool settings.
items:
description: PoolSpec represents the spec of ceph pool
properties:
Expand Down Expand Up @@ -4971,7 +4971,6 @@ spec:
type: object
x-kubernetes-preserve-unknown-fields: true
type: object
nullable: true
type: array
metadataPool:
description: The metadata pool settings
Expand Down Expand Up @@ -5776,6 +5775,176 @@ spec:
type: object
type: array
type: object
namedDataPools:
description: The data pool settings with explicit data pool name. Exactly one of dataPools or namedDataPools (preferred) must be specified.
items:
description: NamedPoolSpec represents the named ceph pool spec
properties:
compressionMode:
description: 'DEPRECATED: use Parameters instead, e.g., Parameters["compression_mode"] = "force" The inline compression mode in Bluestore OSD to set to (options are: none, passive, aggressive, force) Do NOT set a default value for kubebuilder as this will override the Parameters'
enum:
- none
- passive
- aggressive
- force
- ""
nullable: true
type: string
crushRoot:
description: The root of the crush hierarchy utilized by the pool
nullable: true
type: string
deviceClass:
description: The device class the OSD should set to for use in the pool
nullable: true
type: string
enableRBDStats:
description: EnableRBDStats is used to enable gathering of statistics for all RBD images in the pool
type: boolean
erasureCoded:
description: The erasure code settings
properties:
algorithm:
description: The algorithm for erasure coding
type: string
codingChunks:
description: Number of coding chunks per object in an erasure coded storage pool (required for erasure-coded pool type). This is the number of OSDs that can be lost simultaneously before data cannot be recovered.
minimum: 0
type: integer
dataChunks:
description: Number of data chunks per object in an erasure coded storage pool (required for erasure-coded pool type). The number of chunks required to recover an object when any single OSD is lost is the same as dataChunks so be aware that the larger the number of data chunks, the higher the cost of recovery.
minimum: 0
type: integer
required:
- codingChunks
- dataChunks
type: object
failureDomain:
description: 'The failure domain: osd/host/(region or zone if available) - technically also any type in the crush map'
type: string
mirroring:
description: The mirroring settings
properties:
enabled:
description: Enabled whether this pool is mirrored or not
type: boolean
mode:
description: 'Mode is the mirroring mode: either pool or image'
type: string
peers:
description: Peers represents the peers spec
nullable: true
properties:
secretNames:
description: SecretNames represents the Kubernetes Secret names to add rbd-mirror or cephfs-mirror peers
items:
type: string
type: array
type: object
snapshotSchedules:
description: SnapshotSchedules is the scheduling of snapshot for mirrored images/pools
items:
description: SnapshotScheduleSpec represents the snapshot scheduling settings of a mirrored pool
properties:
interval:
description: Interval represent the periodicity of the snapshot.
type: string
path:
description: Path is the path to snapshot, only valid for CephFS
type: string
startTime:
description: StartTime indicates when to start the snapshot
type: string
type: object
type: array
type: object
name:
description: Name of pool
type: string
parameters:
additionalProperties:
type: string
description: Parameters is a list of properties to enable on a given pool
nullable: true
type: object
x-kubernetes-preserve-unknown-fields: true
quotas:
description: The quota settings
nullable: true
properties:
maxBytes:
description: MaxBytes represents the quota in bytes Deprecated in favor of MaxSize
format: int64
type: integer
maxObjects:
description: MaxObjects represents the quota in objects
format: int64
type: integer
maxSize:
description: MaxSize represents the quota in bytes as a string
pattern: ^[0-9]+[\.]?[0-9]*([KMGTPE]i|[kMGTPE])?$
type: string
type: object
replicated:
description: The replication settings
properties:
hybridStorage:
description: HybridStorage represents hybrid storage tier settings
nullable: true
properties:
primaryDeviceClass:
description: PrimaryDeviceClass represents high performance tier (for example SSD or NVME) for Primary OSD
minLength: 1
type: string
secondaryDeviceClass:
description: SecondaryDeviceClass represents low performance tier (for example HDDs) for remaining OSDs
minLength: 1
type: string
required:
- primaryDeviceClass
- secondaryDeviceClass
type: object
replicasPerFailureDomain:
description: ReplicasPerFailureDomain the number of replica in the specified failure domain
minimum: 1
type: integer
requireSafeReplicaSize:
description: RequireSafeReplicaSize if false allows you to set replica 1
type: boolean
size:
description: Size - Number of copies per object in a replicated storage pool, including the object itself (required for replicated pool type)
minimum: 0
type: integer
subFailureDomain:
description: SubFailureDomain the name of the sub-failure domain
type: string
targetSizeRatio:
description: TargetSizeRatio gives a hint (%) to Ceph in terms of expected consumption of the total cluster capacity
type: number
required:
- size
type: object
statusCheck:
description: The mirroring statusCheck
properties:
mirror:
description: HealthCheckSpec represents the health check of an object store bucket
nullable: true
properties:
disabled:
type: boolean
interval:
description: Interval is the internal in second or minute for the health check to run like 60s for 60 seconds
type: string
timeout:
type: string
type: object
type: object
x-kubernetes-preserve-unknown-fields: true
required:
- name
type: object
type: array
preserveFilesystemOnDelete:
description: Preserve the fs in the cluster on CephFilesystem CR deletion. Setting this to true automatically implies PreservePoolsOnDelete is true.
type: boolean
Expand All @@ -5800,7 +5969,6 @@ spec:
type: object
x-kubernetes-preserve-unknown-fields: true
required:
- dataPools
- metadataPool
- metadataServer
type: object
Expand Down

0 comments on commit 16c2741

Please sign in to comment.