Try to make import and export more eaiser

[parth-gr]

Is this a bug report or feature request?

• Feature Request

Currently user runs a Python script at the RHCS cluster to export the data and then manually copy-paste it from the RHCS terminal and then paste it to the Kubernetes terminal

What if it is possible to expose the data directly to a server endpoint and the k8s cluster can auto-fetch the details from there directly?

The possible design will be:

1. Run the Python script at the RHCS cluster
   1.1) Python script will store/output data
   1.2) The Python script should expose the data to the server endpoint

2. The K8s cluster will watch for the event if the server as an update on data and will reconcile the Ceph controller

But the important point to think about is, where the server will be running so both the RHCS and k8s cluster can access the endpoint, would be possible to get the data like any other ceph or radosgw-admin command?

ps: maybe we can just read the file with the JSON output from the ceph cluster node?

Environment:

[parth-gr]

Update, look for the monstore metadata values and update it using the python script,
But we need to figure out how to autorize specific to k8s cluster

[parth-gr]

So the new design will look like.

Installation:

1. Run the python script in the RHCS cluster as usual
   -> It will update the ceph config/mon store with the JSON data
   -> Secondly it will create the connection key which has the information about the ceph user and mon endpoints

2. User has to add the connection key to the cephcluster CR
   -> Once the key is added it will create the specific resources based on the values of the ceph mon store

3. Cluter will be connected

Updation:

1. The user just have to run the Python script in the RHCS cluster it will auto-update the mon store and the resources

[travisn]

So the new design will look like.

Installation:

1. Run the python script in the RHCS cluster as usual
   -> It will update the ceph config/mon store with the JSON data
   -> Secondly it will create the connection key which has the information about the ceph user and mon endpoints

I like this idea, it just requires a Ceph change to allow certain mon store data to be granted access for a specific keyring.

2. User has to add the connection key to the cephcluster CR
   -> Once the key is added it will create the specific resources based on the values of the ceph mon store

We don't want the key in the CR. Instead, the key should be directly stored in the rook-ceph-mon secret to the values ceph-secret and ceph-username.

3. Cluster will be connected

After Rook connects, it would need to load the JSON and apply it from Ceph and to the consumer cluster, right?

Updation:

1. The user just have to run the Python script in the RHCS cluster it will auto-update the mon store and the resources

So Rook would need to check for changes to the mon store setting and apply any updates when the JSON changes?

[parth-gr]

After Rook connects, it would need to load the JSON and apply it from Ceph and to the consumer cluster, right?

Yes, the cluster will watch for mon-config updates and will create the resources, somewhat like #14076

So Rook would need to check for changes to the mon store setting and apply any updates when the JSON changes?

Exactly it would watch for the changes in it

[parth-gr]

I like this idea, it just requires a Ceph change to allow certain mon store data to be granted access for a specific keyring.

For ceph side changes thinking for a design,

The ceph config should have user authentication based on the user keys,
Currently, it looks like,

sh-4.4$ ceph config dump
WHO                    MASK  LEVEL     OPTION                                   VALUE                                     RO
global                       advanced  bdev_flock_retry                         20                                          
global                       advanced  bluefs_buffered_io                       false                                       
global                       basic     log_to_file                              false                                       
global                       advanced  mon_allow_pool_delete                    true                                        
global                       advanced  mon_allow_pool_size_one                  true                                        
global                       advanced  mon_cluster_log_file                                                                 
global                       advanced  mon_data_avail_warn                      10                                          
global                       advanced  mon_warn_on_pool_no_redundancy           false                                       
global                       advanced  osd_pool_default_size                    1                                           
mon                          advanced  auth_allow_insecure_global_id_reclaim    false                                       
mon                          advanced  rgw_zone                                 11                                        * 

I believe RO field can be used as permissions.

So,

1. The set command currently is

config set <who> <name> <value> [--force]                                            Set a configuration option for one or more entities

I should be updated like

config set <who> <whoKey> <name> <value> [--force]    

2. And secondly the get command looks like

config get <who> [<key>]                                                             Show configuration option(s) for an entity

I think the current key is optional which can be marked a non-optional field.

3. and ceph config dump should also updated to ceph osd dump key and it should expects a admin key for authentication.

[travisn]

Let's open a Ceph tracker with the feature request, and just summarize the requirement there. We won't gain much in trying to design it here, since the Ceph team would own the design. The feature request is just to have some mon store settings that are only available depending on the keyring.

[parth-gr]

Added a ceph tracker https://tracker.ceph.com/issues/65583

[parth-gr]

Offline discussion:

We had 2 new proposals for this feature,

1. Add the information of the JSON external data, on a radosnamesapce or a cephblockpool, and specific users can only access those details.

2. Secondly integrating these details with the mgr module and running a separate daemon.