Adding support for RadosGW admin ops API

[leseb]

Hi!

Rook is looking at removing its CLI calls of the radosgw-admin command to directly interact with RadosGW Admin Ops API.
The current existing Go-binding for this API is not as appealing as I thought after looking deeply so I've started my own implementation.

I would like to add it to this repository as I believe go-ceph is the umbrella for all Ceph Go binding.
I realize that this addition will be different from what is being done with rados/rbd/cephfs by important C lib and binding them. However, I see benefit from fostering the go bindings to a single repository, especially under the Ceph organization.

RadosGW Admin Ops API, as its name implies is an API endpoint to do administrative tasks on RadosGW. I'm hoping to bring this new binding to a different maturity level and make it a first-class citizen in Go-Ceph. Hopefully gathering more contributors, primarily coming from both Ceph RGW and Rook teams.

Thanks for your consideration.

[ansiwen]

From previous conversation:

@leseb wrote:

It's a REST API so there is no import C whatsoever. So it's different from what's done for rados/rbd/cephfs.

@phlogistonjohn wrote:

Agreed. Not to put words in Sven's mouth, but I think he was commenting that an ideal way to add support would be through code generation using something like openapi. I don't disagree, but I don't see not having that as a blocker either. I wanted to do code generation for all of go-ceph early on but it proved both difficult and there was too much already done by hand to retrofit a lot of generated code on.

Yes, so my standpoint is, that writing a binding for Rest APIs is nothing a human being should be punished to do. It's different for wrapping a C library, if it is not strictly specified. But for Rest APIs the quota of boilerplate code is very high. So I would instead spend less time in writing an OpenAPI spec for the Rest API instead, and from that you can generate SDKs (binding libraries to Rest APIs are often called SDKs @phlogistonjohn, don't ask me why) for every language, not just Go, and mock server, testing code and documentation.

It would be easier for us (the go-ceph team) to maintain the library, if we don't have to follow what admin ops is doing, but we could convince them to update the OpenAPI spec accordingly. For the C part we never had a real chance to do that, and we are suffering from the lack of API documentation and minor-version-changes. But here I think we have a realistic chance to do it better.

[leseb]

I need to look further into the OpenAPI spec generator so I can give you a proper answer.

[phlogistonjohn]

As @ansiwen accurately quoted - I think that generating code would be preferable, but not having it wouldn't be a blocker.
I worked a little bit with openapi early on when I got involved with the heketi project, so I know a little about it, enough to be dangerous. :-)

As Sven notes, it could pay off to write a general spec - even if it has to live in go-ceph for a time before getting the RGW devs to adopt it. We can discuss the pros/cons of that as needed. @leseb if you want us to be more/less involved with your investigations WRT openapi just say so.

[ansiwen]

I need to look further into the OpenAPI spec generator so I can give you a proper answer.

@leseb since I was dealing with OpenAPI and generated Go code in another project, I'm happy to assist you. You can also have a look at the generated code of that project: https://github.com/Nitrokey/nethsm-pkcs11/tree/master/api
The OpenAPI spec that created this code is here: https://github.com/Nitrokey/nethsm-pkcs11/blob/master/api/api/openapi.yaml

[ansiwen]

@leseb wrote:

I understand you concern of introducing a dependancy in the the library itself but if we want to interact the with rgw admin ops API we must authenticate with s3 and using aws sdk makes that possible. Honestly I've never played with open-api, I don't know what it does.

I still don't quite get it. If we have a thin wrapper for the admin ops API, wouldn't the user just pass in the credentials from the S3 library into the admin ops library? Or does the S3 library define some fancy authentication types, that the admin ops API would take as parameters?

[phlogistonjohn]

I am not quite sure how it typically works, but I think in this case you set a header or header according to an algorithm designated by the aws api, but not of the aws api.
@leseb am I correct in thinking you planned to use github.com/aws/aws-sdk-go/aws/signer/v4 as the authentication mechanism here?

[leseb]

@phlogistonjohn indeed I'd be using github.com/aws/aws-sdk-go/aws/signer/v4.

[phlogistonjohn]

Hi!

Rook is looking at removing its CLI calls of the radosgw-admin command to directly interact with RadosGW Admin Ops API.
The current existing Go-binding for this API is not as appealing as I thought after looking deeply so I've started my own implementation.

Understood. I think it is probably worth taking inspiration from that implementation, and others, but not necessarily using the code.

I would like to add it to this repository as I believe go-ceph is the umbrella for all Ceph Go binding.

Personally, I love this idea.

I realize that this addition will be different from what is being done with rados/rbd/cephfs by important C lib and binding them. However, I see benefit from fostering the go bindings to a single repository, especially under the Ceph organization.

Agreed. There was never a rule that that everything in go-ceph has to be a binding to C APIs, its just that was the smoothest path forward for rados, rbd, and cephfs. The newer cephfs admin and (soon) rbd admin packages themselves are only based on the C libraries enough to use the rados mgr_command and mon_command (etc) functions. I even used interfaces in those sub-packages so in theory you could replace them with something that didn't use the C bindings - even if no such thing exists today.

RadosGW Admin Ops API, as its name implies is an API endpoint to do administrative tasks on RadosGW. I'm hoping to bring this new binding to a different maturity level and make it a first-class citizen in Go-Ceph. Hopefully gathering more contributors, primarily coming from both Ceph RGW and Rook teams.

Thanks for your consideration.

I'm in favor of this idea. There are certainly details to work out but I want to help make this happen. I'd also like to see rook use some of the go-ceph functions for rbd & cephfs - where is makes sense of course. :-)

[leseb]

Ok, I think I'm done with my experiments with OpenAPI, on paper, it seems great but once you start digging it is really overwhelming, especially when trying to make it work with an existing API that might not use the API "standards".

My summary:

• really big as a project, quite overwhelming, the spec is big, I feel like I'm going to spend weeks to figure out which spec option I need to use and probably end up writing a lot of YAML...
• we have limited time
• it feels inadequate when not designing the API from scratch
• it looks like rgw admin ops might need some changes
• obviously, it has numerous benefits like auto-gen code/doc etc but we have a some of it already

In the end, I don't want perfect to be the enemy of good. So I'm going to proceed with my existing internal code and will submit a PR hopefully soon.
Let me know if that works for you to have this in go-ceph, otherwise, I'll find a different place in the Ceph org for it.

Thanks!

[BlaineEXE]

Someone has already implemented a lot of pretty good RGW code for Ceph. https://github.com/QuentinPerez/go-radosgw.

Maybe we could include this in go-ceph easily without having to do much wrapping. Or see if @QuentinPerez would be interested in us including this in the go-ceph code tree.

It's not clear to me if this lib uses the admin API or not, but I am pretty sure it doesn't use the radosgw-admin CLI.

[mattbenjamin]

I thought @leseb had already based his work on upstream contributed code...

[QuentinPerez]

👋 thank you for your gentle message, that would be with pleasure to be merged into go-ceph. here is the conversation if you want to take a look QuentinPerez/go-radosgw#8

[BlaineEXE]

Oh, this is great. I see you've already chatted about this. :)

[anoopcs9]

RGW admin APIs got implemented via #496