-
Notifications
You must be signed in to change notification settings - Fork 2.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ceph: add opa config options to ObjectStoreSpec #8731
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -37,6 +37,7 @@ spec: | |||||
preservePoolsOnDelete: true | ||||||
gateway: | ||||||
# sslCertificateRef: | ||||||
# caBundleRef: | ||||||
port: 80 | ||||||
# securePort: 443 | ||||||
instances: 1 | ||||||
|
@@ -92,6 +93,7 @@ The gateway settings correspond to the RGW daemon settings. | |||||
|
||||||
* `type`: `S3` is supported | ||||||
* `sslCertificateRef`: If specified, this is the name of the Kubernetes secret(`opaque` or `tls` type) that contains the TLS certificate to be used for secure connections to the object store. Rook will look in the secret provided at the `cert` key name. The value of the `cert` key must be in the format expected by the [RGW service](https://docs.ceph.com/docs/master/install/ceph-deploy/install-ceph-gateway/#using-ssl-with-civetweb): "The server key, server certificate, and any other CA or intermediate certificates be supplied in one file. Each of these items must be in PEM form." | ||||||
* `caBundleRef`: If specified, this is the name of the Kubernetes secret (type `opaque`) that contains ca-bundle to use. The secret must be in the same namespace as the Rook cluster. Rook will look in the secret provided at the `cabundle` key name. | ||||||
* `port`: The port on which the Object service will be reachable. If host networking is enabled, the RGW daemons will also listen on that port. If running on SDN, the RGW daemon listening port will be 8080 internally. | ||||||
* `securePort`: The secure port on which RGW pods will be listening. A TLS certificate must be specified either via `sslCerticateRef` or `service.annotations` | ||||||
* `instances`: The number of pods that will be started to load balance this object store. | ||||||
|
@@ -171,12 +173,12 @@ Rook-Ceph always keeps the bucket and the user for the health check, it just doe | |||||
|
||||||
## Security settings | ||||||
|
||||||
Ceph RGW supports encryption via Key Management System (KMS) using HashiCorp Vault. Refer to the [vault kms section](ceph-cluster-crd.md#vault-kms) for detailed explanation. | ||||||
The `security` section contains settings related to Key Management System (KMS) encryption and Open Policy Agent (OPA) authentication for the RGW. | ||||||
|
||||||
Ceph RGW supports encryption via KMS using HashiCorp Vault. Refer to the [vault kms section](ceph-cluster-crd.md#vault-kms) for detailed explanation. | ||||||
If these settings are defined, then RGW establish a connection between Vault and whenever S3 client sends a request with Server Side Encryption, | ||||||
it encrypts that using the key specified by the client. For more details w.r.t RGW, please refer [Ceph Vault documentation](https://docs.ceph.com/en/latest/radosgw/vault/) | ||||||
|
||||||
The `security` section contains settings related to KMS encryption of the RGW. | ||||||
|
||||||
```yaml | ||||||
security: | ||||||
kms: | ||||||
|
@@ -190,7 +192,7 @@ security: | |||||
tokenSecretName: rgw-vault-token | ||||||
``` | ||||||
|
||||||
For RGW, please note the following: | ||||||
For RGW, please note the following wrt KMS: | ||||||
|
||||||
* `VAULT_SECRET_ENGINE` option is specifically for RGW to mention about the secret engine which can be used, currently supports two: [kv](https://www.vaultproject.io/docs/secrets/kv) and [transit](https://www.vaultproject.io/docs/secrets/transit). And for kv engine only version 2 is supported. | ||||||
* The Storage administrator needs to create a secret in the Vault server so that S3 clients use that key for encryption | ||||||
|
@@ -199,6 +201,15 @@ $ vault write -f transit/keys/<mybucketkey> exportable=true # transit engine | |||||
|
||||||
* TLS authentication with custom certs between Vault and RGW are yet to be supported. | ||||||
|
||||||
Here example for OPA config: | ||||||
```yaml | ||||||
opa: | ||||||
url: opa.rook-ceph:8181/v1/data/ceph/authz/allow | ||||||
token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9 | ||||||
verifySSL: true | ||||||
``` | ||||||
The config options include url, token for the OPA service. The TLS certs can be passed with help `caBundleRef`. For more details w.r.t RGW, please refer [Ceph OPA documentation](https://docs.ceph.com/en/latest/radosgw/opa/). | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
||||||
## Deleting a CephObjectStore | ||||||
|
||||||
During deletion of a CephObjectStore resource, Rook protects against accidental or premature | ||||||
|
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -236,6 +236,28 @@ type SecuritySpec struct { | |||||
// +optional | ||||||
// +nullable | ||||||
KeyManagementService KeyManagementServiceSpec `json:"kms,omitempty"` | ||||||
// OpenPolicyAgentService defines spec for the OPA service | ||||||
// +optional | ||||||
// +nullable | ||||||
OpenPolicyAgentService *OpenPolicyAgentServiceSpec `json:"opa,omitempty"` | ||||||
} | ||||||
|
||||||
// OpenPolicyAgentServiceSpec contains options for communicating with the OPA server | ||||||
type OpenPolicyAgentServiceSpec struct { | ||||||
// URL of the OPA service for the authentication | ||||||
// +kubebuilder:validation:MinLength=1 | ||||||
// +kubebuilder:validation:Required | ||||||
// +required | ||||||
URL string `json:"url,omitempty"` | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Since this is required. |
||||||
// Indicate whether the server certificate is validated by the client or not | ||||||
// +optional | ||||||
// +nullable | ||||||
VerifySSL *bool `json:"verifySSL,omitempty"` | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is the behavior of "not set" and "false" the same? If so, how about removing the pointer for simplicity?
Suggested change
|
||||||
// Token is the token for the OPA | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
// +kubebuilder:validation:MinLength=1 | ||||||
// +kubebuilder:validation:Required | ||||||
// +required | ||||||
Token string `json:"token,omitempty"` | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
} | ||||||
|
||||||
// KeyManagementServiceSpec represent various details of the KMS server | ||||||
|
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -21,6 +21,7 @@ import ( | |
"fmt" | ||
"path" | ||
"reflect" | ||
"strconv" | ||
"strings" | ||
|
||
"github.com/hashicorp/vault/api" | ||
|
@@ -310,6 +311,20 @@ func (c *clusterConfig) makeDaemonContainer(rgwConfig *rgwConfig) v1.Container { | |
) | ||
} | ||
} | ||
if c.store.Spec.Security != nil && c.store.Spec.Security.OpenPolicyAgentService != nil { | ||
container.Args = append(container.Args, | ||
cephconfig.NewFlag("rgw use opa authz", "true"), | ||
cephconfig.NewFlag("rgw opa url", | ||
c.store.Spec.Security.OpenPolicyAgentService.URL), | ||
cephconfig.NewFlag("rgw opa token", | ||
c.store.Spec.Security.OpenPolicyAgentService.Token), | ||
) | ||
if c.store.Spec.Security.OpenPolicyAgentService.VerifySSL != nil { | ||
container.Args = append(container.Args, | ||
cephconfig.NewFlag("rgw opa verify ssl", | ||
strconv.FormatBool(*c.store.Spec.Security.OpenPolicyAgentService.VerifySSL))) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is the bool pointer really necessary? If it's not required why not just let the default. |
||
} | ||
} | ||
return container | ||
} | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -210,6 +210,19 @@ func TestSSLPodSpec(t *testing.T) { | |
s, err = c.makeRGWPodSpec(rgwConfig) | ||
assert.NoError(t, err) | ||
podTemplate = cephtest.NewPodTemplateSpecTester(t, &s) | ||
podTemplate.RunFullSuite(cephconfig.RgwType, "default", "rook-ceph-rgw", "mycluster", "quay.io/ceph/ceph:myversion", | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What is this actually testing? |
||
"200", "100", "1337", "500", /* resources */ | ||
"my-priority-class") | ||
verifySSL := true | ||
c.store.Spec.Security = &cephv1.SecuritySpec{ | ||
OpenPolicyAgentService: &cephv1.OpenPolicyAgentServiceSpec{ | ||
URL: "", | ||
Token: "", | ||
VerifySSL: &verifySSL, | ||
}} | ||
s, err = c.makeRGWPodSpec(rgwConfig) | ||
assert.NoError(t, err) | ||
podTemplate = cephtest.NewPodTemplateSpecTester(t, &s) | ||
podTemplate.RunFullSuite(cephconfig.RgwType, "default", "rook-ceph-rgw", "mycluster", "quay.io/ceph/ceph:myversion", | ||
"200", "100", "1337", "500", /* resources */ | ||
"my-priority-class") | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's avoid non-obvious acronyms in the CR property names