New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ceph: retry object health check if creation fails #8708
ceph: retry object health check if creation fails #8708
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm fine with the change but other controllers like pool and cephfs-mirror needs something similar too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just a nit
This pull request has merge conflicts that must be resolved before it can be merged. @BlaineEXE please rebase it. https://rook.io/docs/rook/master/development-flow.html#updating-your-fork |
6f9eb5e
to
7b4f2e6
Compare
done |
7b4f2e6
to
7e8cfcf
Compare
If the CephObjectStore health checker fails to be created, return a reconcile failure so that the reconcile will be run again and Rook will retry creating the health checker. This also means that Rook will not list the CephObjectStore as ready if the health checker can't be started. Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
7e8cfcf
to
5383ba2
Compare
TypeMeta: controllerTypeMeta, | ||
} | ||
objectStore.Spec.Gateway.Port = 80 | ||
setupNewEnvironment := func(additionalObjects ...runtime.Object) *ReconcileCephObjectStore { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add this function to setup a new test environment ReconcileCephObjectStore
so each test is independent.
res, err := r.Reconcile(ctx, req) | ||
assert.NoError(t, err) | ||
assert.True(t, res.Requeue) | ||
}) | ||
|
||
t.Run("success - object store is running", func(t *testing.T) { | ||
// set up an environment that has a ready ceph cluster, and return the reconciler for it | ||
setupEnvironmentWithReadyCephCluster := func() *ReconcileCephObjectStore { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add this additional function to help set up test environment ReconcileCephObjectStores where the CephCluster should be ready, allowing reconcile to proceed. (used in the previously existing successful test and the new test added)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice testing!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. As a follow-up, we need to take the same approach for all health checker, ceph status, osd status, pool mirror, fs mirror. @BlaineEXE are you planning this?
I didn't notice any place where this would be a quick fix for any of the other health checkers, I think because RGW relies on an HTTP API where the rest use the ceph cli IIUC. |
I do want to backport this to 1.7 I think. |
ceph: retry object health check if creation fails (backport #8708)
If the CephObjectStore health checker fails to be created, return a
reconcile failure so that the reconcile will be run again and Rook will
retry creating the health checker. This also means that Rook will not
list the CephObjectStore as ready if the health checker can't be
started.
Signed-off-by: Blaine Gardner blaine.gardner@redhat.com
Description of your changes:
Which issue is resolved by this Pull Request:
Resolves #
Checklist:
make codegen
) has been run to update object specifications, if necessary.