New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ceph: add workaround for Parallel RBD PVC Creation hangs on new pools #8770
Conversation
8c034b2
to
6e516e4
Compare
6e516e4
to
eadbe0f
Compare
eadbe0f
to
38daceb
Compare
@idryomov PTAL |
This commit adds workaround for Parallel RBD PVC Creation hangs on new pools in ceph-csi-troubleshooting.md. Refer: rook#8696 Signed-off-by: Rakshith R <rar@redhat.com>
38daceb
to
6915485
Compare
updated docs with WA which does not leave any stale resources as discussed here ceph/ceph-csi#2521 (comment). PTAL |
|
||
## Parallel RBD PVC creation hangs for new BlockPool | ||
|
||
This issue is specifically present in CephCSI `v3.4.x`, used by rook `>=v1.7.1` and occurs when multiple parallel PVCs creation requests are issued on a newly created uninitialized blockpool. Follow the steps below to workaround the issue: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we currently have a CSI release with the fix? I'm still not clear in what version of CSI this is or will be fixed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we currently have a CSI release with the fix?
@travisn
No, we need ceph pacific release with fix ceph/ceph#43113 to build a new cephcsi built on ceph pacific to resolve this issue.
I'm still not clear in what version of CSI this is or will be fixed.
This will be an issue in CSI v3.4.0
(and also v3.4.1
which will come out soon).
From ceph/ceph#43113 (comment) ,
We should be able to pick up the fix with next ceph pacific release in cephcsi v3.4.2
and make changes in the WA doc at that time.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@travisn @Rakshith-R instead of documenting how about new cephcsi 3.4.1 with octopus as the base image ceph/ceph-csi#2521 (comment). If we do that we don't need to document any workaround.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@travisn @Rakshith-R instead of documenting how about new cephcsi 3.4.1 with octopus as the base image ceph/ceph-csi#2521 (comment). If we do that we don't need to document any workaround.
This is not an option since with ceph octopus deep_copy()
does not work as expected, refer : ceph/ceph-csi#2521 (comment)
Or rook can call rbd pool init <pool_name>
right after creation?
(IMO since it will be used for rbd images so no harm to initialize it too ?)
@idryomov @travisn Do you think its a more preferrable solution ?
cc @Madhu-1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds better instead of documentation 👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So we can avoid this issue if Rook always calls rbd pool init <pool_name>
immediately after pool creation? And we would just do this upon creation of CephBlockPool CRs?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So we can avoid this issue if Rook always calls
rbd pool init <pool_name>
immediately after pool creation? And we would just do this upon creation of CephBlockPool CRs?
Yes, it just needs to be called once. If it is done right after pool create/ setting property, it will avoid this issue.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So we can avoid this issue if Rook always calls
rbd pool init <pool_name>
immediately after pool creation? And we would just do this upon creation of CephBlockPool CRs?Yes, it just needs to be called once. If it is done right after pool create/ setting property, it will avoid this issue.
@Rakshith-R Are you planning on this change in Rook, or somebody else should pick it up?
This pull request has merge conflicts that must be resolved before it can be merged. @Rakshith-R please rebase it. https://rook.io/docs/rook/latest/development-flow.html#updating-your-fork |
Closing this pr, since #8923 is merged and should fix the issue. |
This commit adds workaround for Parallel RBD PVC Creation hangs on
new pools in ceph-csi-troubleshooting.md.
Refer: #8696
Signed-off-by: Rakshith R rar@redhat.com
Description of your changes:
Which issue is resolved by this Pull Request:
Resolves #
Checklist:
make codegen
) has been run to update object specifications, if necessary.