New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MemoryObjectStream can drop items when the receiving end is cancelled #728
Comments
did you mean to include that
|
if I remove that assertion from your reproducer, then I believe your reproducer demonstrates at least one bug, but possibly two:
I think it's worth discussing these two things separately because (1) is the urgent issue (data loss), and it can be fixed first without doing the harder thing of also fixing (2). in more detail:
(1) is the urgent problem (dropped items). since it's straightforward to fix, I think that it should be fixed first, and whether (2) is a bug or not (and, if so, how to resolve it in a backend-agnostic manner) can be figured out later (perhaps in a separate issue). |
Ah, my intention was to create a memory object stream with a buffer. Then the |
I've adjusted the snippet accordingly. |
Compare to the use of import anyio
from anyio import (
CancelScope, create_task_group,
wait_all_tasks_blocked,
)
from trio import open_memory_channel
async def receiver(receive, task_status):
with CancelScope() as cancel_scope:
task_status.started(cancel_scope)
await receive.receive()
async def main():
send, receive = open_memory_channel(1)
with send, receive:
async with create_task_group() as tg:
cancel_scope = await tg.start(receiver, receive)
await wait_all_tasks_blocked()
cancel_scope.cancel()
send.send_nowait("item")
assert receive.receive_nowait() == "item"
anyio.run(main, backend="trio") |
yes, agreed, assuming that your intention is to test for (2). but only (1) is needed to ensure that the stream isn't dropping items. (2) is a stricter criterion. it's not clear to me whether (2) is actually a bug or problematic. what did you think about
|
i have just written some more tests that are also closely related, plus a closely related fix. (these tests are currently in branch #729 (2b09585).) here is a summary of all of these variations of #146 that that branch is currently testing for (I think that this list makes a good summary of the relationships between these issues):
|
Check if the receiving task has a pending cancellation before sending an item. Fixes #728.
Things to check first
I have searched the existing issues and didn't find my bug already reported there
I have checked that my bug is still present in the latest release
AnyIO version
4.3.0
Python version
3.8
What happened?
If a task (A) sends an item to another task (B) via a memory object stream, and task B is in a state of waiting for an item, and has a pending cancellation, the item is still sent to B but as cancellation is then delivered to B, that item is essentially dropped on the floor.
A similar issue was reported in #146, but it seems that it wasn't fixed as thoroughly as I had hoped.
How can we reproduce the bug?
The above snippet reproduces the problem (
WouldBlock
is raised) on both asyncio and Trio. On Trio, ifcreate_memory_object_stream()
is replaced withtrio.open_memory_channel()
, the snippet no longer raises an exception.The text was updated successfully, but these errors were encountered: