Mix anyio and asyncio code.

[laker-93]

Things to check first

• I have searched the existing issues and didn't find my feature already requested there

Feature description

I am looking for documentation to advise if mixing anyio and asyncio code is supported. My use case is a large Python 3.8 asyncio code base. I'd like to use anyio in order to incrementally use SC for creating and cancelling tasks. I have run numerous tests using a mix of both and the behaviour looks correct. However I am aware of #374 which occurs when mixing between anyio 3.x and asyncio where the CancelledError does not propagate out of the task group context manager. Are there any other gotchas one should be aware of when using a mix of asyncio and anyio for task management?

Use case

Incrementally converting from normal asyncio task management to Structured Concurrency with anyio.

[agronholm]

Well, as you can see from the issue, it was resolved. From AnyIO 4.0 onwards, that is no longer a problem. The only gotcha is how scoped cancellation works. A pattern where asyncio code uses await in a finally: block can be adversely affected by scoped cancellation, as any async code running within a cancelled scope can receive multiple cancellation exceptions. This aspect of asyncio is fundamentally broken, however, as there are other reasons finalizing code could get cancelled more than once.

[laker-93]

I am aware it is fixed in 4.0 however I am targeting anyio 3.x and cannot immediately migrate to anyio 4.
Can you provide an example of await being cancelled in a finally block?

[agronholm]

with CancelScope() as scope:
    try:
        connection = await connect()
        ...
    finally:
        await connection.close()   # this might get interrupted if the finally block is entered due to the scope being cancelled

[smurfix]

In this situation you can easily fix the problem by wrapping another with CancelScope(shield=True) around the close.

However, as soon as you need to work with an AsyncExitStack or a similar construct, using a wrapper does not work any more because you're violating the structure nesting rules.

Nowadays anyio's CancelScope has a modifiable shield attribute, so one can still fix this, but it's a rabbit hole where hitting Trio's internal error is all too easily triggered.

[agronholm]

You can't fix this in third party libs.

[smurfix]

You're right, of course; I should have written "work around the problem so that your tests say everthing's OK (but still might fail randomly in production)" instead of "fix".