np.errstate is not asyncio-safe

[eric-wieser]

While it is threadsafe, the with statement gives the false impression that it's always safe. It is not:

import asyncio
import numpy as np

def divide_by_zero():
    np.float64(1) / 0

async def foo():
    for j in range(3):
        with np.errstate(divide='raise'):
            for i in range(2):
                try:
                    divide_by_zero()
                except FloatingPointError:
                    pass
                else:
                    print("foo: Failed to raise!")
                await asyncio.sleep(0.15)

async def bar():
    for j in range(3):
        with np.errstate(divide='ignore'):
            for i in range(2):
                try:
                    divide_by_zero()
                except FloatingPointError:
                    print("bar: raised anyway!")
                await asyncio.sleep(0.11)

loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.gather(foo(), bar()))
loop.close()

gives:

foo: Failed to raise!
bar: raised anyway!
foo: Failed to raise!

because the error-state is thread-local, but asyncio tasks share a thread.

I don't know if there is any way this can be fixed - it seems like this might be a python bug, and context managers might need an __yield_enter__ and __yield_leave__ mechanism, to be notified when control is leaving the suite via a yield/await

[NeilGirdhar]

Your solution seems reasonable except that it adds a lot of overhead since you need to call __yield_enter__ on the entire stack of active asynchronous context managers.

[eric-wieser]

Using tasklocals from pypi would also solve the problem

[eric-wieser]

@njsmith is two years ahead of me here with PEP 521, but reassuringly has come to a similar conclusion to how this would need to be fixed

[NeilGirdhar]

Yeah, he even explains the differences between that proposal and the task locals. So, it seems that one day Python will have this sorted out. Until then, there is probably no point in investing time in a "deprecation context manager". Still having a list of what you would deprecate would be really useful to motivate its creation one day?

[NeilGirdhar]

FYI: A competing proposal, PEP 550, is being discussed on python-ideas.

[1st1]

PEP 567—Context Variables—has been accepted and implemented in Python 3.7. With its new APIs it's now possible to fix this issue.

[NeilGirdhar]

The new np.printoptions should probably be fixed at the same time.

[seberg]

I think this could be attacked now (and would also close gh-19006). Might be nice to see that it doesn't have any performance downsides, but it seemd a bit like contextvars are faster than "TheDict".

This goes way into the depth of the Python C-API, but (as far as I understand right now!) the work used in allocators should outline how this can be achieved: gh-17582