ENH: Implement correct scalar and integer overflow errors for NEP 50

[seberg]

This adds the main missing chunk of NEP 50 support. Unfortunately, I feel this is still fairly complex, although I am not sure it can be helped...

For the ufuncs, it may be possible to move the special handling to later. For the scalars, the special handling has a bit of additional complexity because of the "defer" check (if we did it always it would be easier logic flow).
Unfortunately, for the scalars it makes sense to keep the "faster" version for the safe cast around (for speed reasons).

The other question is whether we want to do this as a distinct method on the DType, although I don't think we have to settle that yet. It should be easy to change.

A small oddity is that the user has to resolve from one of the "int" dtypes (int32, int64, uint64, object). That seems OK to me, although it also seems like something that we may well know better later.

---

Marking as draft, I need to let this sink in a bit to see if I have a better idea. Bug, this should be ready for a test ride and may complete NEP 50 logic (with the exception of possibly allowing Python scalars for example in np.can_cast(123, np.int8))

[seberg]

OK, I still don't love the scalar code, but considering that the ufunc code is at least well localized and the scalar code could use more cleanup (as in conversion function introduction probably via C++), I will mark this as ready for review. I simply did not have a clear idea for how to improve it meaningfully.

[mattip]

Is this line really skipped during testing or is codecoverage mistaken?

[mattip]

whether we want to do this as a distinct method on the DType

I would prefer not to, if we can avoid more methods

[seberg]

whether we want to do this as a distinct method on the DType

I would prefer not to, if we can avoid more methods

Hmm, trying to untangle the state of this. The reason for custom handling was that we can allow these to remain unmodified:

np.uint8(-1)
# And also:
np.array([-1], np.uint8)
arr_uint8[0] = -1

i.e. we still allow the forced cast in some cases, but not in the new paths.

This seems almost exclusively relevant for that use-case where you use -1 to mean UINT_MAX, but is a relevant use-case there.

However, we are already changing quite a lot, so maybe it is OK to just change that as well, with the one possible exception that we could hard-code np.uint(-1) to keep working (but not the other two examples).

The other question is only relevant if we do not want to change the above. That was where I was coming from. But I don't really see that it makes things clearer by not being a method (since methods are the typical path now, I think).

[seberg]

Lets discuss this on Wednesday a bit, maybe we can make a decision then (I think the code coverage/tests should be fine now, but will fix it up if not).

[seberg]

OK, this is rebased to account for gh-22385. (I will have another scroll through myself later today just to make sure there are no small cleanups needed still).

[mattip]

Should the tests reset global state at finish?

[seberg]

Yes, there is a module wide auto-fixture that defaults to "weak_and_warn" and resets the state at the end.

[seberg]

Can we move forward with this, soon? I would like to make sure we have full optional NEP 50 logic by next release for testing.
I suppose this is the last very important thing, maybe np.can_cast(123, dtype=np.uint8) should also be "special" (but even if, I suspect it has much less impact in practice).

[mattip]

Thanks @seberg