BUG: ufunc 'isfinite' not supported for datetime64 data on version 1.24.0

[WillianFuks]

Describe the issue:

Starting from numpy==1.24.0 matplotlib can't process input data of type datetime64 anymore.

Reproduce the code example:

import pandas as pd
import matplotlib.pyplot as plt


d = pd.DataFrame({'x0': 1, 'x1': 3}, index=pd.date_range(start='2020-01-01', periods=10))
ax = plt.subplot(1, 1, 1)
ax.fill_between(d.index, d['x0'], d['x1'])  # raises

Error message:

Traceback (most recent call last)
Cell In[5], line 1
----> 1 ax.fill_between(d.index, d['x0'], d['x1'])

File ~/Documents/repos/tfcausalimpact/.env310/lib/python3.10/site-packages/matplotlib/__init__.py:1423, in _preprocess_data.<locals>.inner(ax, data, *args, **kwargs)
   1420 @functools.wraps(func)
   1421 def inner(ax, *args, data=None, **kwargs):
   1422     if data is None:
-> 1423         return func(ax, *map(sanitize_sequence, args), **kwargs)
   1425     bound = new_sig.bind(ax, *args, **kwargs)
   1426     auto_label = (bound.arguments.get(label_namer)
   1427                   or bound.kwargs.get(label_namer))

File ~/Documents/repos/tfcausalimpact/.env310/lib/python3.10/site-packages/matplotlib/axes/_axes.py:5367, in Axes.fill_between(self, x, y1, y2, where, interpolate, step, **kwargs)
   5365 def fill_between(self, x, y1, y2=0, where=None, interpolate=False,
   5366                  step=None, **kwargs):
-> 5367     return self._fill_between_x_or_y(
   5368         "x", x, y1, y2,
   5369         where=where, interpolate=interpolate, step=step, **kwargs)

File ~/Documents/repos/tfcausalimpact/.env310/lib/python3.10/site-packages/matplotlib/axes/_axes.py:5272, in Axes._fill_between_x_or_y(self, ind_dir, ind, dep1, dep2, where, interpolate, step, **kwargs)
   5268         kwargs["facecolor"] = \
   5269             self._get_patches_for_fill.get_next_color()
   5271 # Handle united data, such as dates
-> 5272 ind, dep1, dep2 = map(
   5273     ma.masked_invalid, self._process_unit_info(
   5274         [(ind_dir, ind), (dep_dir, dep1), (dep_dir, dep2)], kwargs))
   5276 for name, array in [
   5277         (ind_dir, ind), (f"{dep_dir}1", dep1), (f"{dep_dir}2", dep2)]:
   5278     if array.ndim > 1:

File ~/Documents/repos/tfcausalimpact/.env310/lib/python3.10/site-packages/numpy/ma/core.py:2360, in masked_invalid(a, copy)
   2332 def masked_invalid(a, copy=True):
   2333     """
   2334     Mask an array where invalid values occur (NaNs or infs).
   2335 
   (...)
   2357 
   2358     """
-> 2360     return masked_where(~(np.isfinite(getdata(a))), a, copy=copy)

TypeError: ufunc 'isfinite' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

NumPy/Python version information:

1.24.0 3.10.9 (main, Dec 7 2022, 01:12:00) [GCC 9.4.0]

Context for the issue:

All plotting functionality from our packages that rely on dated index data stopped working. The solution for now was to drop numpy version to 1.23.4, not sure if there's some workaround but for what we researched nothing seemed to fix it so far.

[LoicGrobol]

The bug affects ma.masked_invalid for all pandas Series of any dtype: this works in 1.23 bit not in 1.24.0

import pandas as pd
from numpy import ma
s = pd.Series([0.0,1.0,2.0,3.0])
ma.masked_invalid(s)

The issues looks to me like getdata is no longer guaranteed to return something that isfinite knows how to process but I did not dig deeper.

[LoicGrobol]

Correction: the issue seems to come from #22046, which changed the behaviour of masked_invalid from explicitly building an array from its argument to expecting it to be an array already.

I can probably provide a fix PR restoring the explicit array construction but I'm unsure of whether that's the best way forward.

[seberg]

Thanks for reporting. The first part is already fixed by gh-22838. But it seems matplotlib here also notice gh-22720 (i.e. rely on the full mask being instantiated)...

[ecarruda]

It is also breaking float64 types:

SingleBlockManager
Items: Float64Index([ -2.408224685280692, -2.3739711708557603, -2.3397176564308286,
               -2.305464142005897,  -2.271210627580966,  -2.236957113156034,
              -2.2027035987311026,  -2.168450084306171, -2.1341965698812393,
              -2.0999430554563077,
              ...
                4.099943055456308,   4.134196569881238,   4.168450084306171,
                4.202703598731102,   4.236957113156034,   4.271210627580965,
                4.305464142005897,   4.339717656430828,   4.373971170855761,
               4.4082246852806914],
             dtype='float64', length=200)
NumericBlock: 200 dtype: float64

  res = masked_where(~(np.isfinite(getdata(a))), a, copy=copy)

E TypeError: ufunc 'isfinite' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''