Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: matlab files with deeply lists of arrays with different sizes are no longer created #17992

Closed
Rumbelstilzchen opened this issue Feb 17, 2023 · 2 comments · Fixed by #18104
Labels
defect A clear bug or issue that prevents SciPy from being installed or used as expected scipy.io
Milestone

Comments

@Rumbelstilzchen
Copy link

Rumbelstilzchen commented Feb 17, 2023

Describe your issue.

After updating numpy to a version >=1.24 an error is thrown as the behavior of numpy asanyarry has changed from a deprecation warning to an error. For reference:
https://numpy.org/doc/stable/release/1.24.0-notes.html
and
numpy/numpy#22004

Possible workaround:
Wrapping the line 493 of _mio5.py within a try catch:

narr = np.asanyarray(source)

Replace:

narr = np.asanyarray(source)

with

try:
    narr = np.asanyarray(source)
except ValueError:
    narr = np.asanyarray(source, dtype=object)

Reproducing Code Example

import scipy.io as sio
import numpy as np

array_one = np.random.random((5,3))
array_two = np.random.random((6,3))

list_of_arrays = [array_one, array_two]

sio.savemat("D:\\lists.mat", {'data': list_of_arrays}, long_field_names=True, do_compression=True)

Error message

Traceback (most recent call last):
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2021.2.3\plugins\python-ce\helpers\pydev\pydevconsole.py", line 364, in runcode
    coro = func()
  File "<input>", line 10, in <module>
  File "...\venv310\lib\site-packages\scipy\io\matlab\_mio.py", line 300, in savemat
    MW.put_variables(mdict)
  File "...\venv310\lib\site-packages\scipy\io\matlab\_mio5.py", line 884, in put_variables
    self._matrix_writer.write_top(var, name.encode('latin1'), is_global)
  File "...\venv310\lib\site-packages\scipy\io\matlab\_mio5.py", line 633, in write_top
    self.write(arr)
  File "...\venv310\lib\site-packages\scipy\io\matlab\_mio5.py", line 651, in write
    narr = to_writeable(arr)
  File "...\venv310\lib\site-packages\scipy\io\matlab\_mio5.py", line 493, in to_writeable
    narr = np.asanyarray(source)
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.

SciPy/NumPy/Python version information

Scipy: 1.10.0; Numpy: 1.24.2

@Rumbelstilzchen Rumbelstilzchen added the defect A clear bug or issue that prevents SciPy from being installed or used as expected label Feb 17, 2023
@tupui tupui added the scipy.io label Feb 17, 2023
@matthew-brett
Copy link
Contributor

Thanks for the report. Do you happen to have an example that triggers the error that we can use for a test?

@Rumbelstilzchen
Copy link
Author

Sorry, I do not understand. What in addition to the example under "Reproducing Code Example" from my initial post is needed?
On my setup this raises the error and aborts writing the mat-file.

tylerjereddy added a commit to tylerjereddy/scipy that referenced this issue Mar 5, 2023
Fixes scipy#17992

* add a regression test + the suggested fix for the issue
above, allowing for the saving matlab files from ragged
NumPy arrays
@tupui tupui added this to the 1.11.0 milestone Mar 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
defect A clear bug or issue that prevents SciPy from being installed or used as expected scipy.io
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants