MAINT: backports/prep for 1.13.1 #20632
Conversation
* Fixes scipygh-20300 by syncing `pocketfft` again, this time to completely disable `aligned_alloc`; see scipy/pocketfft#3 for details, but in short our more conservative shim was not sufficient for `conda-forge`, so let's just do the same thing NumPy did... [skip cirrus] [skip circle]
This was reported to cause test failures under windows + MKL in conda-forge cf scipy#20471
A slight tolerance violation was reported on conda-forge in scipy#20472
A small tolerance violation reported on conda-forge in scipy#20472
An exact equality was reported as flaky on conda-forge in scipy#20472 The tolerance violations are of the order of fp noise (< 2e-16), and I don't think that pickling/unpickling guarantees bit-to-bit compatibility. In principle, this may invoke recalculations and those may be subject to fp noise. So I think it's OK to only require allclose(atol=eps) instead of exact equality.
tol violation observed on conda-forge on win+blis; suggested in scipy#20474 (comment)
f2py check was just too strict; LAPACK docs indicate that for N=1, lwork>=1 is acceptable: * LWORK (input) INTEGER * The dimension of the array WORK. * If N <= 1, LWORK must be at least 1. * If JOBZ = 'N' and N > 1, LWORK must be at least 2*N+1. * If JOBZ = 'V' and N > 1, LWORK must be at least * 1 + 6*N + 2*N**2. https://www.netlib.org/lapack/explore-3.1.1-html/ssyevd.f.html closes scipygh-20512
Co-authored-by: Ralf Gommers <ralf.gommers@gmail.com>
…ith an infinite weight (scipy#20573) * Fix exact rotations at 180 deg, improve near 180 deg Comments * Tests for exact near 180 deg rotations * Fix tests * Code review updates --------- Co-authored-by: Scott Shambaugh <scottshambaugh@users.noreply.github.com>
* Apply "old version" of the patch provided at scipygh-20527
* Update the SciPy `1.13.1` release notes following a series of backports. [skip cirrus]
tylerjereddy
requested review from
larsoner,
rgommers,
person142,
steppi,
peterbell10,
perimosocordiae,
andyfaff and
ev-br
as code owners
Showed up as a linting error in an unrelated PR for me: ``` scipy/interpolate/_interpolate.py:918:30: UP032 [*] Use f-string instead of `format` call scipy/interpolate/_interpolate.py:1972:30: UP032 [*] Use f-string instead of `format` call ``` This should not happen; the old code is fine, so this check needs to be silenced or fixed separately. Similar to scipygh-20601. [skip ci]
* Fixes scipygh-19423 * Add a few more `case` statements to account for the (i.e., Windows) data types that don't have a fixed width, and add a regression test. [skip circle] [skip cirrus]
* attempt to deal with scipygh-20576
* some minor import fixes following a large series of backports
* more import cleanups after backport activity to make the linter happy
|
I've added substantially more backports and a few manual shims--we're back at parity with merged backport-labeled PRs. Let's see where the regular CI lands. I did have to revert and unlabel one PR (gh-20643) based on local testing with clang-16 on Mac, but the rest made it in, so far. I expanded the TODO list above based on known CI failure in Edit: between gh-20727 and gh-20533, I suspect we'll be able to get the regular CI passing today. |
[skip circle]
* We're seeing CI failures related to an undesirable bump to Python `3.12` in this job, when the intention was clearly to respect the Python version specific in the GHA matrix. I didn't check too closely why exactly it suddenly started happening, but some packages weren't ready for `3.12` yet on this job (`scikit-umfpack` in particular) and I don't see too much harm in adding an extra pin to respect the intention for the Python version.
* attempt to fix `M1 test - openblas` MacOS ARM CI job based on some `gfortran` shims applied to a similar job on `main`
|
The regular CI is finally passing here (1 or 2 known flakes sometimes require restarts). I'm going to proceed with a test pass of the wheel builds before updating the release notes again, because we may see shenanigans with the bump to OpenBLAS |
* This is an empty commit to test the wheel building in the above PR. [wheel build]
|
For MacOS ARM wheel failure |
* Changes needed to get MacOS ARM wheels building again, based on debug work on my fork. [wheel build]
|
I pushed in a patch that helped deal with the MacOS ARM wheel build issue on my fork, but now of course there's another problem with the new |
* Need to pin `pythran` version in a CI job because of a recent new release. [wheel build]
* Increase the Cirrus CI Linux aarch64 job timeout to 72 minutes; it timed out twice in a row via manual restarts.
…) [wheel build] * BUG: fix zipf.pmf for integer k * ENH: increase range for zipfian.pmf for integer k --------- Co-authored-by: Tyler Reddy <tyler.je.reddy@gmail.com> [wheel build]
| CIBW="DYLD_LIBRARY_PATH=$GFORTRAN_LIB:/opt/arm64-builds/lib delocate-listdeps {wheel} &&\ | ||
| DYLD_LIBRARY_PATH=$GFORTRAN_LIB:/opt/arm64-builds/lib delocate-wheel --require-archs {delocate_archs} -w {dest_dir} {wheel}" | ||
| CIBW="DYLD_LIBRARY_PATH=$GFORTRAN_LIB:$LIB_PATH delocate-listdeps {wheel} &&\ | ||
| DYLD_LIBRARY_PATH=$GFORTRAN_LIB:$LIB_PATH delocate-wheel --require-archs {delocate_archs} -w {dest_dir} {wheel}" |
There was a problem hiding this comment.
@rgommers @h-vetinari This PR is taking shape I think--the full CI matrix (including wheels) is finally passing. The changes inline above are probably the most annoying/controversial ones I had to make manually (they are probably a superset of what is really needed to get Mac ARM wheels passing).
Any remaining objections/concerns for 1.13.1 release? We're at parity with the current list of merged backport-labeled PRs (30 in total now). I have two checklist items left above, but they are basically just updating the relnotes and reading the diff carefully before proceeding.
@mdhaber @dschult @perimosocordiae any more backports that are important for stats or sparse for 1.13.1?
I think if folks are happy I'd try to get 1.13.1 out before branching 1.14.0, with a modest delay likely for the latter, but hopefully not too substantial (I'm blocking off time for release notes work tomorrow).
There was a problem hiding this comment.
Any remaining objections/concerns for
1.13.1release? We're at parity with the current list of merged backport-labeled PRs (30 in total now).
None that I can think of. This is already quite large, so +1 for shipping it soon.
The changes for macOS gfortran should be okay - whatever works there I think is fine in a maintenance branch.
There was a problem hiding this comment.
I didn't get to see this earlier, but I think this has all the backports I had in mind.
| @@ -19,6 +19,7 @@ cirrus_wheels_linux_aarch64_task: | |||
| platform: linux | |||
| cpu: 2 | |||
| memory: 4G | |||
| timeout_in: 72m | |||
There was a problem hiding this comment.
I tried restarting twice; this is probably cheaper than needing to hit restart several times during the rel process.
There was a problem hiding this comment.
+1 that makes sense to me, it was always close to the limit.
|
I'll have a look at the Pythran issue. In the EDIT: okay, no issue after all. It was this: which is expected is we do a non-isolated build and only pythran 0.16.0 is installed. So everything is fine in |
* Update the SciPy `1.13.1` release notes following substantial backport activity. [skip cirrus]
|
A new test failure showed up in a Windows CI job (https://github.com/scipy/scipy/actions/runs/9194423643/job/25287826002?pr=20632). I'm going to restart that now and hope it is a fluke (cc @tupui since it is Beyond that, I read through the full diff again and didn't see anything I didn't expect. The rendered versions of the |
|
Regular CI is green again, proceeding with |
|
We have a 1.5:1 commit:backport PR ratio, which is a bit high, but I'd rather not play with commit history/fire. |
|
somehow a Cirrus job still timed out at an hour or so during the rel process.. restarting it is I guess. |
We're about 3.5 weeks from the planned branching for SciPy
1.14.x. Since1.13.0was a bit rushed to support NumPy 2.x, and indeed I made a few oversights for backporting, it may make sense to at least assess how much work it would be to deliver1.13.1. I've cleaned up the backport labels a bit too, since there were almost 50 of them. Many were things I could have backported to1.12.x, but I think that would stretch me (and reviewers/support) too thin at this point.Backports included here (so far):
root(method="lm"). #20401spherical_inforn=0andz=0#20527 (danger: I applied the patch manually to the old version of thespecialcode; I did backport the new test though, and it did fail before/pass after my manual application of a slightly different patch){minimum, maximum}_filter#20653__array__(copy=True)#20533xwith NaN and slice length > 50 #20592zipf.pmfandzipfian.pmffor int32k#20702Backports already merged to the release branch:
TODO:
mainat time of writing)doc/source/.jupyterlite.doit.dbisn't gitignored on this branch/PR when I do the doc build (which otherwise passes at the time of writing) locallyUP031 Use format specifiers ...maybe?