Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python3Packages.numpy: 1.19.4 -> 1.20.1 #111671

Merged
merged 3 commits into from
Feb 20, 2021
Merged

Conversation

rmcgibbo
Copy link
Contributor

@rmcgibbo rmcgibbo commented Feb 2, 2021

Motivation for this change

https://numpy.org/devdocs/release/1.20.0-notes.html

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

Based on joblib/joblib#1136 (and my testing), there's an incompatibility between numpy 1.20 and joblib <= 1.0, so I've also bumped joblib in this PR as well.

I did not rebuild everything since there are quite a lot of downstream packages that depend on numpy, but I did to a random selection of 62 packages:

NIXPKGS_ALLOW_BROKEN=1 nix build -f .  python39Packages.pyrr python39Packages.pysptk python39Packages.pysqueezebox python39Packages.pysrim python39Packages.pyssim python39Packages.pystray python39Packages.pytesseract python39Packages.pytest-aiohttp python39Packages.pytest-arraydiff python39Packages.pytest-black python39Packages.pytest-doctestplus python39Packages.pytest-filter-subpackage python39Packages.pytest-httpbin python39Packages.pytest-mpl python39Packages.pytest-randomly python39Packages.pytest-sanic python39Packages.pytest-vcr python39Packages.python-engineio python39Packages.python-miio python39Packages.python-mystrom python39Packages.python-opendata-transport python39Packages.python-socketio python39Packages.pytools python39Packages.pytorch python39Packages.pytorch-lightning python39Packages.pytorch-metric-learning python39Packages.pyuavcan python39Packages.pywbem python39Packages.pyworld python39Packages.pyxeoma python39Packages.py_stringmatching python39Packages.qdldl python39Packages.qimage2ndarray python39Packages.qrcode python39Packages.quantities python39Packages.rainbowstream python39Packages.rasterio python39Packages.regenmaschine python39Packages.regional python39Packages.reikna python39Packages.reportlab python39Packages.resampy python39Packages.ripser python39Packages.rising python39Packages.roboschool python39Packages.rtmixer python39Packages.s3fs python39Packages.sasmodels python39Packages.scapy python39Packages.scikit-fmm python39Packages.scikit-optimize python39Packages.scikit-tda python39Packages.scipy python39Packages.scrapy-deltafetch python39Packages.scrapy-fake-useragent python39Packages.scrapy-splash python39Packages.scs python39Packages.seabreeze python39Packages.seekpath python39Packages.seqdiag python39Packages.shapely python39Packages.sharedmem --keep-going

The failures are:

  • python39Packages.rainbowstream: Also broken currently on hydra staging.
  • python39Packages.fixtures. Also broken on currently on hydra staging.
  • python39Packages.sasmodels: Also broken currently on hydra staging.
  • python39Packages.scikitlearn: Not sure why. I think these are new failures not on staging though.

This is the scikit-learn failure: https://gist.github.com/rmcgibbo/21392d1c75d2c0809533bc3a97dce4ba

@rmcgibbo
Copy link
Contributor Author

rmcgibbo commented Feb 2, 2021

Based on this comment, on the scikit-learn tracker, I believe that scikit-learn crash is reproduced on their CI and also not yet fixed, so I don't think there's a patch we can pull in yet.

Copy link
Member

@dotlambda dotlambda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should target staging, not staging-next.

Copy link
Member

@dotlambda dotlambda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both commit messages should start with python3Packages.

@rmcgibbo rmcgibbo changed the base branch from staging-next to staging February 2, 2021 18:19
@rmcgibbo
Copy link
Contributor Author

rmcgibbo commented Feb 2, 2021

It's probably best not to break scikit-learn, since that package has a lot of reverse dependencies.

Therefore, I think either I should disable the failing scikit-learn test, or we should wait to merge this PR until a patch is available from scikit-learn upstream, whichever ya'll think is best.

@rmcgibbo
Copy link
Contributor Author

rmcgibbo commented Feb 3, 2021

Therefore, I think either I should disable the failing scikit-learn test, or we should wait to merge this PR until a patch is available from scikit-learn upstream, whichever ya'll think is best.

I found the scikit-learn patch that makes it work with numpy 1.20.0

Copy link
Member

@dotlambda dotlambda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

6 packages built:
python37Packages.matplotlib python37Packages.scikitlearn python38Packages.matplotlib python38Packages.scikitlearn python39Packages.matplotlib python39Packages.scikitlearn

@rmcgibbo
Copy link
Contributor Author

rmcgibbo commented Feb 8, 2021

Let's not merge this before I update the PR to 1.20.1 instead: https://mail.python.org/pipermail/scipy-dev/2021-February/024523.html

@rmcgibbo rmcgibbo changed the title python3Packages.numpy: 1.19.4 -> 1.20.0 python3Packages.numpy: 1.19.4 -> 1.20.1 Feb 10, 2021
Staging automation moved this from WIP to Ready Feb 17, 2021
Python batch upgrade automation moved this from In progress to Reviewer approved Feb 17, 2021
@FRidh FRidh changed the base branch from staging to python-unstable February 20, 2021 09:20
@FRidh FRidh merged commit 1873c55 into NixOS:python-unstable Feb 20, 2021
Staging automation moved this from Ready to Done Feb 20, 2021
Python batch upgrade automation moved this from Reviewer approved to Done Feb 20, 2021
@rmcgibbo rmcgibbo deleted the numpy branch February 20, 2021 16:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Staging
  
Done
Development

Successfully merging this pull request may close these issues.

None yet

4 participants