Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing PyTorch MPS support in CI builds due to older SDK #1464

Open
sbrunk opened this issue Jan 26, 2024 · 6 comments
Open

Missing PyTorch MPS support in CI builds due to older SDK #1464

sbrunk opened this issue Jan 26, 2024 · 6 comments

Comments

@sbrunk
Copy link
Contributor

sbrunk commented Jan 26, 2024

I just gave the macosx-arm64 PyTorch from the CI build a try just to realize that GPU support via MPS is missing. The reason seems that we need a sdk version >= 12.3 but the build is using 12.1:

https://github.com/pytorch/pytorch/blob/af8f37c2b6f9f670ee8154e81ebd9fce7843f60e/CMakeLists.txt#L117C1-L118C46

        if(_macosx_sdk_version VERSION_GREATER_EQUAL 12.3)
            set(_MPS_supported_os_version ON)

https://github.com/bytedeco/javacpp-presets/actions/runs/7650509878/job/20880141743#step:2:2541

-- sdk version: 12.1, mps supported: OFF

It also seems like I'm just in time to have missed the release deadline. 🙈

@saudet
Copy link
Member

saudet commented Jan 26, 2024

That new version of the SDK is probably available only in macos 12.x right? I'm not going to do that for this release anyway

@sbrunk
Copy link
Contributor Author

sbrunk commented Jan 26, 2024

Yeah it looks like 12.1 is the latest version available in the 11 runner:
https://github.com/actions/runner-images/blob/main/images/macos/macos-11-Readme.md#installed-sdks
Higher versions up to 13.1 will need 12:
https://github.com/actions/runner-images/blob/main/images/macos/macos-12-Readme.md#installed-sdks

@sbrunk
Copy link
Contributor Author

sbrunk commented Jan 27, 2024

I've created a branch from current master & the zlib 1.3.1 commit from the release branch and did an update to the macOS 12 runner image to get a first impression of how many presets are breaking.

Looking through the results, we get 4 presets that are failing on macOS 12 (without failing on all other platforms).

  • tensorflow
  • scipy
    ../meson.build:80:0: ERROR: Executables created by fortran compiler /usr/local/bin/gfortran-10 are not runnable.
  • ngraph
  • tvm

Other builds either fail on all platforms (skia and mxnet) or only on linux-arm (seems like a mirror is down) so I'm assuming these are unrelated.


I have a hunch for scipy: gfortran-10 is part for the macOS 11 image, but the macOS 12 image starts from gfortran-11. I'll try to add gfortran-10 back in the deploy-macosx workflow. It's part of gcc so it will still be installed already with gcc@10

Perhaps I should also remove gcc-13?

@saudet
Copy link
Member

saudet commented Jan 29, 2024

No, please use GCC 11. Newer versions of GCC tend to be too buggy to be usable. We'll need to update all these lines too:
https://github.com/bytedeco/javacpp-presets/blob/master/.github/actions/deploy-macosx/action.yml#L44-L60

@saudet
Copy link
Member

saudet commented Jan 29, 2024

Ah, yes, please remove GCC 13. Are you saving GCC 10 is still available? If that still works that's OK, yes

@sbrunk
Copy link
Contributor Author

sbrunk commented Jan 29, 2024

Yes it looks like GCC 10 just not installed by default anymore in the new image but it can still be installed via Homebrew like it's already done. I'm doing another run with GCC 13 removed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants