Update to Pytorch 2.3.0

[HGuillemet]

Included in this PR:

• Update to PyTorch 2.3.0
• Add AOTInductor (new way to run models exported from Python)

[HGuillemet]

The C++ API can change depending on the platform.
Currently blocking is Half constructor and cast operator that uses float16_t or ARM64 and float on x86_64 (which doesn't have float16_t):

#if defined(__aarch64__) && !defined(C10_MOBILE) && !defined(__CUDACC__)
  inline Half(float16_t value);
  inline operator float16_t() const;
#else
  inline C10_HOST_DEVICE Half(float value);
  inline C10_HOST_DEVICE operator float() const;
#endif

I'm considering adding some explicit JNI bridge that will call either the float or float16 version depending on the platform, and skip the platform-dependent C++ API.
@saudet, any better idea ?

[saudet]

Just disable float16_t support, it's not portable.

[HGuillemet]

If Parser is told to parse only the #if !defined(__arch64__) branch I get a linking error on Mac Apple Silicon because there is no C++ constructor taking a float 32 or a casting operator returning a float 32 (see check error log).

[HGuillemet]

I can try to patch libtorch source to disable the #if defined(__arch64__) branch and force the creation of float 32 variants on Mac, hoping there is no other arm64-specific parts of he code in libtorch relying on the float16 variant. But it seems more hazardous than the custom JNI trick. What do you think ?

EDIT: I'll try to just add the float variants on Mac, not removing the float16 variants. That seems safer.

[saudet]

If Parser is told to parse only the #if !defined(__arch64__) branch I get a linking error on Mac Apple Silicon because there is no C++ constructor taking a float 32 or a casting operator returning a float 32 (see check error log).

I assume we can convert from/to float and float16_t just like we can between float and double, so it should work if we add casts.

[HGuillemet]

The x64_64-gpu check is killed, probably due to out-of-memory, when compiling some transformer-related cuda code.
Any idea how to work around that ?
Decrease MAKEJ to 2 or 3 ?

[saudet]

Increasing swap space doesn't work?

[HGuillemet]

I'll try that.
Do I change deploy-ubuntu/action.yml or do I add a swap file in pytorch cppbuild ?

[HGuillemet]

I added a check for pytorch in action.yml, but if that works, the best would be to set a dedicated environment variable in workflow files, like SWAP_SPACE and use it in action.yml

[saudet]

Like you said, after build, it doesn't matter if the annotation is there or not, so don't worry about it

[HGuillemet]

My changes to deploy-ubuntu are not run by the worker. No idea why.

[saudet]

To use the actions from your fork, you'll need to change the URL in the workflow

[HGuillemet]

A swap space of 2Gb works, 4 and 0 do not.

[saudet]

Why not 4GB? What happens?

[HGuillemet]

I'm not sure because the log file of the 4G attempt seems truncated. But there remains only 10G of disk space after installations and creation of swap file, and before build. I won't be surprised if we run out of disk space.
We really are at the limits of what the worker can offer...

[saudet]

So, it's probably a better idea to figure out how to free some space?

[HGuillemet]

Better than what ?
Build passes for now, so we are ok.
But in prevision of next Pytorch releases that will need probably more disk space and more memory to compile, finding spare disk space would be useful.
Also upgrading to cuda 12.4 maybe could help.

[saudet]

2GB might not be enough for PyTorch 2.4.0, so before merging this let's wait and see I guess

[saudet]

This looks useful: https://github.com/marketplace/actions/free-disk-space-ubuntu

[saudet]

We can probably just erase a couple of those without the tool though:

=> Android library: Saved 14GiB
=> .NET runtime: Saved 2.7GiB
=> Haskell runtime: Saved 0B
=> Large misc. packages: Saved 5.3GiB
=> Tool cache: Saved 5.9GiB
=> Swap storage: Saved 4.0GiB

Total: Saved 31GiB

Does that mean swap is enabled by default now??

[HGuillemet]

This looks useful: https://github.com/marketplace/actions/free-disk-space-ubuntu

Indeed. We can use this freeing action.

But why waiting for 2.4.0 before merging ?

Does that mean swap is enabled by default now??

I changed deploy-ubuntu so that it reads a SWAP_SIZE env variable and set this variable to 4 in the mkl workflow (deploy-ubuntu added 4G of swap for mkl before my change), and 2 for pytorch.

[saudet]

Let's not make an option, let's just set the swap to a value that works for everything like 4GB, which means using an action here is annoying, so let's just ditch android and dotnet instead like actions/runner-images#2606 (comment)

[HGuillemet]

Ok for the ditching, but keeping option to use some disk space for extra swap, depending on the workflow needs, seems interesting to me.

[saudet]

Why do you want to make an option? It's not going to used by anything

[HGuillemet]

Maybe some other builds would fail if we remove 4G of disk for the swap. Like pytorch currently does.
Why do you want to remove the option ? There are already other options like CI_DEPLOY_NEED_* why not CI_DEPLOY_NEED_SWAP ?

[saudet]

Those options are there because some builds fail when they are true, but some others fail when they are false. That doesn't happen for something like swap space

[HGuillemet]

Is dotnet used by any build ?
Is /usr/local/android used by android builds ?

[HGuillemet]

11Gb saved from ditching android and dotnet