Pass -stdlib=libc++ for CUDA with clang trunk.

[jlebar]

Without this, clang defaults to libstdc++. This doesn't compile with clang trunk:

/opt/compiler-explorer/gcc-snapshot/lib/gcc/x86_64-linux-gnu/14.0.0/../../../../include/c++/14.0.0/bits/std_abs.h:137:7: error: __float128 is not supported on this target

[jlebar]

cc @Artem-B

[jlebar]

cc @mattgodbolt

[partouf]

Can you share a link to a specific example, I'm a little curious as to why no one else has mentioned anything about this for 2-3 years

[Artem-B]

It's a fairly recent development. I guess it was introduced with a libstdc++ upgrade on godbolt machines.
All you need to reproduce it is to compile the default CUDA example w/ clang: https://cuda.godbolt.org/z/G6oxT8oW3

Looks like the early signs of this were already present couple of years ago. E.g. boostorg/math#181 (comment). It's possible that something in clang had changed recently and enabled support for float128 by default on the host side and we've propagated the macros used to detect it from the host to the GPU compilation. I'll take a look at what could be done about that on clang side.

[mattgodbolt]

I'd much prefer pinning to a known good gcc version, not to change the C++ standard library. That's a far more surprising (to me) change, as it's not the default.

[Artem-B]

Huh. I've found the reason I can't reproduce the issue locally.
I'm on Debian Linux, and in the libstdc++ installed on my machine I see this:

#if !defined(__STRICT_ANSI__) && defined(_GLIBCXX_USE_FLOAT128) && !defined(__CUDACC__)
  __extension__ inline _GLIBCXX_CONSTEXPR
  __float128
  abs(__float128 __x)
  { return __x < 0 ? -__x : __x; }
#endif

Note the __CUDACC__ bit. It's not present in the pristine copy of libstdc++.

AFAICT, it's been added by debian: https://tracker.debian.org/media/packages/g/gcc-11/changelog-11.2.0-17

gcc-10 (10.2.0-6) unstable; urgency=medium
...
  * Update patch for CUDA and __float128. Closes: #968672.

I'm still not sure why the behavior had changed on CE.

The bad news is that I've found no way to disable fp128 support in clang via command line options. It appears to be hardcoded based on the target triple. Looks like AMDGPU backend had already run into the same problem and worked around it in clang: https://github.com/llvm/llvm-project/blob/3a67b912386e70073efcb2225c6354b85048b1ae/clang/lib/Basic/Targets/AMDGPU.cpp#L345

Looks like I'll need to do something similar for NVPTX back-end, too.

[mattgodbolt]

I'm not sure I'm following any of this, unfortunately: I don't have enough context to know what the right thing to do here is. I only know the default C++ library should be whatever Ubuntu gives you by default..

[Artem-B]

The short version is that it will need to be fixed in clang and there's no way (AFAICT) to avoid the problem by disabling __float128 or by using an older libstdc++ (afaict the issue goes back to before gcc-10).

Temporarily passing -stdlib=libc++ may be a reasonable trade-off for the time being.

I still don't understand what kept clang working until recently. If we can figure it out, it may provide another workaround.

[jlebar]

I would also be happy taking this patch while we wait for Art to fix it properly.

…

On Mon, Aug 21, 2023 at 11:30 PM Artem Belevich ***@***.***> wrote: The short version is that it will need to be fixed in clang and there's no way (AFAICT) to avoid the problem by disabling __float128 or by using an older libstdc++ (afaict the issue goes back to before gcc-10). Temporarily passing -stdlib=libc++ may be a reasonable trade-off for the time being. I still don't understand what kept clang working until recently. If we can figure it out, it may provide another workaround. — Reply to this email directly, view it on GitHub <#5373 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABEZBZWVDCCUQJ73IZBITDXWRGYBANCNFSM6AAAAAA3MF65JQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[Artem-B]

@mattgodbolt : was CE at some point using stock ubuntu packages for libstdc++? Apparently Ubuntu had patched libstdc++ headers to deal with this issue http://changelogs.ubuntu.com/changelogs/pool/main/g/gcc-10/gcc-10_10.5.0-1ubuntu1~20.04/changelog :

gcc-8 (8-20171023-1) experimental; urgency=medium

  * GCC 8 snapshot, taken from the trunk 20171023.
  * Mask __float128 from CUDA compilers. LP: #1717257.

That probably explains the "how did it work before" part.

Right now it appears that CE uses DIY builds gcc and libstdc++ and they no longer have that patch.

So, another possible workaround would be to have gcc+libstdc++ installed from a standard ubuntu package and point cuda clang there with --gcc-toolchain=.

[mattgodbolt]

was CE at some point using stock ubuntu packages for libstdc++?

Never knowingly, no. We've always passed --gcc-toolchain for clang compilers. I can't speak for CUDA stuff, as I don't have enough context/bandwidth to monitor this as closely.

[Artem-B]

__float128 should be fixed by llvm/llvm-project@318bff6
It just missed the last clang update on godbolt.org, but should be in effect on the next one.