Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build fails on systems where libatomic.so is provided by compiler-rt #39546

Closed
dylanaraps opened this issue Jul 27, 2021 · 16 comments
Closed

build fails on systems where libatomic.so is provided by compiler-rt #39546

dylanaraps opened this issue Jul 27, 2021 · 16 comments
Labels
build Issues and PRs related to build files or the CI.

Comments

@dylanaraps
Copy link

Version

v16.5.0

Platform

Linux desert 5.12.6 #14 SMP Thu Jul 8 23:57:15 EEST 2021 x86_64 GNU/Linux

Subsystem

No response

What steps will reproduce the bug?

  1. Attempt to build nodejs in an environment without libatomic (where libatomic is provided by compiler-rt).
  2. Build fails due to missing libatomic.

In this environment, no linker flags are needed as compiler-rt provides libatomic implicitly.

The following workaround is used to get builds working:

# When libatomic does not exist, remove the linker argument which tries to use it.
[ -r /usr/lib/libatomic.so ] || {
    sed 's/-latomic//' node.gyp > _ 
    mv -f _ node.gyp
}

How often does it reproduce? Is there a required condition?

This is always reproducible in an environment without libatomic (where libatomic is provided by compiler-rt).

What is the expected behavior?

The build succeeds and libatomic is not linked when unneeded.

What do you see instead?

The build attempts to link to libatomic and fails due to the library not existing in the filesystem. The library is provided by compiler-rt in this environment.

Additional information

No response

@gireeshpunathil
Copy link
Member

@dylanaraps - is there a better way to qualify systems without libatomic.so ? Is it

  • a specific type of OS
  • a specific OS configuration
  • a specific user configuration

Also if you are planning to contribute, here is an example of how to do this in node.gyp in a more natural way: #30099

@dylanaraps
Copy link
Author

It's hard to say. This is rather niche. It's a Linux distribution using LLVM/Clang (and no GNU components; gcc, binutils, etc).
LLVM provides libatomic via compiler-rt implicitly. (https://github.com/wyvertux/wyverkiss)

The environment can be detected like this:

  • If toolchain is LLVM/Clang and compiler-rt is available and libatomic is not available

@targos
Copy link
Member

targos commented Jul 28, 2021

With which options do you run the configure script?

@dylanaraps
Copy link
Author

  ./configure \
      --shared-zlib \
      --shared-openssl \
      --with-intl=none \
      --without-etw \
      --without-dtrace \
      --without-report \
      --without-node-snapshot \
      --without-node-code-cache \ 
      --ninja

@dylanaraps
Copy link
Author

dylanaraps commented Jul 28, 2021

Basically, the build system assumes that libatomic is always available on Linux. Problem is that LLVM + compiler-rt can be used to create an environment contrary to this assumption (where libatomic is implicitly provided by compiler-rt). Does this explain things better?

Expected behavior would be some kind of detection of this environment (and for the linker flag -latomic to not be passed).

@konimex
Copy link

konimex commented Jul 28, 2021

One might ask, how do we detect if $CC is using libatomic or compiler-rt?

There's no good way (aside of $CC -print-file-name=libatomic.so, but this only checks if the file exists) but the next best thing would be detecting libgcc_s and compiler-rt, since compiler-rt replaces both libatomic and libgcc_s in an LLVM-only env. (And I don't think GCC can be built without libatomic since I think it's essential, but it's outside my knowledge realm at this point). So we can detect it by using $CC -print-libgcc-file-name. GCC (and Clang in a GCC env) will detect /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/11.1.0/libgcc.a, but Clang in an LLVM-only env will detect /usr/lib/clang/12.0.1/lib/linux/libclang_rt.builtins-x86_64.a

@targos
Copy link
Member

targos commented Jul 28, 2021

Is there prior art of such detection in other projects? Maybe cpython?

@targos targos added the build Issues and PRs related to build files or the CI. label Aug 9, 2021
@targos
Copy link
Member

targos commented Aug 9, 2021

@nodejs/build-files

@williamh
Copy link

Hi, I am the packager of nodejs for Gentoo Linux.
This is an issue for us reported here.
As you can see from the bug report, we have a hack in place for nodejs 18 to get around it, but this is a patch I would like to get rid of at some point.
What can I do to assist with fixing this in the nodejs build system?

Thanks,

William

@bnoordhuis
Copy link
Member

Untested, but as libatomic is only needed on some architectures, the diff below probably sidesteps the issue for x86_64:

diff --git a/node.gyp b/node.gyp
index 8fe89d8b1bc..12204cb2f60 100644
--- a/node.gyp
+++ b/node.gyp
@@ -115,7 +115,9 @@
           '-Wl,-bnoerrmsg',
         ],
       }],
-      ['OS == "linux" and llvm_version != "0.0"', {
+      ['OS == "linux" and '
+       'llvm_version != "0.0" and '
+       'target_arch in "arm loong64 mips mips64 mips64el mipsel ppc riscv64"', {
         'libraries': ['-latomic'],
       }],
     ],

LMK if that works for you and I'll turn it into a pull request.

nvinson added a commit to nvinson/magpie that referenced this issue Sep 15, 2022
@nvinson
Copy link

nvinson commented Sep 15, 2022

LMK if that works for you and I'll turn it into a pull request.

I just tried it and it still failed with

ld.lld: error: undefined symbol: __atomic_is_lock_free

Github has already linked to the ebuild & support files I used to test with, but I am posting the output of emerge --info below. A lot of the information is Gentoo specific, but it should still have diagnostic value here.

Portage 3.0.36 (python 3.10.7-final-0, default/linux/amd64/17.1/desktop, gcc-12.2.0, glibc-2.35-r8, 5.18.14-gentoo x86_64)
=================================================================
System uname: Linux-5.18.14-gentoo-x86_64-Intel-R-_Core-TM-_i7-4771_CPU_@_3.50GHz-with-glibc2.35
KiB Mem:    32557860 total,  10730172 free
KiB Swap:    2097148 total,   2015500 free
Timestamp of repository gentoo: Thu, 15 Sep 2022 21:31:46 +0000
Head commit of repository gentoo: 9eab52df292dba4d871398c81edab3437ffa2576

Timestamp of repository brother-overlay: Mon, 12 Sep 2022 19:25:29 +0000
Head commit of repository brother-overlay: d9faa8af87d81688313b3e308849f82642b7c437

Head commit of repository magpie: 0cc95ac94b4f616ee68011c724a1cdaa284c4748

sh bash 5.1_p16-r2
ld GNU ld (Gentoo 2.39 p4) 2.39.0
app-misc/pax-utils:        1.3.5::gentoo
app-shells/bash:           5.1_p16-r2::gentoo
dev-java/java-config:      2.3.1::gentoo
dev-lang/perl:             5.36.0::gentoo
dev-lang/python:           3.10.7::gentoo, 3.11.0_rc2::gentoo
dev-lang/rust:             1.62.1::gentoo
dev-util/cmake:            3.24.1::gentoo
dev-util/meson:            0.63.2-r1::gentoo
sys-apps/baselayout:       2.8-r2::gentoo
sys-apps/openrc:           0.45.2::gentoo
sys-apps/sandbox:          2.29::gentoo
sys-devel/autoconf:        2.13-r2::gentoo, 2.69-r6::gentoo, 2.71-r2::gentoo
sys-devel/automake:        1.16.5::gentoo
sys-devel/binutils:        2.39-r2::gentoo
sys-devel/binutils-config: 5.4.1::gentoo
sys-devel/clang:           14.0.6-r1::gentoo
sys-devel/gcc:             12.2.0::gentoo
sys-devel/gcc-config:      2.5-r1::gentoo
sys-devel/libtool:         2.4.7::gentoo
sys-devel/lld:             14.0.6::gentoo
sys-devel/llvm:            14.0.6-r2::gentoo
sys-devel/make:            4.3::gentoo
sys-kernel/linux-headers:  5.19::gentoo (virtual/os-headers)
sys-libs/glibc:            2.35-r8::gentoo
Repositories:

gentoo
    location: /var/db/repos/gentoo
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/gentoo.git
    priority: -1000
    sync-git-verify-commit-signature: true

brother-overlay
    location: /var/db/repos/brother-overlay
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/brother-overlay.git
    masters: gentoo

magpie
    location: /var/db/repos/magpie
    sync-type: git
    sync-uri: https://github.com/nvinson/magpie.git
    masters: gentoo

ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="* -@EULA"
AR="llvm-ar"
CBUILD="x86_64-pc-linux-gnu"
CC="clang"
CFLAGS="-march=native -O2 -pipe -D_FORTIFY_SOURCE=3 -fstack-protector-strong -flto=thin"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /opt/brother/scanner/brscan4/brsanenetdevice4.cfg /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXX="clang++"
CXXFLAGS="-march=native -O2 -pipe -D_FORTIFY_SOURCE=3 -fstack-protector-strong -flto=thin"
DISTDIR="/var/cache/portage/distfiles"
EMERGE_DEFAULT_OPTS="--quiet-build"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
FCFLAGS="-march=native -O2 -pipe"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-march=native -O2 -pipe"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="en_US.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -fuse-ld=lld -rtlib=compiler-rt -unwindlib=libunwind"
MAKEOPTS="-j8"
NM="llvm-nm"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
RANLIB="llvm-ranlib"
SHELL="/bin/zsh"
USE="X a52 aac acl acpi alsa amd64 branding bzip2 cairo cdda cdr clang cleartype cli corefonts crypt cups dbus dri dts dvd dvdr elogind encode exif flac fortran gdbm gif glamor gpm gtk gui iconv icu ipv6 jpeg lcms libglvnd libnotify libtirpc mad mng mp3 mp4 mpeg multilib ncurses nls nptl ogg opengl openmp pam pango pcre pdf png policykit ppds qt5 readline sdl seccomp spell split-usr ssl startup-notification svg theora tiff truetype udev udisks unicode upower usb vaapi vorbis vpx wxwidgets x264 xattr xcb xml xv xvid zlib" ABI_X86="64" ADA_TARGET="gnat_2020" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 fma3 mmx mmxext popcnt sse sse2 sse3 sse4 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput" KERNEL="linux" L10N="en-US en" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-4 php8-0" POSTGRES_TARGETS="postgres12 postgres13" PYTHON_SINGLE_TARGET="python3_10" PYTHON_TARGETS="python3_10" RUBY_TARGETS="ruby27" USERLAND="GNU" VIDEO_CARDS="intel i965" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  ADDR2LINE, ARFLAGS, AS, ASFLAGS, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LEX, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS

@bnoordhuis
Copy link
Member

ld.lld: error: undefined symbol: __atomic_is_lock_free

https://reviews.llvm.org/D85044?id=287068 - looks like compiler-rt does not implement it at the time of writing. Guess that means it's not a drop-in replacement for libatomic yet.

@svmhdvn
Copy link

svmhdvn commented Sep 30, 2023

@bnoordhuis @williamh As of this gentoo bug report comment[0], it seems that the problem is resolved and __atomic_is_lock_free should now be in compiler-rt in this LLVM upstream commit[1]. I've submitted a PR in gentoo here[2] to fix the nodejs ebuild and tested it with a local build in my musl+clang profile.

What's the status of this issue now in nodejs?

[0] https://bugs.gentoo.org/869992#c13
[1] llvm/llvm-project@00530de
[2] gentoo/gentoo#33141

@bnoordhuis
Copy link
Member

I think this can be closed on our end. Thanks for the update.

@marv
Copy link

marv commented Nov 3, 2023

@bnoordhuis I'm afraid the problem is still not fixed and building with the LLVM toolchain using compiler-rt still requires downstream patching

It had been using the patch from #39546 (comment) so far to build node on a x86_64 system using the full LLVM toolchain and musl C library (no -latomic needed / available). Recently I had to revisit the situation because I've been trying to build node on a riscv64gc-unknown-linux-musl system using the LLVM toolchain. Here the mentioned patch doesn't do the right thing as it - like mentioned in the comment - only sidesteps the problem for x86_64:

Untested, but as libatomic is only needed on some architectures, the diff below probably sidesteps the issue for x86_64

Whether libatomic is needed or not does not (only) depend on the architecture but the toolchain that is used (in my case I use the full/pure LLVM toolchain on x86_64 and riscv64 and don't want libatomic to be linked since it's GCC specific)

While looking at the situation I couldn't explain the logic regarding libatomic in node.gyp:

'OS == "linux" and llvm_version != "0.0"'

llvm_version != "0.0" is true whenever the LLVM toolchain is used (in my case llvm_version is 17.0), but since -latomic is GCC specific I expected it to be the other way around: llvm_version == "0.0" (read: if toolchain is NOT LLVM), this was already reported in #30093. Is this a remnant from a time before LLVM 12 when compiler-rt didn't implement __atomic_is_lock_free? (#28532 (comment)) Or because the llvm_version detection didn't work correctly between open-source Clang and Xcode clang? This was mentioned in #30093 (comment):

This will only work with open-source versions of clang, and versions of Apple's Xcode clang prior to Xcode 7. As of Xcode 7, Apple no longer advertises its compiler as being "based on" a particular open source llvm version; Apple's llvm/clang has diverged too much from open source llvm/clang for any such association to be meaningful.

Changing the condition to llvm_version == "0.0" allows me to build node on x86_64-unknown-linux-musl using the LLVM toolchain

On riscv64gc-unknown-linux-musl another fix is necessary because the following assumption in https://github.com/nodejs/node/blob/main/tools/v8_gypfiles/v8.gyp#L1075C1-L1081C12 is only true for GCC (I think):

        # Platforms that don't have Compare-And-Swap (CAS) support need to link atomic library
        # to implement atomic memory access
        ['v8_current_cpu in ["mips64", "mips64el", "ppc", "arm", "riscv64", "loong64"]', {
          'link_settings': {
            'libraries': ['-latomic', ],
          },
        }],

Using the full LLVM toolchain we don't want/need/can link against libatomic here either. I worked around this by removing "riscv64" from so list so that the condition isn't fulfilled. Would it be a correct fix to wrap this in a llvm_version == "0.0" if statement?

@bnoordhuis
Copy link
Member

Neither riscv or musl are officially supported targets (i.e. they're self-serve) so you're welcome to send a pull request as long as it doesn't cause regressions on targets that are supported.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Issues and PRs related to build files or the CI.
Projects
None yet
Development

No branches or pull requests

9 participants