Can't build clap@4.5.4 under MODULE.bazel

[Zemnmez]

Hi there, thanks for all your hard work on this. I've been trying to move to using MODULE.bazel for everything I can and I've made a best-effort at moving rules_rust. You can find a minimal reproducing commit here: zemn-me/monorepo@5fb22bf

When I try to build my one tag which has a clap dependency, I get:

thomas@DESKTOP-B82ERE8 ~/d/monorepo> bazel test //rs/...                                                                                                                                                                                                                 130
WARNING: Option 'experimental_remote_build_event_upload' is deprecated: Use --remote_build_event_upload instead
INFO: Invocation ID: 94ac485c-4357-43c7-b5eb-c170ec1bb011
INFO: Streaming build results to: https://app.buildbuddy.io/invocation/94ac485c-4357-43c7-b5eb-c170ec1bb011
WARNING: Option 'experimental_remote_build_event_upload' is deprecated: Use --remote_build_event_upload instead
ERROR: /home/thomas/.cache/bazel/_bazel_thomas/a41610ee02ad9c69b13020a7f1cf3deb/external/rules_rust~~i~rrra__clap-4.3.11/BUILD.bazel:13:13: Compiling Rust rlib clap v4.3.11 (75 files) failed: (Exit 1): process_wrapper failed: error executing Rustc command (from target @@rules_rust~~i~rrra__clap-4.3.11//:clap) bazel-out/k8-opt-exec-ST-13d3ddad9198/bin/external/rules_rust~/util/process_wrapper/process_wrapper --arg-file ... (remaining 65 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
error[E0463]: can't find crate for `clap_derive`
   --> external/rules_rust~~i~rrra__clap-4.3.11/src/lib.rs:102:9
    |
102 | pub use clap_derive::{self, *};
    |         ^^^^^^^^^^^ can't find crate

error: aborting due to 1 previous error

For more information about this error, try `rustc --explain E0463`.
INFO: Elapsed time: 2.356s, Critical Path: 0.10s
INFO: 3 processes: 3 internal.
ERROR: Build did NOT complete successfully
//rs/hello_world:hello_world_fmt                                (cached) PASSED in 0.1s
//rs/ts:ts_fmt                                                  (cached) PASSED in 0.2s
//rs/ts/testing:testing_fmt                                     (cached) PASSED in 0.0s

Executed 0 out of 3 tests: 3 tests pass.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
All tests passed but there were other errors during the build.

INFO: Streaming build results to: https://app.buildbuddy.io/invocation/94ac485c-4357-43c7-b5eb-c170ec1bb011
FAILED: 
    Fetching module extension crate in @@rules_rust~//crate_universe:extension.bzl; starting

As you can see from below, I'd assumed this came from not having proc macro deps set correctly and tried to fix via crate.annotation and proc_macro_deps, but no dice.

From 5fb22bf3987ae0df5ec813aca9949267cfa173b6 Mon Sep 17 00:00:00 2001
From: Thomas Neil James Shadwell <thomas@shadwell.im>
Date: Mon, 1 Apr 2024 01:43:39 +0000
Subject: [PATCH] Move rules_rust to module.bazel

---
 Cargo.Bazel.lock => Cargo.lock |   250 +-
 Cargo.toml                     |     5 +-
 MODULE.bazel                   |    26 +
 MODULE.bazel.lock              | 12197 ++++++++++++++++++++++++++++++-
 WORKSPACE                      |    26 -
 bzl/deps.bzl                   |     6 -
 cargo-bazel-lock.json          |  5177 -------------
 rs/cmd/sha256/main.rs          |     1 +
 8 files changed, 11968 insertions(+), 5720 deletions(-)
 rename Cargo.Bazel.lock => Cargo.lock (73%)
 delete mode 100644 cargo-bazel-lock.json

diff --git a/Cargo.toml b/Cargo.toml
index 98c26f8acc..170ff6a5a7 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -1,3 +1,5 @@
+# to resync with lockfile:
+# CARGO_BAZEL_REPIN=1 bazel sync --only=cargo
 [package]
 name = "monorepo"
 version = "0.1.0"
@@ -7,8 +9,9 @@ edition = "2021"
 
 [dependencies]
 clap = { version = "=4.5.4", features = ["derive"] }
+clap_derive = "=4.5.4"
 swc_common = "=0.33.21"
 sha2 = "=0.10.8"
 hex = "=0.4.3"
 serde_json = "=1.0.115"
-swc_atoms = "=0.6.5"
\ No newline at end of file
+swc_atoms = "=0.6.5"
diff --git a/MODULE.bazel b/MODULE.bazel
index a5aa1252fe..ae4eb8a509 100644
--- a/MODULE.bazel
+++ b/MODULE.bazel
@@ -70,3 +70,29 @@ npm.npm_translate_lock(
     verify_node_modules_ignored = "//:.bazelignore",
 )
 use_repo(npm, "npm")
+
+bazel_dep(name = "rules_rust", version = "0.40.0")
+
+rust = use_extension("@rules_rust//rust:extensions.bzl", "rust")
+rust.toolchain(
+    edition = "2021",
+    versions = ["1.77.0"],
+)
+use_repo(rust, "rust_toolchains")
+
+register_toolchains("@rust_toolchains//:all")
+
+crate = use_extension("@rules_rust//crate_universe:extension.bzl", "crate")
+crate.from_cargo(
+    name = "cargo",
+    cargo_lockfile = "//:Cargo.lock",
+    manifests = ["//:Cargo.toml"],
+)
+use_repo(crate, "cargo")
+
+crate.annotation(
+    crate = "clap",
+    proc_macro_deps = [
+        "@cargo//:clap_derive",
+    ],
+)
diff --git a/rs/cmd/sha256/main.rs b/rs/cmd/sha256/main.rs
index 864998ca76..d9577a4444 100644
--- a/rs/cmd/sha256/main.rs
+++ b/rs/cmd/sha256/main.rs
@@ -2,6 +2,7 @@ use sha2::{Digest, Sha256};
 use std::{convert, env::args, fs::File, io};
 
 #[derive(Debug)]
+#[allow(dead_code)] // not sure why I have to do this
 enum RunError {
     Io(io::Error),
 }

[Zemnmez]

I'm seeing this issue on a different ref, so this may be my own issue 🤔

[aaronmondal]

I believe I'm hitting the same issue in

• Achieve perfect reproducibility for dev setups TraceMachina/nativelink#874

So far I've found that this appears to be caused by proc-macros using the wrong glibc version, or at least they use a different glibc detection mechanism than C++ targets. (Tried --@rules_rust//rust/settings:experimental_use_cc_common_link, same behavior).

When running the PR I linked with nix develop and then bazel test ... --keep_going some of the errors look like this:

error[E0428]: the name `main` is defined multiple times
 --> external/rules_rust~0.42.1~crate~crates__axum-0.6.20/build.rs:7:1
  |
2 | fn main() {
  | --------- previous definition of the value `main` here
...
7 | fn main() {}
  | ^^^^^^^^^ `main` redefined here
  |
  = note: `main` must be defined only once in the value namespace of this module

error[E0463]: can't find crate for `rustversion`
 --> external/rules_rust~0.42.1~crate~crates__axum-0.6.20/build.rs:6:3
  |
6 | #[rustversion::not(nightly)]
  |   ^^^^^^^^^^^ can't find crate

error[E0463]: can't find crate for `rustversion`
 --> external/rules_rust~0.42.1~crate~crates__axum-0.6.20/build.rs:1:3
  |
1 | #[rustversion::nightly]
  |   ^^^^^^^^^^^ can't find crate

error: aborting due to 3 previous errors


error: /root/.cache/bazel/_bazel_root/fde423357588c647741e6859a17e4f8d/execroot/_main/bazel-out/k8-opt-exec-ST-13d3ddad9198/bin/external/rules_rust~override~crate~crates__ser
de_derive-1.0.197/libserde_derive-2616503466.so: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.36' not found (required by /root/.cache/bazel/_bazel_root/fde423357588c6477
41e6859a17e4f8d/execroot/_main/bazel-out/k8-opt-exec-ST-13d3ddad9198/bin/external/rules_rust~override~crate~crates__serde_derive-1.0.197/libserde_derive-2616503466.so)

The build works fine on systems with glibc2.39, but it doesn't work on systems with glibc2.35 (like GitHub's Ubuntu 22.04 runners).

It looks like the host glibc leaks into proc-macros somehow and breaks the build in mysterious ways 😅

@Zemnmez My guess is that the remote exec setup you're using has a similar glibc version mismatch. What does --keep_going say?