Tests integration-tests::integration fail with error for setup of scripts/my-script.sh

[lcheylus]

Build for cargo-nextest current version on OpenBSD-current/amd64, using my patch for libc crate (add missing waitid function, see rust-lang/libc#3490) => build debug OK

When I try to run auto-tests for nextest, I have 3 tests KO for integration-tests::integration:

$ cargo local-nt run
(...)
Finished test [unoptimized + debuginfo] target(s) in 1m 47s
    Starting 215 tests across 13 binaries
        PASS [   0.052s] cargo-nextest dispatch::tests::test_test_binary_argument_parsing
        PASS [   0.345s] cargo-nextest dispatch::tests::test_argument_parsing
        PASS [  13.281s] integration-tests::integration test_list_binaries_only
        PASS [  15.179s] integration-tests::integration test_list_default
        PASS [  15.076s] integration-tests::integration test_list_full
        PASS [  14.711s] integration-tests::integration test_list_full_after_build
        PASS [  11.591s] integration-tests::integration test_list_host_after_build
        PASS [  16.763s] integration-tests::integration test_list_target_after_build
        FAIL [  12.685s] integration-tests::integration test_relocated_run

--- STDOUT:              integration-tests::integration test_relocated_run ---

running 1 test
test test_relocated_run ... FAILED

failures:

failures:
    test_relocated_run

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 14 filtered out; finished in 12.66s
--- STDERR:              integration-tests::integration test_relocated_run ---
thread 'test_relocated_run' panicked at integration-tests/tests/integration/main.rs:332:5:
assertion `left == right` failed: correct exit code for command
command: "/home/fox/dev/cargo-nextest.git/target/debug/cargo-nextest-dup" "nextest" "--manifest-path" "/tmp/nextest-fixtureTMysng/src/Cargo.toml" "run" "--binaries-metadata" "/tmp/nextest-fixtureTMysng/src/test-subdir/binaries_metadata.json" "--cargo-metadata" "/tmp/nextest
-fixtureTMysng/src/test-subdir/cargo_metadata.json" "--workspace-remap" "../../../../../tmp/nextest-fixtureTMysng/src" "--target-dir-remap" "../../../../../tmp/nextest-fixtureTMysng/src/test-subdir"
exit code: Some(105)
--- stdout ---


--- stderr ---
info: experimental features enabled: setup-scripts
    Starting 27 tests across 14 binaries (5 skipped)
       SETUP [      1/1] my-script-unix: ./scripts/my-script.sh
error: [double-spawn] failed to exec `"./scripts/my-script.sh"`

Caused by:
  No such file or directory (os error 2)
  SETUP FAIL [      1/1] my-script-unix: ./scripts/my-script.sh

   Canceling due to setup script failure
------------
     Summary [   0.077s] 0/27 tests run: 0 passed, 0 skipped
error: setup script failed



  left: Some(105)
 right: Some(100)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

   Canceling due to test failure: 1 test still running
(...)
     Summary [  66.409s] 11/215 tests run: 8 passed, 3 failed, 0 skipped
        FAIL [  12.685s] integration-tests::integration test_relocated_run
        FAIL [  14.149s] integration-tests::integration test_run
        FAIL [  13.702s] integration-tests::integration test_run_after_build
error: test run failed

For the 3 failed tests, same error with setup fail of script ./scripts/my-script.sh => error code 105 / SETUP_SCRIPT_FAILED (according to nextest-metadata/src/exit_codes.rs).

After some debug, this script my-script.sh comes from fixtures/nextest-tests/scripts/my-script.sh.

I don't find the root cause explaining why the setup of this script failed.

Have you any clue to help me with these errors and to debug it ?

[sunshowers]

My best guess is that your OpenBSD install doesn't have /bin/bash in it. That script is pretty simple so in 4a0de0b I've made it depend only on /bin/sh. Let me know if this doesn't work.

[lcheylus]

OK, after rebuild with the current version, all tests are OK on OpenBSD-current/amd64 :)

$ cargo local-nt run
(...)
   Compiling cargo-nextest v0.9.66 (/home/fox/dev/cargo-nextest.git/cargo-nextest)
   Compiling nextest-metadata v0.10.0 (/home/fox/dev/cargo-nextest.git/nextest-metadata)
   Compiling nextest-workspace-hack v0.1.0 (/home/fox/dev/cargo-nextest.git/workspace-hack)
    Finished test [unoptimized + debuginfo] target(s) in 2m 05s
    Starting 215 tests across 13 binaries
(...)
        PASS [   0.050s] quick-junit::fixture_tests fixtures
        PASS [   3.943s] nextest-runner::integration basic::test_termination
------------
     Summary [ 137.724s] 215 tests run: 215 passed, 0 skipped

For info, in script shells, it's better to use #!/usr/bin/env bash instead of #!/bin/bash => Bash is not always installed as /bin/bash. Here it was the root cause of this issue : on OpenBSD, bash is installed in /usr/local/bin (after install of the package bash).

Thanks for this fix.

[sunshowers]

Got it, thanks for the info! I'd never come across a Unix where bash hadn't been installed to /bin/bash so this was a new one, haha