Bad File Descriptor when trying to connect TCP listener

[maxwellflitton]

I have the following code:

#[tokio::main(flavor="current_thread")]
async fn main() {
    use std::os::wasi::io::FromRawFd;
    let std_listener = unsafe { StDTcpListener::from_raw_fd(3) };
    std_listener.set_nonblocking(true).unwrap();
    println!("Server listening on port 8080");
}

It compiles to the target wasm32-wasi fine, but when I try and run it with the following command:

export FD_COUNT="3"
sudo wasmtime run ./wasi-server.wasm -W tcp-listen=127.0.0.1:8080

and I get the following error:

called `Result::unwrap()` on an `Err` value: Os { code: 8, kind: Uncategorized, message: "Bad file descriptor" }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Error: failed to run main module `./wasi-server.wasm`

Caused by:
    0: failed to invoke command default
    1: error while executing at wasm backtrace:
           0: 0x12410 - <unknown>!__rust_start_panic
           1: 0x12132 - <unknown>!rust_panic
           2: 0x12065 - <unknown>!std::panicking::rust_panic_with_hook::h39e3a8d718bdeee9
           3: 0x11440 - <unknown>!std::panicking::begin_panic_handler::{{closure}}::he926e6c0f7818bbb
           4: 0x1136b - <unknown>!std::sys_common::backtrace::__rust_end_short_backtrace::h8c8c9b1b5bff6dcb
           5: 0x11a3c - <unknown>!rust_begin_unwind
           6: 0x16f87 - <unknown>!core::panicking::panic_fmt::h113f422b14ac6c5e
           7: 0x18a23 - <unknown>!core::result::unwrap_failed::h39cc7ddeefacf4e6
           8: 0x2602 - <unknown>!<core::pin::Pin<P> as core::future::future::Future>::poll::hafd7cce1e3bb831b
           9:  0xab4 - <unknown>!tokio::runtime::scheduler::current_thread::Context::enter::h6571e16d8eddbc1b
          10: 0x2476 - <unknown>!tokio::runtime::context::scoped::Scoped<T>::set::hd5c0c2879f3c3696
          11:  0xc9a - <unknown>!tokio::runtime::scheduler::current_thread::CoreGuard::block_on::h530f9f4b35b42418
          12: 0x215e - <unknown>!tokio::runtime::context::runtime::enter_runtime::h8de416387b833c99
          13:  0xf9b - <unknown>!wasi_server::main::h9a3299a258d097bb
          14: 0x234b - <unknown>!std::sys_common::backtrace::__rust_begin_short_backtrace::h08ca4841d5529c37
          15: 0x235a - <unknown>!std::rt::lang_start::{{closure}}::h64de60455dc43b4c
          16: 0xea12 - <unknown>!std::rt::lang_start_internal::h96857314850acb81
          17: 0x1224 - <unknown>!__main_void
          18:  0x69d - <unknown>!_start
       note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information
    2: wasm trap: wasm `unreachable` instruction executed

[FrankReh]

What version of wasmtime are you using? I only know more recent versions where the "tcplisten" parameter is paired with "-S" and has to be given immediately after the "run" argument, like so:

wasmtime \
  run \
  -S tcplisten=127.0.0.1:8080 \
  ./wasi-server.wasm

[maxwellflitton]

@FrankReh thanks for the response. I am using the following version:

wasmtime-cli 20.0.0 (9e1084ffa 2024-04-22)

I've now switched to the following command:

wasmtime run -S tcplisten=127.0.0.1:8080 ./wasi-server.wasm

And I get the following error:
Error: components do not support --tcplisten

[FrankReh]

Yes, that's as far as I get now too and that is puzzling. Now I'm looking into the commit that introduced that code to understand things better. Coincidentally, I had asked why this was happening a day or two ago myself with no response yet.

[FrankReh]

Okay. What you and I were missing is another -S option. This is working for me now.

wasmtime run -S preview2=n -S tcplisten=127.0.0.1:8080 ./wasi-server.wasm

The transition from the Rust Wasm world only being modules to now having both Wasm modules and Wasm components has created some confusion - at least for me. I incorrectly assumed if wasmtime were passed a Wasm module, it would use the file's type embedded in the header to run the module setup but that was wrong. There is a flag that can be set on the command line to make the choice explicit.

With that new version of the command, your example works and prints out the line you were expecting. Note that the actual set your main was doing isn't needed. The TCP listener is automatically set to non blocking by the wasmtime CLI code like this.

    pub fn compute_preopen_sockets(&self) -> Result<Vec<TcpListener>> {
        let mut listeners = vec![];

        for address in &self.common.wasi.tcplisten {
            let stdlistener = std::net::TcpListener::bind(address)
                .with_context(|| format!("failed to bind to address '{}'", address))?;

            let _ = stdlistener.set_nonblocking(true)?;

            listeners.push(stdlistener)
        }
        Ok(listeners)
    }

[pchickey]

The transition from the Rust Wasm world only being modules to now having both Wasm modules and Wasm components has created some confusion - at least for me. I incorrectly assumed if wasmtime were passed a Wasm module, it would use the file's type embedded in the header to run the module setup but that was wrong. There is a flag that can be set on the command line to make the choice explicit.

Sorry this has been confusing, I think that our error messages and the choice of -S preview2=n for this option made it extra confusing. There are two host implementations of WASI P1 in the wasmtime tree, and in the wasmtime-cli executable.

• the concept of a tcp socket being available as a preopened fd is from a totally undocumented and untested extension of the WASI P1 implementation that was contributed by a team that dissolved when their startup ran out of money. I asked around for the better part of a year if anyone was depending on this functionality and could contribute docs or tests, and nobody spoke up, so I think you may be the first to use it in a while. We don't recommend anyone actually use that code, because it is both undocumented and untested, and the design of sockets themselves diverged from that in WASI P2.
• There is a legacy WASI implementation in the wasi-common crate that supports WASI P1 only. It has support for the tcp listener preopen. For wasmtime-cli to use the legacy implementation, you must pass the -S preview2=n flag. It would have been a much better choice if we had made this flag -S implementation=legacy, and I'll discuss with the team whether we can change it!
• The new, default implementation, which lives in the wasmtime-wasi crate supports WASI P1 when used with modules, and WASI P2 when used with components. We recommend everyone use this implementation, however, it does not support the tcp socket preopen when used with WASI P1 modules, because we consider that feature of the legacy implementation a historical wart that never got adoption. Instead, in P2 we use the interfaces defined in the wasi-sockets proposal, which provides much more functionality, has a great set of docs and tests, and many folks actively developing and maintaining it. Wasi-sockets doesn't piggyback on the descriptor type like P1 sockets did. For programs that need it, wasi-libc provides a posix API to the underlying wasi-sockets interface, thanks to @dicej in implement basic TCP/UDP server support WebAssembly/wasi-libc#481 add wasip2 implementations of more socket APIs WebAssembly/wasi-libc#482 implement getsockname, getpeername, and getaddrinfo WebAssembly/wasi-libc#488 .

[FrankReh]

@pchickey Wow! That's a lot of interesting background. Thanks for that.

There was probably a busy loop making the mio demo work anyway.

[alexcrichton]

I don't have much more to add over what @pchickey already mentioned, but otherwise can confirm that this is working as expected. Some things I can say are:

From the OP @maxwellflitton you're using:

wasmtime run ./wasi-server.wasm -W tcp-listen=127.0.0.1:8080

That's actually passing the -W argument to wasi-server.wasm as a CLI argument, not to Wasmtime itself. I believe you've already discovered this as the next attempt was:

wasmtime run -S tcplisten=127.0.0.1:8080 ./wasi-server.wasm

where the error Error: components do not support --tcplisten popped out. That's a bad error message since the flag has been renamed (it used to be --tcplisten), but the "correct fix" for this is what @FrankReh pointed out with -Spreview2=n which, as @pchickey pointed out, is a terrible name for a flag (and also difficult to find)

To me the action items here are:

• Upgrade mio's support of sockets to work with wasi-sockets on the wasip2 target. If acceptable remove the support for wasip1 since sockets were never fully finished/specified at the spec level.
• Rename -Spreview2=n to -S implementation=...
• Improve the error message for -S tcplisten=... when using wasmtime-wasi's implementation of WASI