Even in release builds, there is a segfault now in the EnvReplacer, the stack trace does many fold_expr -> fold_bin_expr -> fold_expr -> .... Happens on this file, which has a string concatenation expression containing 750 +: https://github.com/highlightjs/highlight.js/blob/105a11a13eedbf830c0e80cc052028ceb593837f/src/languages/isbl.js

Usually, this is only a problem (and has happened for a long time) in non-release builds.

But only on Linux and Windows, macOS works fine.

The same asset also works fine in a standalone Rust projects (with the exact code as the JSTransformer, just without the Node process).
So I guess the stack occupied by Node plus the recursion in swc triggers the crash, and with just Rust the stack space is enough. And macOS probably also has more stack

The same asset also works fine in a standalone Rust projects (with the exact code as the JSTransformer, just without the Node process). So I guess the stack occupied by Node plus the recursion in swc triggers the crash, and with just Rust the stack space is enough. And macOS probably also has more stack

Not really an area I have any experience in, but a quick search shows that is possible to increase the stack size with some linker flags: https://www.reddit.com/r/rust/comments/872fc4/how_to_increase_the_stack_size/

Is that worth experimenting with?

EDIT: read something else that mentioned threads on linux specifically have a smaller stack, maybe we need to increase the Rayon thread stack size (that's where this code is running, right?): https://docs.rs/rayon/latest/rayon/struct.ThreadPoolBuilder.html#method.stack_size

I want to try bisecting which swc version introduced this, because not much changed regarding the AST depth

So increasing the stack size would maybe just be a workaround until some future version.

As it turns out, this also occurs in Deno and Turbopack: swc-project/swc#8840

I've rebased onto v2 and bumped swc again (but not much changed ac92e7a) and now Windows CI appears to work (even without your PR), but maybe it's also not deterministic

And indeed, now it's failing again

Ugh yeah. It's definitely non deterministic. The maybe_grow I found worked on my machine apparently doesn't cut it for CI. I suspect the data in the stack is larger (maybe longer paths or something).

I think the CI failure is happening on something other than the EnvReplacer because it's happening on yarn build. So I'm trying to figure out why we're not getting back traces. Any ideas on why the rayon threads (even with RUST_BACKTRACE=1) don't report anything? I think this is the relevant code for that. Perhaps it has something to do with the napi env and the way we're threading.

Does yarn build work for you locally?
PARCEL_WORKERS=0 might be worth a try

I think the CI failure is happening on something other than the EnvReplacer because it's happening on yarn build

But EnvReplacer also runs in that case? I don't quite understand what you mean

why we're not getting back traces

I think Rust simply doesn't emit "nice" panics for stack traces. You get get a segfault/invalid instruction.

You'd need to compile with debugging information to get nice stack traces (debug=true in the Cargo profile).

Though that might affect the outcome, we had an internal rayon panic we saw quite a bit internally go away when we started building the canary release with debuginfo (even though it gets extracted to a separate file and stripped from the binary).

Weirdly I couldn't reproduce even with PARCEL_WORKERS=0 on my machine. After a bunch of fiddling it looks like CI passes with a "red zone" of 32KiB in the main fold in the EnvReplacer. Hopefully that number holds 🙏

And yeah it looks like you're right about stack overflows not reporting a back trace. I should have known better than to trust gpt lol.

No, Github doesn't see it as a approving review...