Skip to content

Commit

Permalink
fix #714: remove the webassembly _exit(0) hack
Browse files Browse the repository at this point in the history
  • Loading branch information
evanw committed Dec 6, 2022
1 parent 055f9e3 commit c3b0890
Show file tree
Hide file tree
Showing 10 changed files with 13 additions and 100 deletions.
3 changes: 0 additions & 3 deletions .gitignore
Expand Up @@ -7,17 +7,14 @@
/esbuild
/github/
/npm/@esbuild/android-arm/esbuild.wasm
/npm/@esbuild/android-arm/exit0.js
/npm/@esbuild/android-arm/wasm_exec_node.js
/npm/@esbuild/android-arm/wasm_exec.js
/npm/@esbuild/android-x64/esbuild.wasm
/npm/@esbuild/android-x64/exit0.js
/npm/@esbuild/android-x64/wasm_exec_node.js
/npm/@esbuild/android-x64/wasm_exec.js
/npm/esbuild-wasm/browser.js
/npm/esbuild-wasm/esbuild.wasm
/npm/esbuild-wasm/esm/
/npm/esbuild-wasm/exit0.js
/npm/esbuild-wasm/lib/
/npm/esbuild-wasm/wasm_exec_node.js
/npm/esbuild-wasm/wasm_exec.js
Expand Down
10 changes: 10 additions & 0 deletions CHANGELOG.md
Expand Up @@ -99,6 +99,16 @@

The primary branch for this repository was previously called `master` but is now called `main`. This change mirrors a similar change in many other projects.

* Remove esbuild's `_exit(0)` hack for WebAssembly ([#714](https://github.com/evanw/esbuild/issues/714))

Node had an unfortunate bug where the node process is unnecessarily kept open while a WebAssembly module is being optimized: https://github.com/nodejs/node/issues/36616. This means cases where running `esbuild` should take a few milliseconds can end up taking many seconds instead.

The workaround was to force node to exit by ending the process early. This was done by esbuild in one of two ways depending on the exit code. For non-zero exit codes (i.e. when there is a build error), the `esbuild` command could just call `process.kill(process.pid)` to avoid the hang. But for zero exit codes, esbuild had to load a N-API native node extension that calls the operating system's `exit(0)` function.

However, this problem has essentially been fixed in node starting with version 18.3.0. So I have removed this hack from esbuild. If you are using an earlier version of node with `esbuild-wasm` and you don't want the `esbuild` command to hang for a while when exiting, you can upgrade to node 18.3.0 or higher to remove the hang.

The fix came from a V8 upgrade: [this commit](https://github.com/v8/v8/commit/bfe12807c14c91714c7db1485e6b265439375e16) enabled [dynamic tiering for WebAssembly](https://v8.dev/blog/wasm-dynamic-tiering) by default for all projects that use V8's WebAssembly implementation. Previously all functions in the WebAssembly module were optimized in a single batch job but with dynamic tiering, V8 now optimizes individual WebAssembly functions as needed. This avoids unnecessary WebAssembly compilation which allows node to exit on time.

## 0.15.18

* Performance improvements for both JS and CSS
Expand Down
37 changes: 3 additions & 34 deletions Makefile
Expand Up @@ -221,37 +221,6 @@ test-yarnpnp: platform-wasm
version-go:
node scripts/esbuild.js --update-version-go

wasm-napi-exit0-darwin-x64:
node -e 'console.log(`#include <unistd.h>\nvoid* napi_register_module_v1(void* a, void* b) { _exit(0); }`)' \
| clang -x c -dynamiclib -mmacosx-version-min=10.5 -o lib/npm/exit0/darwin-x64-LE.node -
ls -l lib/npm/exit0/darwin-x64-LE.node

wasm-napi-exit0-darwin-arm64:
node -e 'console.log(`#include <unistd.h>\nvoid* napi_register_module_v1(void* a, void* b) { _exit(0); }`)' \
| clang -x c -dynamiclib -mmacosx-version-min=10.5 -o lib/npm/exit0/darwin-arm64-LE.node -
ls -l lib/npm/exit0/darwin-arm64-LE.node

wasm-napi-exit0-linux-x64:
node -e 'console.log(`#include <unistd.h>\nvoid* napi_register_module_v1(void* a, void* b) { _exit(0); }`)' \
| gcc -x c -shared -o lib/npm/exit0/linux-x64-LE.node -
strip lib/npm/exit0/linux-x64-LE.node
ls -l lib/npm/exit0/linux-x64-LE.node

wasm-napi-exit0-linux-arm64:
node -e 'console.log(`#include <unistd.h>\nvoid* napi_register_module_v1(void* a, void* b) { _exit(0); }`)' \
| gcc -x c -shared -o lib/npm/exit0/linux-arm64-LE.node -
strip lib/npm/exit0/linux-arm64-LE.node
ls -l lib/npm/exit0/linux-arm64-LE.node

wasm-napi-exit0-win32-x64:
# This isn't meant to be run directly but is a rough overview of the instructions
echo '__declspec(dllexport) void* napi_register_module_v1(void* a, void* b) { ExitProcess(0); }' > main.c
echo 'setlocal' > main.bat
echo 'call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat" x64' >> main.bat
echo 'cl.exe /LD main.c /link /DLL /NODEFAULTLIB /NOENTRY kernel32.lib /OUT:lib/npm/exit0/win32-x64-LE.node' >> main.bat
main.bat
rm -f main.*

platform-all:
@$(MAKE) --no-print-directory -j4 \
platform-android-arm \
Expand Down Expand Up @@ -576,10 +545,10 @@ clean:
rm -f npm/@esbuild/win32-arm64/esbuild.exe
rm -f npm/@esbuild/win32-ia32/esbuild.exe
rm -f npm/@esbuild/win32-x64/esbuild.exe
rm -f npm/esbuild-wasm/esbuild.wasm npm/esbuild-wasm/wasm_exec*.js npm/esbuild-wasm/exit0.js
rm -rf npm/@esbuild/android-arm/bin npm/@esbuild/android-arm/esbuild.wasm npm/@esbuild/android-arm/wasm_exec*.js npm/@esbuild/android-arm/exit0.js
rm -f npm/esbuild-wasm/esbuild.wasm npm/esbuild-wasm/wasm_exec*.js
rm -rf npm/@esbuild/android-arm/bin npm/@esbuild/android-arm/esbuild.wasm npm/@esbuild/android-arm/wasm_exec*.js
rm -rf npm/@esbuild/android-arm64/bin
rm -rf npm/@esbuild/android-x64/bin npm/@esbuild/android-x64/esbuild.wasm npm/@esbuild/android-x64/wasm_exec*.js npm/@esbuild/android-x64/exit0.js
rm -rf npm/@esbuild/android-x64/bin npm/@esbuild/android-x64/esbuild.wasm npm/@esbuild/android-x64/wasm_exec*.js
rm -rf npm/@esbuild/darwin-arm64/bin
rm -rf npm/@esbuild/darwin-x64/bin
rm -rf npm/@esbuild/freebsd-arm64/bin
Expand Down
Binary file removed lib/npm/exit0/darwin-arm64-LE.node
Binary file not shown.
Binary file removed lib/npm/exit0/darwin-x64-LE.node
Binary file not shown.
Binary file removed lib/npm/exit0/linux-arm64-LE.node
Binary file not shown.
Binary file removed lib/npm/exit0/linux-x64-LE.node
Binary file not shown.
Binary file removed lib/npm/exit0/win32-x64-LE.node
Binary file not shown.
42 changes: 0 additions & 42 deletions npm/esbuild-wasm/bin/esbuild
Expand Up @@ -3,11 +3,8 @@
// Forward to the automatically-generated WebAssembly loader from the Go compiler

const module_ = require('module');
const crypto = require('crypto');
const path = require('path');
const zlib = require('zlib');
const fs = require('fs');
const os = require('os');

const wasm_exec_node = path.join(__dirname, '..', 'wasm_exec_node.js');
const esbuild_wasm = path.join(__dirname, '..', 'esbuild.wasm');
Expand All @@ -23,45 +20,6 @@ function instantiate(bytes, importObject) {
return Promise.resolve({ instance, module });
}

// Node has an unfortunate bug where the node process is unnecessarily kept open while a
// WebAssembly module is being optimized: https://github.com/nodejs/node/issues/36616.
// This means cases where running "esbuild" should take a few milliseconds can end up
// taking many seconds instead. To work around this bug, it is possible to force node to
// exit by calling the operating system's exit function. That's what this code does.
process.on('exit', code => {
// If it's a non-zero exit code, we can just kill our own process to stop. This will
// preserve the fact that there is a non-zero exit code although the exit code will
// be different. We cannot use this if the exit code is supposed to be zero.
if (code !== 0) {
try {
process.kill(process.pid, 'SIGINT');
} catch (e) {
}
return;
}

// Otherwise if the exit code is zero, try to fall back to a binary N-API module that
// calls the operating system's "exit(0)" function.
const nativeModule = `${process.platform}-${os.arch()}-${os.endianness()}.node`;
const base64 = require('../exit0')[nativeModule];
if (base64) {
try {
const data = zlib.inflateRawSync(Buffer.from(base64, 'base64'));
const hash = crypto.createHash('sha256').update(base64).digest().toString('hex').slice(0, 16);
const tempFile = path.join(os.tmpdir(), `${hash}-${nativeModule}`);
try {
if (fs.readFileSync(tempFile).equals(data)) {
require(tempFile);
}
} finally {
fs.writeFileSync(tempFile, data);
require(tempFile);
}
} catch (e) {
}
}
});

// Node has another bug where using "fs.read" to read from stdin reads
// everything successfully and then throws an error, but only on Windows. Go's
// WebAssembly support uses "fs.read" so it hits this problem. This is a patch
Expand Down
21 changes: 0 additions & 21 deletions scripts/esbuild.js
Expand Up @@ -186,26 +186,6 @@ exports.buildWasmLib = async (esbuildPath) => {
fs.writeFileSync(path.join(esmDir, minify ? 'browser.min.js' : 'browser.js'), browserESM)
}

// Generate the "exit0" stubs
const exit0Map = {};
const exit0Dir = path.join(repoDir, 'lib', 'npm', 'exit0');
for (const entry of fs.readdirSync(exit0Dir)) {
if (entry.endsWith('.node')) {
const absPath = path.join(exit0Dir, entry);
const compressed = zlib.deflateRawSync(fs.readFileSync(absPath), { level: 9 });
exit0Map[entry] = compressed.toString('base64');
}
}
const exit0Code = `
// Each of these is a native module that calls "exit(0)". This is a workaround
// for https://github.com/nodejs/node/issues/36616. These native modules are
// stored in a string both to make them smaller and to hide them from Yarn 2,
// since they make Yarn 2 unzip this package.
module.exports = ${JSON.stringify(exit0Map, null, 2)};
`;
fs.writeFileSync(path.join(npmWasmDir, 'exit0.js'), exit0Code);

// Join with the asynchronous WebAssembly build
await goBuildPromise;

Expand All @@ -217,7 +197,6 @@ module.exports = ${JSON.stringify(exit0Map, null, 2)};
fs.mkdirSync(path.join(dir, 'bin'), { recursive: true })
fs.writeFileSync(path.join(dir, 'wasm_exec.js'), wasm_exec_js);
fs.writeFileSync(path.join(dir, 'wasm_exec_node.js'), wasm_exec_node_js);
fs.writeFileSync(path.join(dir, 'exit0.js'), exit0Code);
fs.copyFileSync(path.join(npmWasmDir, 'bin', 'esbuild'), path.join(dir, 'bin', 'esbuild'));
fs.copyFileSync(path.join(npmWasmDir, 'esbuild.wasm'), path.join(dir, 'esbuild.wasm'));
}
Expand Down

0 comments on commit c3b0890

Please sign in to comment.