Tests take a long time to run #47146

targos · 2023-03-18T09:53:12Z

There was a time (not so long a go, maybe ~1 year) when make -s jstest took 2-3 minutes on my previous computer.
Now I have a decent Mac with M2 Pro and the same command takes 8 minutes! I don't know what changed, but I doubt we more than doubled the number of tests since then.
Are there some relatively new tests which are specifically slow?

The text was updated successfully, but these errors were encountered:

RaisinTen · 2023-03-20T08:42:19Z

test/parallel/test-single-executable-application.js has been added recently and it's a slow test. On my Intel Mac, it runs for ~38 seconds but I would expect it to be a bit faster on M2 Pro.

bnoordhuis · 2023-03-20T10:18:21Z

The test runner unfortunately doesn't have an option to print the running time of individual tests but lowering the timeout (default is 120 seconds) will identify the slow ones:

$ python tools/test.py -J --timeout 5 parallel

richardlau · 2023-03-20T12:28:14Z

test-webcrypto takes over a minute to run on my system.

targos · 2023-03-20T13:11:53Z

$ python tools/test.py -J --timeout 5
=== release test-http-server-headers-timeout-keepalive ===
Path: parallel/test-http-server-headers-timeout-keepalive
Command: out/Release/node /Users/mzasso/git/nodejs/node/test/parallel/test-http-server-headers-timeout-keepalive.js
--- TIMEOUT ---


=== release test-http-server-request-timeout-keepalive ===
Path: parallel/test-http-server-request-timeout-keepalive
Command: out/Release/node /Users/mzasso/git/nodejs/node/test/parallel/test-http-server-request-timeout-keepalive.js
--- TIMEOUT ---


=== release test-single-executable-application ===
Path: parallel/test-single-executable-application
Command: out/Release/node /Users/mzasso/git/nodejs/node/test/parallel/test-single-executable-application.js
--- TIMEOUT ---


[04:11|% 100|+ 3721|-   3]: Done

$ time ./node test/parallel/test-http-server-headers-timeout-keepalive.js
./node test/parallel/test-http-server-headers-timeout-keepalive.js  0.11s user 0.02s system 0% cpu 15.088 total
$ time ./node test/parallel/test-http-server-request-timeout-keepalive.js
./node test/parallel/test-http-server-request-timeout-keepalive.js  0.11s user 0.02s system 0% cpu 15.090 total
$ time ./node test/parallel/test-single-executable-application.js
./node test/parallel/test-single-executable-application.js  18.78s user 0.53s system 98% cpu 19.641 total

$ make -s jstest

...

[05:27|% 100|+ 3878|-   0]: Done

All tests passed.

Full run took 5m30s this time. Quite far from the 8 minutes I got last time 🤔

joyeecheung · 2023-03-27T20:16:43Z

I think #44342 should help this a bit (which should enable us to move these sequential tests to parallel, but I haven't checked locally yet)

joyeecheung · 2023-03-27T22:15:29Z

I opened #47274 (superseding #44342), which locally cut the time by 19s

PR-URL: #47283 Refs: #47146 Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>

PR-URL: #47273 Refs: #47146 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

To avoid failures when there is another running process occupying the port 9229 which may happen if there is a stale process, use the --port argument of node-inspect to use a random port in tests that don't have to work on port 9229. The following tests are not touched: - test-debugger-custom-port: tests a specific port - test-debugger-debug-brk: tests a specific port - test-debugger-invalid-args: tests other inspect combinations - test-debugger-pid: node-inspect does not support -p and --port together - test-debugger-launch: tests that default port is 9229 PR-URL: #47274 Refs: #47146 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Moshe Atlow <moshe@atlow.co.il> Reviewed-By: James M Snell <jasnell@gmail.com>

Before ``` ❯ tools/test.py "sequential/test-debugger*" [00:25|% 100|+ 32|- 0]: Done All tests passed. ❯ tools/test.py -J "parallel/test-debugger*" [00:05|% 100|+ 6|- 0]: Done All tests passed. ``` After ``` ❯ tools/test.py "sequential/test-debugger*" [00:06|% 100|+ 5|- 0]: Done All tests passed. ❯ tools/test.py -J "parallel/test-debugger*" [00:05|% 100|+ 33|- 0]: Done All tests passed. ``` PR-URL: #47274 Refs: #47146 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Moshe Atlow <moshe@atlow.co.il> Reviewed-By: James M Snell <jasnell@gmail.com>