[v20.x] backport vm-related memory fixes #49874

PR-URL: nodejs#45684 Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Robert Nagy <ronagy@icloud.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: Moshe Atlow <moshe@atlow.co.il> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>

PR-URL: nodejs#49826 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: Qingyu Deng <i@ayase-lab.com> Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com> Reviewed-By: Daeyeon Jeong <daeyeon.dev@gmail.com> Reviewed-By: Moshe Atlow <moshe@atlow.co.il> Reviewed-By: Darshan Sen <raisinten@gmail.com> Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com> Reviewed-By: Chemi Atlow <chemi@atlow.co.il> Reviewed-By: LiviaMedeiros <livia@cirno.name>

This enables the option `--force-node-api-uncaught-exceptions-policy` for a specific Node-API addon when it is compiled with `NAPI_EXPERIMENTAL` (and this would be the default behavior when `NAPI_VERSION` 10 releases). This would not break existing Node-API addons. PR-URL: nodejs#49313 Refs: nodejs#36510 Reviewed-By: Michael Dawson <midawson@redhat.com> Reviewed-By: Gabriel Schulhof <gabrielschulhof@gmail.com>

Refs: https://github.com/nodejs/node-core-utils/releases/tag/v4.0.0 PR-URL: nodejs#49851 Refs: nodejs/node-core-utils#700 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>

PR-URL: nodejs#49788 Reviewed-By: Tobias Nießen <tniessen@tnie.de> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>

Original commit message: Ignore --predictable when computing flag list hashes This allows reproducible code cache generation. Refs: nodejs#48749 Change-Id: Ib4693de60ddff1fe41d95c10980f763463db3f95 Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4681766 Reviewed-by: Leszek Swirski <leszeks@chromium.org> Commit-Queue: Joyee Cheung <joyee@igalia.com> Cr-Commit-Position: refs/heads/main@{#88943} Refs: v8/v8@b33bf2d PR-URL: nodejs#49703 Refs: v8/v8@de9a5de Reviewed-By: Jiawen Geng <technicalcute@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Richard Lau <rlau@redhat.com>

Original commit message: Ignore flags implied by --predictable during hash computation https://chromium-review.googlesource.com/c/v8/v8/+/4681766 added code to ignore --predictable during hash computation of flags in order to produce reproducible code cache. This turns out to be not enough since the flags implied by --predictable also need to be ignored for it to work. This patch makes sure that they are ignored. Change-Id: Ifa36641efe3ca105706fd293be46fc974055d2d4 Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4851287 Commit-Queue: Joyee Cheung <joyee@igalia.com> Reviewed-by: Leszek Swirski <leszeks@chromium.org> Reviewed-by: Patrick Thier <pthier@chromium.org> Cr-Commit-Position: refs/heads/main@{#90022} Refs: v8/v8@de9a5de PR-URL: nodejs#49703 Refs: v8/v8@b33bf2d Reviewed-By: Jiawen Geng <technicalcute@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Richard Lau <rlau@redhat.com>

PR-URL: nodejs#49683 Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>

PR-URL: nodejs#49858 Refs: nodejs#49683 Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: Richard Lau <rlau@redhat.com>

This has been flaky on many platforms for months. Mark it as flaky for now to avoid blocking the CI. PR-URL: nodejs#49854 Refs: nodejs#49853 Refs: nodejs/reliability#673 Reviewed-By: Tobias Nießen <tniessen@tnie.de> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: LiviaMedeiros <livia@cirno.name> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>

PR-URL: nodejs#49691 Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>

PR-URL: nodejs#49871 Refs: nodejs@fbe28e2 Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Chemi Atlow <chemi@atlow.co.il>

PR-URL: nodejs#49692 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Robert Nagy <ronagy@icloud.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>

PR-URL: nodejs#49834 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Robert Nagy <ronagy@icloud.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>

PR-URL: nodejs#49878 Fixes: nodejs#49853 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com> Reviewed-By: Chemi Atlow <chemi@atlow.co.il>

PR-URL: nodejs#49803 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>

PR-URL: nodejs#49856 Refs: nodejs/performance#106 Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>

PR-URL: nodejs#49857 Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: Michaël Zasso <targos@protonmail.com>

PR-URL: nodejs#38771 Reviewed-By: Darshan Sen <raisinten@gmail.com>

This is a non-ABI breaking solution for v8/v8@b60a03d and v8/v8@0aa622e which are necessary for backporting vm-related memory fixes to v20.x.

Previously when managing the importModuleDynamically callback of vm.compileFunction(), we use an ID number as the host defined option and maintain a per-Environment ID -> CompiledFnEntry map to retain the top-level referrer function returned by vm.compileFunction() in order to pass it back to the callback, but it would leak because with how we used v8::Persistent to maintain this reference, V8 would not be able to understand the cycle and would just think that the CompiledFnEntry was supposed to live forever. We made an attempt to make that reference known to V8 by making the CompiledFnEntry weak and using a private symbol to make CompiledFnEntry strongly references the top-level referrer function in nodejs#46785, but that turned out to be unsound, because the there's no guarantee that the top-level function must be alive while import() can still be initiated from that function, since V8 could discard the top-level function and only keep inner functions alive, so relying on the top-level function to keep the CompiledFnEntry alive could result in use-after-free which caused a revert of that fix. With this patch we use a symbol in the host defined options instead of a number, because with the stage-3 symbol-as-weakmap-keys proposal we could directly use that symbol to keep the referrer alive using a WeakMap. As a bonus this also keeps the other kinds of referrers alive as long as import() can still be initiated from that Script/Module, so this also fixes the long-standing crash caused by vm.Script being GC'ed too early when its importModuleDynamically callback still needs it. PR-URL: nodejs#48510 Refs: nodejs#44211 Refs: nodejs#42080 Refs: nodejs#47096 Refs: nodejs#43205 Refs: nodejs#38695 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>

Previously we maintain a strong persistent reference to the ModuleWrap to retrieve the ID-to-ModuleWrap mapping from the HostImportModuleDynamicallyCallback using the number ID stored in the host-defined options. As a result the ModuleWrap would be kept alive until the Environment is shut down, which would be a leak for user code. With the new symbol-based host-defined option we can just get the ModuleWrap from the JS-land WeakMap so there's now no need to maintain this strong reference. This would at least fix the leak for vm.SyntheticModule. vm.SourceTextModule is still leaking due to the strong persistent reference to the v8::Module. PR-URL: nodejs#48510 Refs: nodejs#44211 Refs: nodejs#42080 Refs: nodejs#47096 Refs: nodejs#43205 Refs: nodejs#38695 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>

Replace the persistent handles to v8::Module and v8::UnboundScript with an internal reference that V8's GC is aware of to fix the leaks. PR-URL: nodejs#48510 Refs: nodejs#44211 Refs: nodejs#42080 Refs: nodejs#47096 Refs: nodejs#43205 Refs: nodejs#38695 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>

PR-URL: nodejs#49671 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: Michaël Zasso <targos@protonmail.com>

Previously we simply create a lot of the target objects and check if the process crash due to OOM. Due to how we use emphemeron GC to handle memory management, which is inefficient but necessary for correctness, the tests can produce false positives as the GC isn't efficient enough to catch up with a very fast heap growth. This patch uses a new checkIfCollectable() utility to terminate the test early once we detect that any of the target object can actually be garbage collected. This should lower the chance of false positives. As a drive-by this also allows us to use setImmediate() to grow the heap even faster to make the tests run faster. PR-URL: nodejs#49671 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: Michaël Zasso <targos@protonmail.com>

Similar to the test-vm-source-text-module-leak fix, use a snapshot to force a thorough GC in order to prevent false positives. PR-URL: nodejs#49710 Refs: nodejs/reliability#669 Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[v20.x] backport vm-related memory fixes #49874

[v20.x] backport vm-related memory fixes #49874

Commits on Sep 28, 2023