lib/src: exit on gc unhandled promise #15126

BridgeAR · 2017-09-01T00:03:31Z

This is a follow up on #12010. Is is not complete and mainly a rebase with a few minor fixes.
All contributions go to @Fishrock123, @addaleax, @matthewloring and @ofrobots.

I have no idea about C++ and it is difficult for me to continue any further, therefore I would ask someone to step foward to continue the work here 😄

~~One N-API test is failing and I do not know how to properly fix that one.~~

Edit: I fixed the test but it seems like I have a regression somewhere in the error message printed. I am looking into that soon.

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

Affected core subsystem(s)

lib, src, doc

addaleax · 2017-09-01T01:02:38Z

One N-API test is failing and I do not know how to properly fix that one.

I think explicitly silencing by adding a no-op unhandled rejection handler should be fine :)

addaleax · 2017-09-01T01:04:08Z

By the way, the idea of adding a flag to Node that would restore the current behaviour was floated somewhere, and since then I have come to think that would be a really good thing to have.

jasnell · 2017-09-01T01:25:38Z

Yeah. I've been working on this a bit this week in between other things. The strategy I've been working on is to use flags to enable a couple of possible behaviors with the current behavior set as the default. My goal has been to have a pull request sometime next week.

BridgeAR · 2017-09-01T01:57:12Z

By the way, the idea of adding a flag to Node that would restore the current behaviour was floated somewhere, and since then I have come to think that would be a really good thing to have.

Is that not somewhat independent from this PR?

addaleax · 2017-09-01T02:29:20Z

@BridgeAR Yes, but landing this PR and then introducing the flag would create a period where the current behaviour would not be available, and with respect to timing I’m a bit worried that might hit the Node 9 timeframe perfectly :| That’s all, I’d just like to make sure it’s at least considered.

TimothyGu · 2017-09-01T05:13:10Z

test/message/promise_fast_handled_reject.out

@@ -1,3 +1,6 @@
+(node:30929) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): ReferenceError: consol is not defined
+(node:30929) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 2): ReferenceError: consol is not defined


The PID should be replaced with an asterisk for wildcard matching.

silverwind · 2017-09-01T07:20:49Z

doc/api/deprecations.md

-Unhandled promise rejections are deprecated. In the future, promise rejections
-that are not handled will terminate the Node.js process with a non-zero exit
-code.
-


Shouldn't it be marked EOL instead of being removed?

Yes. It should be marked End-of-life

benjamingr

Hey, thanks a lot for picking this up!

Left some comments, just to make sure we agree on the semantics we eventually want:

Not handling a promise within the "after-nextTick" should still produce a warning but not a deprecation warning. (TODO(Benjamingr) re-read our research from last time on the impact on async/await patterns)
Not handling a promise that performed GC should exit the process and print its stack trace like a regular exception.

benjamingr · 2017-09-01T10:23:07Z

test/parallel/test-promises-handled-reject.js

+const common = require('../common');
+
+const p = new Promise((res, rej) => {
+  consol.log('oops'); // eslint-disable-line no-undef


No reason to cause an implicit reference error here - this can just be throw new ReferenceError(...)

I did not want to change to much on the code and left the tests pretty much as they were. I will change it though.

benjamingr · 2017-09-01T10:23:34Z

test/parallel/test-promises-gc-before-handled.js

+const common = require('../common');
+
+const p = new Promise((res, rej) => {
+  consol.log('oops'); // eslint-disable-line no-undef


No reason to cause an implicit reference error here - this can just be throw new ReferenceError(...)

benjamingr · 2017-09-01T10:24:21Z

test/parallel/test-promises-gc-before-handled.js

+// Manually call GC due to possible memory constraints with attempting to
+// trigger it "naturally".
+setTimeout(common.mustCall(() => {
+  global.gc();


This looks really flakey, why 3 times? Why these delays in the setTimeout?

Are we testing that V8 doesn't drop a live reference in GC here?

We have been forced to check gc-based behaviour in setTimeouts in other cases in recent V8 versions. I don’t think anybody of has really gotten to the ground of it so far.

Oh, I see, then a comment would be nice.

I am not certain how that comment should look like. Do you have a example?

benjamingr · 2017-09-01T10:26:12Z

test/message/promise_reject.js

+
+Promise.reject(new Error('oops'));
+
+// Manually call GC due to possible memory constraints with attempting to


Why is this inside a setTimeout with 2 delay? Can't the handlers be added prior and then the global.gc calls made synchronously? (Also, why 3 times)

See above for the 3 times. I did not write these tests originally and it is somewhat difficult for me to know why it was exactly written the way it is. I think the two milliseconds delay is arbitrary. I am also not sure about your second question. As far as I see it the setTimeout is meant to make sure the rejected promise is fully recognized to be unhandled.

probably was arbitrary

benjamingr · 2017-09-01T10:26:24Z

test/message/promise_fast_reject.js

+const assert = require('assert');
+
+new Promise(function(res, rej) {
+  consol.log('One'); // eslint-disable-line no-undef


ditto about explicit errors and throwing directly rather than consol

benjamingr · 2017-09-01T10:28:37Z

test/message/promise_fast_handled_reject.js

+  consol.log('Three'); // eslint-disable-line no-undef
+});
+
+new Promise((res, rej) => {


I'm not sure what this tests, new Promise runs synchronously

Right. It should probably be a nextTick instead.

benjamingr · 2017-09-01T10:28:50Z

test/message/promise_fast_handled_reject.js

+  }), 1);
+});
+
+process.on('rejectionHandled', () => {}); // Ignore


The default behavior for rejectionHandled is ignore.

It will print a warning otherwise and I wanted to have the output as clean as possible.

benjamingr · 2017-09-01T10:29:27Z

test/addons-napi/test_promise/test.js

@@ -6,6 +6,8 @@ const assert = require('assert');

 let expected_result, promise;

+process.on('unhandledRejection', () => {});


Can we fix the test instead of ignoring the errors?

If we can't it might indicate a design problem

@addaleax recommended to ignore this. I did not find the cause of the issue here especially as the output is really little.

benjamingr · 2017-09-01T10:34:07Z

lib/internal/process/promises.js

-  let deprecationWarned = false;
+exports.setup = function setup(scheduleMicrotasks) {
+  const promiseRejectEvent = process._promiseRejectEvent;
+  const hasBeenNotifiedProperty = new Map();


Can you explain why a regular Map is sufficient here?

A WeakMap was used for a very specific reason in order to not interfere with GC.

On a related note, is there any way we can test the unhandled rejection tracking code does not leak memory?

(I realize it doesn't do the same thing exactly and tracking happens in C++, but still)

WeakMaps can't be enumerated, but no enumeration is done here; when rejectionHandled is called there is still a strong reference to the promise (the promise argument) so it should still exist in the WeakMap 🤔

Not sure why a Map is needed either; it just creates the need for .delete calls.

The reason why I changed it is that I can not see a reason to use WeakMap. All entries are either going to be handled and in that case explicitly deleted or they result in a unhandled rejection that will now terminate the process. Using the WeakMap has a performance overhead and the GC has to do more work.

benjamingr · 2017-09-01T10:36:05Z

lib/internal/process/promises.js

-    if (hasBeenNotified !== undefined) {
-      hasBeenNotifiedProperty.delete(promise);
+    hasBeenNotifiedProperty.delete(promise);
+    if (hasBeenNotified) {


the .delete can be inside the if I think

It should always be deleted, otherwise the reference would be kept. This function is only called once per handled rejection and it can not be freed otherwise.

benjamingr · 2017-09-01T10:39:56Z

By the way, the idea of adding a flag to Node that would restore the current behaviour was floated somewhere, and since then I have come to think that would be a really good thing to have.

The current behavior can be restored by adding an unhandledRejection handler and logging - if I understand correctly.

bnoordhuis · 2017-09-04T11:48:41Z

I only looked at this briefly but it has the same issue as the old PR, doesn't it? It adds a lot of overhead per promise. Weakly persistent handles aren't cheap.

benjamingr · 2017-09-04T11:59:38Z

@bnoordhuis yes, this isn't here yet - both behavior and overhead

src: use std::map for the promise reject map Refs: nodejs#5292 Refs: nodejs/promises#26 Refs: nodejs#6355 Refs: nodejs#6375

BridgeAR · 2017-09-26T05:19:55Z

I addressed the comments and rebased due to conflicts.

BridgeAR · 2017-10-01T23:59:49Z

Ping

mhdawson · 2017-10-02T18:53:35Z

@BridgeAR not 100% sure by reading through the comments so far. Is this now in a state where its ready to land ? Just asking because of the earlier comments about not being there in terms of behavior and overhead.

Fishrock123

just pointing out an obvious bit of my WIP

there may be other spots too

Fishrock123 · 2017-10-07T21:36:26Z

src/node.cc

+
+  // Make some sort of list size check so as to not leak memory.
+  if (env->promise_tracker_.Size() > 10000) {
+    // XXX(Fishrock123): Do some intelligent logic here?


for hopefully obvious reasons this shouldn't be merged in this state

Fishrock123 · 2017-10-07T21:41:04Z

src/node.cc

+    Local<Value> err = GetPromiseReason(&env, orp);
+    Local<Message> message = Exception::CreateMessage(isolate, err);
+
+    // XXX(Fishrock123): Should this just call ReportException and


mcollina · 2017-10-09T07:05:48Z

I am in favour of the approach. I would love to see this landed in Node 9.

benjamingr · 2017-10-24T09:49:19Z

@BridgeAR status?

billinghamj · 2018-01-11T01:39:11Z

It looks like once this is merged, all code currently causing PromiseRejectionHandledWarning console messages to appear will immediately break. It sounds like a decision was reached some time ago.

Where can I go to find the discussion in which this decision was reached, so I can learn more about the reasoning etc.?

I think one could argue that asynchronously handling exceptions is not an invalid use of Promises. I also think you may be surprised by how many codebases will be affected - unless telemetry data is available, of course.

Edit: apologies, I now see that this will only occur when the object is GC'd - which will mean catching the Promise is impossible anyway.

Might be worth linking some more context/background in the PR description.

targos · 2018-02-03T16:49:11Z

@BridgeAR do you still want to pursue this? I would like to see it land early enough for Node 10.

BridgeAR · 2018-02-03T16:51:07Z

In general: yes. I am not sure if I can work on it next week though. The week after is probably possible.

But it is also fine for me if someone else wants to work on it right now.

BridgeAR · 2018-04-12T18:37:51Z

Just a heads up: I am currently looking into this again and try to have a update about mid next week.

BridgeAR · 2018-04-17T12:39:16Z

Closing in favor of #20097

nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. labels Sep 1, 2017

BridgeAR added the promises Issues and PRs related to ECMAScript promises. label Sep 1, 2017

vkurchatkin added the semver-major PRs that contain breaking changes and should be released in the next major version. label Sep 1, 2017

TimothyGu reviewed Sep 1, 2017

View reviewed changes

silverwind reviewed Sep 1, 2017

View reviewed changes

benjamingr requested changes Sep 1, 2017

View reviewed changes

BridgeAR added the wip Issues and PRs that are still a work in progress. label Sep 5, 2017

benjamingr mentioned this pull request Sep 12, 2017

lib,src: eagerly exit process on unhanded promise rejections #12734

Closed

4 tasks

BridgeAR mentioned this pull request Sep 20, 2017

lib,src: exit process on unhandled promise rejection cleanup #12010

Closed

4 tasks

Fishrock123 and others added 6 commits September 26, 2017 01:54

lib,src: throw on unhanded promise rejections

711d699

src: use std::map for the promise reject map Refs: nodejs#5292 Refs: nodejs/promises#26 Refs: nodejs#6355 Refs: nodejs#6375

lib: follow up to exit early

af11384

[squash] fix linter errors

19c7eb3

[squash] fix a test and remove unnecessary fn

69bfdd6

[squash] address comments

5d2c83e

[squash] fix test

8fd484e

BridgeAR force-pushed the exit-unhandled-promise branch from 0452cde to 8fd484e Compare September 26, 2017 05:19

Fishrock123 suggested changes Oct 7, 2017

View reviewed changes

Fishrock123 reviewed Oct 7, 2017

View reviewed changes

MylesBorins force-pushed the master branch from b7405ab to 7f086dd Compare December 8, 2017 16:37

BridgeAR added the notable-change PRs with changes that should be highlighted in changelogs. label Dec 15, 2017

TimothyGu mentioned this pull request Jan 11, 2018

premature doEmitWarning Unhandled promise rejection #18108

Closed

maclover7 force-pushed the master branch from bb5575a to 993b716 Compare January 26, 2018 22:02

cjihrig force-pushed the master branch from 993b716 to 082f952 Compare January 26, 2018 22:36

BridgeAR mentioned this pull request Mar 2, 2018

assert: adds rejects() and doesNotReject() #18023

Closed

4 tasks

BridgeAR closed this Apr 17, 2018

BridgeAR deleted the exit-unhandled-promise branch April 1, 2019 23:37

mmarchini mentioned this pull request Sep 15, 2020

Voting on the default behavior for Promise unhandled rejections nodejs/TSC#916

Closed

		@@ -1,3 +1,6 @@
		(node:30929) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): ReferenceError: consol is not defined
		(node:30929) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 2): ReferenceError: consol is not defined


		Promise.reject(new Error('oops'));

		// Manually call GC due to possible memory constraints with attempting to

		@@ -6,6 +6,8 @@ const assert = require('assert');

		let expected_result, promise;

		process.on('unhandledRejection', () => {});

lib/src: exit on gc unhandled promise #15126

lib/src: exit on gc unhandled promise #15126

Conversation

BridgeAR commented Sep 1, 2017 • edited

Checklist

Affected core subsystem(s)

addaleax commented Sep 1, 2017

addaleax commented Sep 1, 2017

jasnell commented Sep 1, 2017

BridgeAR commented Sep 1, 2017

addaleax commented Sep 1, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

benjamingr left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BridgeAR Sep 21, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

benjamingr commented Sep 1, 2017

bnoordhuis commented Sep 4, 2017

benjamingr commented Sep 4, 2017

BridgeAR commented Sep 26, 2017 • edited

BridgeAR commented Oct 1, 2017

mhdawson commented Oct 2, 2017

Fishrock123 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mcollina commented Oct 9, 2017

benjamingr commented Oct 24, 2017

billinghamj commented Jan 11, 2018 • edited

targos commented Feb 3, 2018

BridgeAR commented Feb 3, 2018

BridgeAR commented Apr 12, 2018

BridgeAR commented Apr 17, 2018

BridgeAR commented Sep 1, 2017 •

edited

BridgeAR Sep 21, 2017 •

edited

BridgeAR commented Sep 26, 2017 •

edited

billinghamj commented Jan 11, 2018 •

edited