async_hooks: don't reuse resource in HttpAgent #27581

Flarna · 2019-05-06T14:02:44Z

As discussed in nodejs/diagnostics#248, #21313 and https://docs.google.com/document/d/1g8OrG5lMIUhRn1zbkutgY83MiTSMx-0NHDs8Bf-nXxM/preview reusing the resource object is a blocker for landing a resource based async hooks API and get rid of the promise destroy hook.

This PR ensures that HttpAgent uses the a new resource object in case the socket handle gets reused.

fyi @nodejs/diagnostics

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
commit message follows commit guidelines

As discussed in nodejs/diagnostics#248, nodejs#21313 and https://docs.google.com/document/d/1g8OrG5lMIUhRn1zbkutgY83MiTSMx-0NHDs8Bf-nXxM/preview reusing the resource object is a blocker for landing a resource based async hooks API and get rid of the promise destroy hook. This PR ensures that HttpAgent uses the a new resource object in case the socket handle gets reused.

BridgeAR · 2019-05-13T08:39:33Z

@nodejs/diagnostics PTAL

Trott · 2019-05-13T21:21:42Z

I'll review this if I have to, but that's going to largely be a rubber-stamp review, and I'd prefer someone better-suited to review it do so. @nodejs/diagnostics @nodejs/http

Flarna · 2019-05-13T21:53:14Z

Maybe @nodejs/async_hooks should take a look as in the end it's a preparation for a resource based solution proposed by @mcollina in #21313.

mcollina

LGTM on green CI.

nodejs-github-bot · 2019-05-14T07:15:36Z

CI: https://ci.nodejs.org/job/node-test-pull-request/23104/

test/async-hooks/test-http-agent-handle-reuse.js

Trott

Some minor suggestions for the test, but the test LGTM with or without the suggestions.

nodejs-github-bot · 2019-05-16T09:16:24Z

CI: https://ci.nodejs.org/job/node-test-pull-request/23138/

Flarna · 2019-05-16T23:05:12Z

CI fail looks like #27617

nodejs-github-bot · 2019-05-17T20:52:13Z

CI: https://ci.nodejs.org/job/node-test-pull-request/23166/

addaleax · 2019-05-19T21:16:20Z

src/async_wrap.cc

-      args[0]->IsNumber() ? args[0].As<Number>()->Value() : kInvalidAsyncId;
-  wrap->AsyncReset(execution_async_id);
+      args[1]->IsNumber() ? args[1].As<Number>()->Value() : kInvalidAsyncId;
+  wrap->AsyncReset(resource, execution_async_id);
 }


This doesn’t have to stop this from landing, but thinking out loud a bit … I think we don’t have to deviate from the original async_hooks design this strongly, at least as far as the 1:1 correspondence of native objects and JS objects is concerned (which also has the advantage of having the resource objects provide information about the resource itself, which this PR undoes).

I think ideally what we would want for these cases (and maybe others) is to make the resource object passed to async_hooks refer to the native object, and give the native object a property over which we can figure out the current async_hooks resource object assoicated with it.

As far as I know for promises the resource passed via AsyncHooks is not linking to the Promise object itself at this time. So at least there we deviate already.

To my understanding passing the native resource would avoid to implement currentResource (#21313) as long as they are reused as setting a property on them or use them in a WeakMap would result in leak (except if the property gets removed via delete hook but this is the hook we want to get rid of).

I fully agree that it is not optimal to use different object types as resources depending on reuse or not.

My idea in this is different but not sure if it can implemented efficiently: Instead of passing the native resource object itself (which is often internal and not well documented, sometimes partly initialized,...) we could pass a well defined object for all resources. This object offers an APIs related to the current executing resource and even more related to scheduling (a name I have seen in other languages: ExecutionContext). With such an object in placed it's easier to extend the API as we are no longer bound to already existing low level objects.
If we consider the currently passed resource objects and types as part of the AsyncHooks API it's in my opinion more or less impossible to get it out of experimental as to much low level APIs are then part of the public API which would hinder refactorings,...

I could imagine that it would be also more efficient to think/signal more like a scheduler and issue only one executionChanged event with old and new context instead of a after event directly followed by a before event.

But again, this is just some idea I have in mind, not yet tried to implement it and not even sure if we get the needed information from v8/libuv. For sure this would be a breaking change.

My idea in this is different but not sure if it can implemented efficiently: Instead of passing the native resource object itself (which is often internal and not well documented, sometimes partly initialized,...) we could pass a well defined object for all resources. This object offers an APIs related to the current executing resource and even more related to scheduling (a name I have seen in other languages: ExecutionContext). With such an object in placed it's easier to extend the API as we are no longer bound to already existing low level objects.

@Flarna Yeah, I think we are in agreement here :)

addaleax · 2019-05-19T21:17:51Z

Landed in 3d9d1ad

As discussed in nodejs/diagnostics#248, #21313 and https://docs.google.com/document/d/1g8OrG5lMIUhRn1zbkutgY83MiTSMx-0NHDs8Bf-nXxM/preview reusing the resource object is a blocker for landing a resource based async hooks API and get rid of the promise destroy hook. This PR ensures that HttpAgent uses the a new resource object in case the socket handle gets reused. PR-URL: #27581 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com>

PR-URL: #34439 Fixes: #34401 Refs: #27581 Reviewed-By: Vladimir de Turckheim <vlad2t@hotmail.com> Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>

nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. http Issues or PRs related to the http subsystem. labels May 6, 2019

Flarna mentioned this pull request May 10, 2019

Deep dive discussion on {Async,Promise}_hooks & performance nodejs/diagnostics#248

Closed

BridgeAR added the review wanted PRs that need reviews. label May 13, 2019

mcollina approved these changes May 14, 2019

View reviewed changes

Trott reviewed May 14, 2019

View reviewed changes

test/async-hooks/test-http-agent-handle-reuse.js Show resolved Hide resolved

Trott reviewed May 14, 2019

View reviewed changes

test/async-hooks/test-http-agent-handle-reuse.js Outdated Show resolved Hide resolved

Trott reviewed May 14, 2019

View reviewed changes

test/async-hooks/test-http-agent-handle-reuse.js Show resolved Hide resolved

Trott approved these changes May 14, 2019

View reviewed changes

Trott added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label May 14, 2019

fixup: correct typo

ef62c5a

addaleax reviewed May 19, 2019

View reviewed changes

addaleax closed this May 19, 2019

Flarna deleted the async-hooks-reuse-http-agent branch May 19, 2019 21:48

BridgeAR mentioned this pull request May 21, 2019

v12.3.0 proposal #27799

Merged

4 tasks

Flarna mentioned this pull request Aug 7, 2019

process: improve nextTick performance #25461

Merged

2 tasks

Jeff-Lewis mentioned this pull request Dec 4, 2019

context is lost on 'end' event of http request stream Jeff-Lewis/cls-hooked#36

Open

Flarna mentioned this pull request Jan 28, 2020

async_hooks: add executionAsyncResource #30959

Closed

4 tasks

puzpuzpuz mentioned this pull request Jul 20, 2020

async_hooks: don't reuse resource in HttpAgent when queued #34439

Closed

3 tasks

puzpuzpuz added a commit that referenced this pull request Jul 30, 2020

async_hooks: don't reuse resource in HttpAgent when queued

019ea07

PR-URL: #34439 Fixes: #34401 Refs: #27581 Reviewed-By: Vladimir de Turckheim <vlad2t@hotmail.com> Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>

codebytere pushed a commit that referenced this pull request Aug 5, 2020

async_hooks: don't reuse resource in HttpAgent when queued

14656e1

PR-URL: #34439 Fixes: #34401 Refs: #27581 Reviewed-By: Vladimir de Turckheim <vlad2t@hotmail.com> Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

async_hooks: don't reuse resource in HttpAgent #27581

async_hooks: don't reuse resource in HttpAgent #27581

Flarna commented May 6, 2019 •

edited

BridgeAR commented May 13, 2019

Trott commented May 13, 2019

Flarna commented May 13, 2019

mcollina left a comment

nodejs-github-bot commented May 14, 2019

Trott left a comment

nodejs-github-bot commented May 16, 2019

Flarna commented May 16, 2019 •

edited

nodejs-github-bot commented May 17, 2019

addaleax May 19, 2019

Flarna May 19, 2019

addaleax May 19, 2019

addaleax commented May 19, 2019

async_hooks: don't reuse resource in HttpAgent #27581

async_hooks: don't reuse resource in HttpAgent #27581

Conversation

Flarna commented May 6, 2019 • edited

Checklist

BridgeAR commented May 13, 2019

Trott commented May 13, 2019

Flarna commented May 13, 2019

mcollina left a comment

Choose a reason for hiding this comment

nodejs-github-bot commented May 14, 2019

Trott left a comment

Choose a reason for hiding this comment

nodejs-github-bot commented May 16, 2019

Flarna commented May 16, 2019 • edited

nodejs-github-bot commented May 17, 2019

addaleax May 19, 2019

Choose a reason for hiding this comment

Flarna May 19, 2019

Choose a reason for hiding this comment

addaleax May 19, 2019

Choose a reason for hiding this comment

addaleax commented May 19, 2019

Flarna commented May 6, 2019 •

edited

Flarna commented May 16, 2019 •

edited