stream: pre-allocate _events #50428

ronag · 2023-10-27T14:27:57Z

19:50:22 streams/creation.js kind='duplex' n=50000000                                                    ***      8.93 %       ±1.20% ±1.60%  ±2.10%
19:50:22 streams/creation.js kind='readable' n=50000000                                                  ***      3.39 %       ±1.30% ±1.74%  ±2.30%
19:50:22 streams/creation.js kind='transform' n=50000000                                                 ***     58.62 %       ±1.72% ±2.29%  ±2.99%
19:50:22 streams/creation.js kind='writable' n=50000000                                                  ***     11.96 %       ±1.20% ±1.60%  ±2.08%
19:50:22 streams/destroy.js kind='duplex' n=1000000                                                      ***      5.30 %       ±2.48% ±3.30%  ±4.29%
19:50:22 streams/destroy.js kind='readable' n=1000000                                                    ***      7.73 %       ±2.38% ±3.17%  ±4.12%
19:50:22 streams/destroy.js kind='transform' n=1000000                                                   ***     16.09 %       ±1.93% ±2.57%  ±3.34%
19:50:22 streams/destroy.js kind='writable' n=1000000                                                    ***      6.01 %       ±2.11% ±2.80%  ±3.65%
19:50:22 streams/pipe-object-mode.js n=5000000                                                           ***     14.94 %       ±5.86% ±7.80% ±10.17%
19:50:22 streams/pipe.js n=5000000                                                                       ***     14.50 %       ±4.97% ±6.62%  ±8.64%
19:50:22 streams/readable-async-iterator.js sync='no' n=100000                                                   -0.00 %       ±5.84% ±7.78% ±10.12%
19:50:22 streams/readable-async-iterator.js sync='yes' n=100000                                           **      5.06 %       ±3.67% ±4.94%  ±6.54%
19:50:22 streams/readable-bigread.js n=1000                                                               **      4.02 %       ±2.77% ±3.71%  ±4.87%
19:50:22 streams/readable-bigunevenread.js n=1000                                                                 0.81 %       ±1.80% ±2.40%  ±3.14%
19:50:22 streams/readable-boundaryread.js type='buffer' n=2000                                           ***     22.18 %       ±1.51% ±2.02%  ±2.63%
19:50:22 streams/readable-boundaryread.js type='string' n=2000                                           ***      4.77 %       ±1.36% ±1.81%  ±2.35%
19:50:22 streams/readable-from.js type='array' n=10000000                                                ***     23.14 %       ±4.33% ±5.76%  ±7.51%
19:50:22 streams/readable-from.js type='async-generator' n=10000000                                      ***      6.25 %       ±0.79% ±1.05%  ±1.36%
19:50:22 streams/readable-from.js type='sync-generator-with-async-values' n=10000000                     ***      8.63 %       ±1.02% ±1.35%  ±1.76%
19:50:22 streams/readable-from.js type='sync-generator-with-sync-values' n=10000000                      ***     12.72 %       ±3.93% ±5.24%  ±6.82%
19:50:22 streams/readable-readall.js n=5000                                                                      -1.10 %       ±5.57% ±7.41%  ±9.64%
19:50:22 streams/readable-unevenread.js n=1000                                                           ***      6.29 %       ±0.49% ±0.65%  ±0.85%
19:50:22 streams/writable-manywrites.js len=1024 callback='no' writev='no' sync='no' n=100000                    -0.78 %       ±0.99% ±1.31%  ±1.71%
19:50:22 streams/writable-manywrites.js len=1024 callback='no' writev='no' sync='yes' n=100000             *      3.11 %       ±2.41% ±3.21%  ±4.18%
19:50:22 streams/writable-manywrites.js len=1024 callback='no' writev='yes' sync='no' n=100000                    0.90 %       ±1.75% ±2.33%  ±3.03%
19:50:22 streams/writable-manywrites.js len=1024 callback='no' writev='yes' sync='yes' n=100000            *     -2.74 %       ±2.45% ±3.27%  ±4.25%
19:50:22 streams/writable-manywrites.js len=1024 callback='yes' writev='no' sync='no' n=100000                    0.12 %       ±0.88% ±1.18%  ±1.54%
19:50:22 streams/writable-manywrites.js len=1024 callback='yes' writev='no' sync='yes' n=100000                  -0.57 %       ±1.86% ±2.49%  ±3.25%
19:50:22 streams/writable-manywrites.js len=1024 callback='yes' writev='yes' sync='no' n=100000                  -2.04 %       ±2.35% ±3.14%  ±4.10%
19:50:22 streams/writable-manywrites.js len=1024 callback='yes' writev='yes' sync='yes' n=100000                 -1.62 %       ±2.09% ±2.78%  ±3.62%
19:50:22 streams/writable-manywrites.js len=32768 callback='no' writev='no' sync='no' n=100000                    1.42 %       ±2.64% ±3.55%  ±4.67%
19:50:22 streams/writable-manywrites.js len=32768 callback='no' writev='no' sync='yes' n=100000                   0.52 %       ±2.67% ±3.56%  ±4.63%
19:50:22 streams/writable-manywrites.js len=32768 callback='no' writev='yes' sync='no' n=100000                   2.15 %       ±2.44% ±3.27%  ±4.31%
19:50:22 streams/writable-manywrites.js len=32768 callback='no' writev='yes' sync='yes' n=100000                 -0.64 %       ±2.51% ±3.34%  ±4.35%
19:50:22 streams/writable-manywrites.js len=32768 callback='yes' writev='no' sync='no' n=100000                   2.31 %       ±2.56% ±3.43%  ±4.52%
19:50:22 streams/writable-manywrites.js len=32768 callback='yes' writev='no' sync='yes' n=100000                 -0.01 %       ±2.33% ±3.10%  ±4.04%
19:50:22 streams/writable-manywrites.js len=32768 callback='yes' writev='yes' sync='no' n=100000                  0.73 %       ±2.70% ±3.62%  ±4.76%
19:50:22 streams/writable-manywrites.js len=32768 callback='yes' writev='yes' sync='yes' n=100000                 1.68 %       ±2.08% ±2.77%  ±3.61%

nodejs-github-bot · 2023-10-27T14:28:02Z

Review requested:

@nodejs/streams

rluvaton · 2023-10-27T14:37:27Z

Will continue the review tomorrow night, but one general note, I think you should avoid deleting those events when removing listener, and IIRC I think we use the number of keys exists in order to see how many events we listen to

mcollina

lgtm

ronag · 2023-10-27T16:32:37Z

https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1466/

nodejs-github-bot · 2023-10-27T17:53:40Z

CI: https://ci.nodejs.org/job/node-test-pull-request/55272/

nodejs-github-bot · 2023-10-27T19:39:31Z

CI: https://ci.nodejs.org/job/node-test-pull-request/55274/

nodejs-github-bot · 2023-10-28T06:42:40Z

CI: https://ci.nodejs.org/job/node-test-pull-request/55279/

lib/internal/streams/duplex.js

rluvaton · 2023-10-28T15:53:11Z

lib/events.js

@@ -750,6 +758,7 @@ EventEmitter.prototype.removeAllListeners =
          else
            delete events[type];
        }
+        this[kShapeMode] = false;


Why? What if we just have an object template (template in the abstract way not v8 one) and use that when resetting? (Same for the one below)

(This is not really important comment as it's not happening on a regular basis)

I don't understand

Maybe reset _events to the template you provided (e.g. with data event key) instead to the default one (i.e. empty with proto null)

That would require us to keep the template in memory and then create copies. Maybe having some kind of reset/initEvents override is better but starts to make things complciated.

rluvaton · 2023-10-28T15:58:28Z

lib/internal/streams/duplex.js

@@ -63,6 +63,24 @@ function Duplex(options) {
  if (!(this instanceof Duplex))
    return new Duplex(options);

+  this._events ??= {


I don't really think we should modify the private property _events like that, maybe have an option or an api for setting events template in the EventEmitter class?

Follow up PR welcome?

nodejs-github-bot · 2023-10-29T16:30:00Z

Landed in 2aaa21f

No longer necessary given recent stream and event optimziations. Refs: nodejs#50428 Refs: nodejs#50439 PR-URL: nodejs#50440

PR-URL: nodejs#50428 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

PR-URL: #50428 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com> Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

Qard · 2023-12-28T12:23:37Z

This seems to have broken the emitter.eventNames() function. Because it pre-allocates the _events object it will now report that it has handlers for all those pre-allocated names even if none actually exist.

Refs: nodejs#50428

Refs: #50428 PR-URL: #51925 Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>

Refs: nodejs#50428 PR-URL: nodejs#51925 Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>

nodejs-github-bot added the needs-ci PRs that need a full CI run. label Oct 27, 2023

ronag requested review from mcollina and benjamingr October 27, 2023 15:11

ronag force-pushed the stream-events branch from f3d826a to 321c41c Compare October 27, 2023 15:23

mcollina approved these changes Oct 27, 2023

View reviewed changes

ronag force-pushed the stream-events branch from 39d2410 to 7293910 Compare October 27, 2023 16:11

ronag force-pushed the stream-events branch 3 times, most recently from 54755f2 to 4e46fdf Compare October 27, 2023 16:46

ronag added request-ci Add this label to start a Jenkins CI on a PR. author ready PRs that have at least one approval, no pending requests for changes, and a CI started. labels Oct 27, 2023

ronag requested a review from rluvaton October 27, 2023 17:51

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Oct 27, 2023

ronag force-pushed the stream-events branch 2 times, most recently from cc6dfe0 to 89ad099 Compare October 27, 2023 19:35

ronag added the request-ci Add this label to start a Jenkins CI on a PR. label Oct 27, 2023

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Oct 27, 2023

github-actions bot mentioned this pull request Oct 28, 2023

CI Reliability 2023-10-28 nodejs/reliability#700

Open

22 tasks

ronag added the request-ci Add this label to start a Jenkins CI on a PR. label Oct 28, 2023

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Oct 28, 2023

ronag force-pushed the stream-events branch 3 times, most recently from 673f925 to 507478e Compare October 28, 2023 06:51

ronag added the request-ci Add this label to start a Jenkins CI on a PR. label Oct 28, 2023

rluvaton reviewed Oct 28, 2023

View reviewed changes

github-actions bot mentioned this pull request Oct 29, 2023

CI Reliability 2023-10-29 nodejs/reliability#701

Open

21 tasks

rluvaton mentioned this pull request Oct 29, 2023

events: improve events performance #50422

Closed

2 tasks

ronag added the commit-queue Add this label to land a pull request using GitHub Actions. label Oct 29, 2023

nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Oct 29, 2023

nodejs-github-bot merged commit 2aaa21f into nodejs:main Oct 29, 2023
53 checks passed

ronag added a commit to nxtedition/node that referenced this pull request Oct 29, 2023

stream: remove LazyTransform

06c89b4

No longer necessary given recent stream and event optimziations. Refs: nodejs#50428 Refs: nodejs#50439 PR-URL: nodejs#50440

ronag added a commit to nxtedition/node that referenced this pull request Oct 29, 2023

stream: remove LazyTransform

2bad63b

No longer necessary given recent stream and event optimziations. Refs: nodejs#50428 Refs: nodejs#50439 PR-URL: nodejs#50440

ronag added a commit to nxtedition/node that referenced this pull request Oct 29, 2023

stream: remove LazyTransform

3bab912

No longer necessary given recent stream and event optimziations. Refs: nodejs#50428 Refs: nodejs#50439 PR-URL: nodejs#50440

github-actions bot mentioned this pull request Oct 30, 2023

CI Reliability 2023-10-30 nodejs/reliability#702

Open

25 tasks

ronag added a commit to nxtedition/node that referenced this pull request Oct 30, 2023

stream: remove LazyTransform

c249549

No longer necessary given recent stream and event optimziations. Refs: nodejs#50428 Refs: nodejs#50439 PR-URL: nodejs#50440

This was referenced Oct 31, 2023

CI Reliability 2023-10-31 nodejs/reliability#703

Open

CI Reliability 2023-11-01 nodejs/reliability#704

Open

targos mentioned this pull request Nov 12, 2023

v21.2.0 release proposal #50681

Merged

UlisesGascon mentioned this pull request Dec 12, 2023

v20.11.0 proposal #51124

Merged

This was referenced Dec 28, 2023

Fix net plugin tests DataDog/dd-trace-js#3906

Merged

stream.eventNames() broken on 21.2.0+ #51302

Open

deokjinkim added a commit to deokjinkim/node that referenced this pull request Feb 29, 2024

doc: fix actual result of example is different in events

316c57d

Refs: nodejs#50428

deokjinkim mentioned this pull request Feb 29, 2024

doc: fix actual result of example is different in events #51925

Merged

targos pushed a commit that referenced this pull request Mar 7, 2024

doc: fix actual result of example is different in events

87d2acc

Refs: #50428 PR-URL: #51925 Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>

richardlau pushed a commit that referenced this pull request Mar 25, 2024

doc: fix actual result of example is different in events

34edd42

Refs: #50428 PR-URL: #51925 Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>

richardlau pushed a commit that referenced this pull request Mar 25, 2024

doc: fix actual result of example is different in events

fc0b389

Refs: #50428 PR-URL: #51925 Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

stream: pre-allocate _events #50428

stream: pre-allocate _events #50428

ronag commented Oct 27, 2023 •

edited

nodejs-github-bot commented Oct 27, 2023 •

edited by ronag

rluvaton commented Oct 27, 2023

mcollina left a comment

ronag commented Oct 27, 2023

nodejs-github-bot commented Oct 27, 2023

nodejs-github-bot commented Oct 27, 2023

nodejs-github-bot commented Oct 28, 2023

rluvaton Oct 28, 2023

ronag Oct 28, 2023

rluvaton Oct 29, 2023

ronag Oct 29, 2023

rluvaton Oct 28, 2023

ronag Oct 28, 2023

nodejs-github-bot commented Oct 29, 2023

Qard commented Dec 28, 2023 •

edited

stream: pre-allocate _events #50428

stream: pre-allocate _events #50428

Conversation

ronag commented Oct 27, 2023 • edited

nodejs-github-bot commented Oct 27, 2023 • edited by ronag

rluvaton commented Oct 27, 2023

mcollina left a comment

Choose a reason for hiding this comment

ronag commented Oct 27, 2023

nodejs-github-bot commented Oct 27, 2023

nodejs-github-bot commented Oct 27, 2023

nodejs-github-bot commented Oct 28, 2023

rluvaton Oct 28, 2023

Choose a reason for hiding this comment

ronag Oct 28, 2023

Choose a reason for hiding this comment

rluvaton Oct 29, 2023

Choose a reason for hiding this comment

ronag Oct 29, 2023

Choose a reason for hiding this comment

rluvaton Oct 28, 2023

Choose a reason for hiding this comment

ronag Oct 28, 2023

Choose a reason for hiding this comment

nodejs-github-bot commented Oct 29, 2023

Qard commented Dec 28, 2023 • edited

ronag commented Oct 27, 2023 •

edited

nodejs-github-bot commented Oct 27, 2023 •

edited by ronag

Qard commented Dec 28, 2023 •

edited