stream: fix not calling cleanup() when unpiping all streams. #18266

MoonBall · 2018-01-20T03:46:46Z

After a readable stream pipes more than one writable streams, I calls readable.unpipe() to remove all piped writable stream. I find that only the first piped writable stream calls the cleanup(), the others don't call cleanup().

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

Affected core subsystem(s)

stream

lpinca · 2018-01-20T07:21:41Z

lib/_stream_readable.js

@@ -758,8 +758,10 @@ Readable.prototype.unpipe = function(dest) {
    state.pipesCount = 0;
    state.flowing = false;

-    for (var i = 0; i < len; i++)
+    for (var i = 0; i < len; i++) {
+      unpipeInfo = { hasUnpiped: false };


For other reviewers, we need a new object becausedest sets unpipeInfo.hasUnpiped to true when 'unpipe' is emitted.

On second thought, doesn't this make the hasUnpiped guard useless? I mean, no matter what, dest will always find hasUnpiped false.

I found the PR which adds the hasUnpiped. you can see it.

@lpinca hasUnpiped solved the situation that a readable stram piped the same writable stream more than one time. This change has no influence to it.

Ok yes, it seems correct as this is for the "remove all" case.

lpinca · 2018-01-20T07:22:56Z

lib/_stream_readable.js

@@ -758,8 +758,10 @@ Readable.prototype.unpipe = function(dest) {
    state.pipesCount = 0;
    state.flowing = false;

-    for (var i = 0; i < len; i++)
+    for (var i = 0; i < len; i++) {
+      unpipeInfo = { hasUnpiped: false };
      dests[i].emit('unpipe', this, unpipeInfo);


Tiny nit: I'd just create the new object here, without using the above assignment.

There are three situations to use unpipeInfo. The above assignment can be used in two situations. I’m also fine to assign it just before using it.

lpinca · 2018-01-20T08:08:34Z

lib/_stream_readable.js

@@ -758,8 +758,12 @@ Readable.prototype.unpipe = function(dest) {
    state.pipesCount = 0;
    state.flowing = false;

-    for (var i = 0; i < len; i++)
+    // use the above assignment
+    dests[0].emit('unpipe', this, unpipeInfo);


What I meant in my previous comment was:

for (var i = 0; i < len; i++) dests[i].emit('unpipe', this, { hasUnpiped: false });

Refs: nodejs#12746

targos · 2018-01-20T08:49:14Z

/cc @nodejs/streams

apapirovski

The fix here seems good although I would prefer a test that accounts for more stuff within cleanup().

mcollina

LGTM

MoonBall · 2018-01-20T14:12:53Z

@apapirovski I added more test contents.

lpinca · 2018-01-21T20:21:16Z

test/parallel/test-stream-pipe-unpipe-streams.js

+
+    const unpipeChecker = common.mustCall(() => {
+      assert.strictEqual(
+        dest.listenerCount('unpipe'), 1,


Nit: I'd simplify this as

assert.deepStrictEqual(dest.listeners('unpipe'), [unpipeChecker]);

lpinca · 2018-01-21T20:30:18Z

test/parallel/test-stream-pipe-unpipe-streams.js

+    srcCheckEventNames.forEach((eventName) => {
+      assert.strictEqual(
+        source.listenerCount(eventName), 0,
+        `source's '${eventName}' event listeners don't be cleaned up`


Nit: replace "don't be cleaned up" with "not removed", same below.

BridgeAR · 2018-02-01T16:05:28Z

CI https://ci.nodejs.org/job/node-test-pull-request/12891/

addaleax · 2018-02-04T15:51:14Z

@MoonBall I think with this we can remove unpipeInfo altogether?

MoonBall · 2018-02-04T16:41:47Z

@addaleax The #12746 will be reverted If we remove unpipeInfo altogether. This PR didn't change the intention of #12746. It just fixed that calling unpipe() doesn't remove all piped stream.

This PR makes sure the object emitted as the 'unpipe' event in the destination stream is not shared between destination, as it would be muted. Refs: #12746 PR-URL: #18266 Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Anatoli Papirovski <apapirovski@mac.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Anna Henningsen <anna@addaleax.net>

mcollina · 2018-02-05T11:32:39Z

Landed as a52b7a2.

This PR makes sure the object emitted as the 'unpipe' event in the destination stream is not shared between destination, as it would be muted. Refs: #12746 PR-URL: #18266 Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Anatoli Papirovski <apapirovski@mac.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Anna Henningsen <anna@addaleax.net>

nodejs-github-bot added the stream Issues and PRs related to the stream subsystem. label Jan 20, 2018

lpinca approved these changes Jan 20, 2018

View reviewed changes

lpinca reviewed Jan 20, 2018

View reviewed changes

stream: fix not calling cleanup() when unpiping all streams.

3500d5f

Refs: nodejs#12746

MoonBall force-pushed the fix-unpiping-all-piped-streams branch from 08153b1 to 3500d5f Compare January 20, 2018 08:26

apapirovski approved these changes Jan 20, 2018

View reviewed changes

mcollina approved these changes Jan 20, 2018

View reviewed changes

mcollina added lts-watch-v6.x labels Jan 20, 2018

陈刚 added 2 commits January 20, 2018 22:04

Add more test content

020956f

Fix lint

8da1b72

lpinca approved these changes Jan 21, 2018

View reviewed changes

Fix code style

eba4582

jasnell approved these changes Jan 22, 2018

View reviewed changes

maclover7 force-pushed the master branch from bb5575a to 993b716 Compare January 26, 2018 22:02

cjihrig force-pushed the master branch from 993b716 to 082f952 Compare January 26, 2018 22:36

BridgeAR approved these changes Feb 1, 2018

View reviewed changes

BridgeAR added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Feb 1, 2018

addaleax approved these changes Feb 4, 2018

View reviewed changes

mcollina closed this Feb 5, 2018

ryan-ally mentioned this pull request Apr 7, 2024

[Snyk] Security upgrade eslint from 6.8.0 to 9.0.0 ryan-ally/node#105

Open

This was referenced Apr 7, 2024

[Snyk] Security upgrade eslint from 7.32.0 to 9.0.0 aliscco/alisco-node#1361

Open

[Snyk] Security upgrade eslint from 5.16.0 to 9.0.0 aliscco/alisco-node#1362

Open

[Snyk] Security upgrade eslint from 5.16.0 to 9.0.0 aliscco/alisco-node#1363

Open

leonardoadame mentioned this pull request Apr 7, 2024

[Snyk] Security upgrade eslint from 4.19.1 to 9.0.0 leonardoadame/node#759

Open

This was referenced Apr 7, 2024

[Snyk] Security upgrade eslint from 4.19.1 to 9.0.0 aliscco/alisco-node#1364

Open

[Snyk] Security upgrade eslint from 4.19.1 to 9.0.0 aliscco/alisco-node#1365

Open

[Snyk] Security upgrade eslint from 4.19.1 to 9.0.0 aliscco/alisco-node#1366

Open

baby636 mentioned this pull request Apr 7, 2024

[Snyk] Security upgrade eslint from 7.32.0 to 9.0.0 baby636/node#145

Open

ryan-ally mentioned this pull request Apr 7, 2024

[Snyk] Security upgrade eslint from 6.8.0 to 9.0.0 ryan-ally/node#106

Open

aliscco mentioned this pull request Apr 7, 2024

[Snyk] Security upgrade eslint from 3.19.0 to 9.0.0 aliscco/alisco-node#1372

Open

leonardoadame mentioned this pull request Apr 7, 2024

[Snyk] Security upgrade eslint from 3.19.0 to 9.0.0 leonardoadame/node#770

Open

This was referenced Apr 8, 2024

[Snyk] Security upgrade eslint from 4.19.1 to 9.0.0 aliscco/alisco-node#1375

Open

[Snyk] Security upgrade eslint from 7.32.0 to 9.0.0 aliscco/alisco-node#1376

Open

[Snyk] Security upgrade eslint from 5.16.0 to 9.0.0 aliscco/alisco-node#1377

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

stream: fix not calling cleanup() when unpiping all streams. #18266

stream: fix not calling cleanup() when unpiping all streams. #18266

MoonBall commented Jan 20, 2018 •

edited

lpinca Jan 20, 2018 •

edited

lpinca Jan 20, 2018

MoonBall Jan 20, 2018

MoonBall Jan 20, 2018

lpinca Jan 20, 2018

lpinca Jan 20, 2018

MoonBall Jan 20, 2018 •

edited

lpinca Jan 20, 2018

MoonBall Jan 20, 2018

targos commented Jan 20, 2018

apapirovski left a comment •

edited

mcollina left a comment

MoonBall commented Jan 20, 2018 •

edited

lpinca Jan 21, 2018

MoonBall Jan 22, 2018

lpinca Jan 21, 2018

MoonBall Jan 22, 2018

BridgeAR commented Feb 1, 2018

addaleax commented Feb 4, 2018

MoonBall commented Feb 4, 2018 •

edited

mcollina commented Feb 5, 2018

stream: fix not calling cleanup() when unpiping all streams. #18266

stream: fix not calling cleanup() when unpiping all streams. #18266

Conversation

MoonBall commented Jan 20, 2018 • edited

Checklist

Affected core subsystem(s)

lpinca Jan 20, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MoonBall Jan 20, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

targos commented Jan 20, 2018

apapirovski left a comment • edited

Choose a reason for hiding this comment

mcollina left a comment

Choose a reason for hiding this comment

MoonBall commented Jan 20, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BridgeAR commented Feb 1, 2018

addaleax commented Feb 4, 2018

MoonBall commented Feb 4, 2018 • edited

mcollina commented Feb 5, 2018

MoonBall commented Jan 20, 2018 •

edited

lpinca Jan 20, 2018 •

edited

MoonBall Jan 20, 2018 •

edited

apapirovski left a comment •

edited

MoonBall commented Jan 20, 2018 •

edited

MoonBall commented Feb 4, 2018 •

edited