Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[x] stream: eof & pipeline compat #29724

Closed
wants to merge 5 commits into from
Closed

Conversation

ronag
Copy link
Member

@ronag ronag commented Sep 26, 2019

Some broken streams might emit 'close' before 'end' or 'finish'. However, if they have actually been ended or finished, this should not result in a premature close error.

This fixes #29699 while still maintaining what could be considered correct behaviour.

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

@ronag
Copy link
Member Author

ronag commented Sep 26, 2019

ping @mafintosh

@lpinca lpinca added the stream Issues and PRs related to the stream subsystem. label Sep 26, 2019
@ronag ronag force-pushed the fix-eof-compat branch 4 times, most recently from 617e410 to ec1fb21 Compare September 26, 2019 19:00
@ronag ronag force-pushed the fix-eof-compat branch 10 times, most recently from b1deba7 to 53e0fe6 Compare September 26, 2019 19:32
doc/api/stream.md Outdated Show resolved Hide resolved
@ronag ronag force-pushed the fix-eof-compat branch 5 times, most recently from 92d3138 to b2f52b5 Compare September 26, 2019 20:24
@ronag ronag changed the title stream: eof compat stream: eof & pipeline compat Sep 26, 2019
@ronag ronag force-pushed the fix-eof-compat branch 2 times, most recently from bbba57c to 9e9b654 Compare September 26, 2019 20:49
@ronag
Copy link
Member Author

ronag commented Dec 31, 2019

Is this still relevant? Been stale for a while...

Happy new year!

@ronag
Copy link
Member Author

ronag commented Jan 1, 2020

rebased to fix conflicts

@nodejs-github-bot
Copy link
Collaborator

@ronag
Copy link
Member Author

ronag commented Jan 1, 2020

CI

@nodejs-github-bot
Copy link
Collaborator

@ronag
Copy link
Member Author

ronag commented Jan 9, 2020

ping @mafintosh

Make stream.finished callback invoked if stream is already
closed/destroyed.
PR-URL: nodejs#29664
Reviewed-By: Minwoo Jung <minwoo@nodesource.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Some broken streams might emit 'close' before 'end' or 'finish'.
However, if they have actually been ended or finished, this
should not result in a premature close error.

Fixes: nodejs#29699
@ronag
Copy link
Member Author

ronag commented Jan 18, 2020

rebased to fix conflict

@ronag
Copy link
Member Author

ronag commented Jan 18, 2020

Turns out this actually had a little bit of an issue, the stream might still be in a pending state after destroyed so checking that is not enough.

@ronag ronag force-pushed the fix-eof-compat branch 3 times, most recently from a858773 to bfca3d1 Compare January 18, 2020 19:47
@mafintosh
Copy link
Member

This is still quite complex, but tried to do another pass at it. Can give an slight 👍. As mentioned in my comment I think it's worth investing in making a happy-path for when a stream is autoDestroy: true since that's much easier to reason about.

@ronag
Copy link
Member Author

ronag commented Jan 21, 2020

@mafintosh

but tried to do another pass at it. Can give an slight 👍.

Thx!

I think it's worth investing in making a happy-path for when a stream is autoDestroy: true since that's much easier to reason about.

I'm not quite sure I understand what you mean by this? Something like the following?

  if (autoDestroy && emitClose) {
    if (opts.error !== false) stream.on('error', onerror);

    function onclose() {
      callback.call(stream);
    }

    if (closed) {
      process.nextTick(callback.bind(stream));
    } else {
      stream.on('close', onclose);
    }

    return () => {
      stream.removeListener('close', onclose);
      stream.removeListener('error', onerror);
    };
  }

  // Rest is compat and edge cases...

@ronag
Copy link
Member Author

ronag commented Jan 25, 2020

My general feeling is that landing this safely is uncertain due to the complexity of the changes and uncertainty of reviewers.

There are actually several changes here that do not need to be directly related and could be split up.

I'm closing this in favour of smaller on more specific PR's. Please let me know if there are any objections to this.

@ronag ronag closed this Jan 25, 2020
@ronag ronag changed the title stream: eof & pipeline compat [x] stream: eof & pipeline compat Jan 25, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stream Issues and PRs related to the stream subsystem.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

stream.finished behaviour change
8 participants