Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarify: tee is unsafe for independent consumption since it backpressures to the faster consumer #1233

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

yonran
Copy link

@yonran yonran commented May 31, 2022

Unfortunately, the built-in tee algorithm backpressures to the faster consumer and enqueues chunks without limit to the slower consumer. Therefore, it is generally not safe to use when the two branches are read by independent consumers at different speed, since the consumer will eventually run out of memory.

I believe that this was an unfortunate design mistake because it conflicts with the suggestion in Stream requirements:. You must be able to pipe a stream to more than one writable stream. which said, “The tee stream can use a number of strategies to govern how the speed of its outputs affect the backpressure signals it gives off, but the simplest strategy is to pass aggregate backpressure signals directly up the chain, thus letting the speed of the slowest output determine the speed of the tee.” This also differs from analogous operations in other libraries such as repeated nodejs Readable.pipe or akka-streams Source.alsoTo which backpressure to the slower consumer and only require a bounded queue.

As I said in a node-fetch issue node-fetch/node-fetch#1568, I think this unbounded enqueueing algorithm of ReadableStream.tee() from #311 was a mistake. The built-in operations of a ReadableStream should only need a fixed-size queue. , but at this point it is too late to fix it.

But at least we can clarify the informational text, which implied that it is safe to tee() a stream and consume it independently. It is generally not safe to consume a tee()’d stream independently unless the difference in consumed data between the faster consumer and the slower consumer fits in memory.

  • At least two implementers are interested (and none opposed):
  • Tests are written and can be reviewed and commented upon at:
  • Implementation bugs are filed:

(See WHATWG Working Mode: Changes for more details.)


Preview | Diff

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant