Note how to defer file descriptor creation/use a stream Ctor #454
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When appending many files and creating readstreams for each one, you run the real risk of blowing you file descriptor limit. Generally its better to defer the creation of readStreams until you need to read the file.
Reading into the combined stream docs, it appeared that this would support the `(next) => next(fs.createReadStream('foo')` api, however its not quite that simple. This module expects a `path` property on the value so I documented that expectation.
There is probably room for improvement here, API wise, however this should help assist people from having to read too much code to achieve this outcome.
Thank you for and open to any suggestions.
bcomnes
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When appending many files and creating readstreams for each one, you run the real risk of blowing you file descriptor limit. Generally its better to defer the creation of readStreams until you need to read the file.
Reading into the combined stream docs, it appeared that this would support the
(next) => next(fs.createReadStream('foo'))api, however its not quite that simple. This module expects apathproperty on the value so I documented that expectation.There is probably room for improvement here, API wise, however this should help assist people from having to read too much code to achieve this outcome.
Thank you for and open to any suggestions.