fs: implement byob mode for readableWebStream() #46933

debadree25 · 2023-03-03T11:08:18Z

~~This is an initial draft~~ trying to implement allowing usage of 'byob' mode for readable web streams created from file handles.

~~Would like some feedback before I start working on tests :-)~~

Have added tests and doc updates,
the implementation is similar to how the implementation is here https://nodejs.org/api/webstreams.html#class-readablestreambyobreader

Fixes: #45853

Fixes: nodejs#45853

debadree25 · 2023-03-07T17:32:05Z

also cc'ing @nodejs/whatwg-stream

debadree25 · 2023-03-16T10:25:13Z

Ping @nodejs/whatwg-stream , @nodejs/streams ?

nodejs-github-bot · 2023-03-16T10:26:47Z

CI: https://ci.nodejs.org/job/node-test-pull-request/50396/

debadree25 · 2023-03-29T18:44:33Z

no one to review? 😕
any direction on whether this direction is appropriate or not would also be helpful!

benjamingr · 2023-04-08T12:44:01Z

no one to review? 😕 any direction on whether this direction is appropriate or not would also be helpful!

I suspect many people (at least me) are in spring break holidays Mar/Apr (easter/passover/holi/nowruz/songkram etc)

Also, web streams typically don't get a ton of love (that makes your PRs more welcome not less! And it's good that you're tackling them) but most of the node/streams people (at least matteo, ronag and myself) aren't web-stream experts.

debadree25 · 2023-04-08T14:35:00Z

Hey @benjamingr yeah i also thought people are probably busy rn! no problem at all I will leave the PR open for the time being no hurries 😄😄!

benjamingr

Ah this just implements the ByoB stuff for fs on top of web streams - that should be fine and much smaller to review

Notable changes: crypto: * update root certificates to NSS 3.89 (Node.js GitHub Bot) #47659 dns: * (SEMVER-MINOR) expose getDefaultResultOrder (btea) #46973 doc: * add ovflowd to collaborators (Claudio Wunder) #47844 * add KhafraDev to collaborators (Matthew Aitken) #47510 fs: * (SEMVER-MINOR) add support for mode flag to specify the copy behavior (Tetsuharu Ohzeki) #47084 * (SEMVER-MINOR) add recursive option to readdir and opendir (Ethan Arrowood) #41439 * (SEMVER-MINOR) add support for mode flag to specify the copy behavior (Tetsuharu Ohzeki) #47084 * (SEMVER-MINOR) implement byob mode for readableWebStream() (Debadree Chatterjee) #46933 http: * (SEMVER-MINOR) prevent writing to the body when not allowed by HTTP spec (Gerrard Lindsay) #47732 * (SEMVER-MINOR) remove internal error in assignSocket (Matteo Collina) #47723 * (SEMVER-MINOR) add highWaterMark opt in http.createServer (HinataKah0) #47405 module: * change default resolver to not throw on unknown scheme (Gil Tayar) #47824 node-api: * (SEMVER-MINOR) define version 9 (Chengzhong Wu) #48151 * (SEMVER-MINOR) deprecate napi_module_register (Vladimir Morozov) #46319 stream: * (SEMVER-MINOR) preserve object mode in compose (Raz Luvaton) #47413 * (SEMVER-MINOR) add setter & getter for default highWaterMark (#46929) (Robert Nagy) #46929 test: * unflake test-vm-timeout-escape-nexttick (Santiago Gimeno) #48078 test_runner: * (SEMVER-MINOR) add shorthands to `test` (Chemi Atlow) #47909 * (SEMVER-MINOR) support combining coverage reports (Colin Ihrig) #47686 * (SEMVER-MINOR) execute before hook on test (Chemi Atlow) #47586 * (SEMVER-MINOR) expose reporter for use in run api (Chemi Atlow) #47238 tools: * update LICENSE and license-builder.sh (Santiago Gimeno) #48078 wasi: * (SEMVER-MINOR) no longer require flag to enable wasi (Michael Dawson) #47286 PR-URL: TODO

Notable changes: crypto: * update root certificates to NSS 3.89 (Node.js GitHub Bot) #47659 dns: * (SEMVER-MINOR) expose getDefaultResultOrder (btea) #46973 doc: * add ovflowd to collaborators (Claudio Wunder) #47844 * add KhafraDev to collaborators (Matthew Aitken) #47510 fs: * (SEMVER-MINOR) add support for mode flag to specify the copy behavior (Tetsuharu Ohzeki) #47084 * (SEMVER-MINOR) add recursive option to readdir and opendir (Ethan Arrowood) #41439 * (SEMVER-MINOR) add support for mode flag to specify the copy behavior (Tetsuharu Ohzeki) #47084 * (SEMVER-MINOR) implement byob mode for readableWebStream() (Debadree Chatterjee) #46933 http: * (SEMVER-MINOR) prevent writing to the body when not allowed by HTTP spec (Gerrard Lindsay) #47732 * (SEMVER-MINOR) remove internal error in assignSocket (Matteo Collina) #47723 * (SEMVER-MINOR) add highWaterMark opt in http.createServer (HinataKah0) #47405 module: * change default resolver to not throw on unknown scheme (Gil Tayar) #47824 node-api: * (SEMVER-MINOR) define version 9 (Chengzhong Wu) #48151 * (SEMVER-MINOR) deprecate napi_module_register (Vladimir Morozov) #46319 stream: * (SEMVER-MINOR) preserve object mode in compose (Raz Luvaton) #47413 * (SEMVER-MINOR) add setter & getter for default highWaterMark (#46929) (Robert Nagy) #46929 test: * unflake test-vm-timeout-escape-nexttick (Santiago Gimeno) #48078 test_runner: * (SEMVER-MINOR) add shorthands to `test` (Chemi Atlow) #47909 * (SEMVER-MINOR) support combining coverage reports (Colin Ihrig) #47686 * (SEMVER-MINOR) execute before hook on test (Chemi Atlow) #47586 * (SEMVER-MINOR) expose reporter for use in run api (Chemi Atlow) #47238 tools: * update LICENSE and license-builder.sh (Santiago Gimeno) #48078 wasi: * (SEMVER-MINOR) no longer require flag to enable wasi (Michael Dawson) #47286 PR-URL: #48694

Notable changes: crypto: * update root certificates to NSS 3.89 (Node.js GitHub Bot) #47659 deps: * update ada to 2.0.0 (Node.js GitHub Bot) #47339 dns: * (SEMVER-MINOR) expose getDefaultResultOrder (btea) #46973 doc: * add ovflowd to collaborators (Claudio Wunder) #47844 * add KhafraDev to collaborators (Matthew Aitken) #47510 fs: * (SEMVER-MINOR) add support for mode flag to specify the copy behavior (Tetsuharu Ohzeki) #47084 * (SEMVER-MINOR) add recursive option to readdir and opendir (Ethan Arrowood) #41439 * (SEMVER-MINOR) add support for mode flag to specify the copy behavior (Tetsuharu Ohzeki) #47084 * (SEMVER-MINOR) implement byob mode for readableWebStream() (Debadree Chatterjee) #46933 http: * (SEMVER-MINOR) prevent writing to the body when not allowed by HTTP spec (Gerrard Lindsay) #47732 * (SEMVER-MINOR) remove internal error in assignSocket (Matteo Collina) #47723 * (SEMVER-MINOR) add highWaterMark opt in http.createServer (HinataKah0) #47405 module: * change default resolver to not throw on unknown scheme (Gil Tayar) #47824 node-api: * (SEMVER-MINOR) define version 9 (Chengzhong Wu) #48151 * (SEMVER-MINOR) deprecate napi_module_register (Vladimir Morozov) #46319 stream: * (SEMVER-MINOR) preserve object mode in compose (Raz Luvaton) #47413 * (SEMVER-MINOR) add setter & getter for default highWaterMark (#46929) (Robert Nagy) #46929 test: * unflake test-vm-timeout-escape-nexttick (Santiago Gimeno) #48078 test_runner: * (SEMVER-MINOR) add shorthands to `test` (Chemi Atlow) #47909 * (SEMVER-MINOR) support combining coverage reports (Colin Ihrig) #47686 * (SEMVER-MINOR) execute before hook on test (Chemi Atlow) #47586 * (SEMVER-MINOR) expose reporter for use in run api (Chemi Atlow) #47238 tools: * update LICENSE and license-builder.sh (Santiago Gimeno) #48078 url: * drop ICU requirement for parsing hostnames (Yagiz Nizipli) #47339 * use ada::url_aggregator for parsing urls (Yagiz Nizipli) #47339 * (SEMVER-MINOR) implement URL.canParse (Matthew Aitken) #47179 wasi: * (SEMVER-MINOR) no longer require flag to enable wasi (Michael Dawson) #47286 PR-URL: #48694

Notable changes: crypto: * update root certificates to NSS 3.89 (Node.js GitHub Bot) #47659 * use WebIDL converters in WebCryptoAPI (Filip Skokan) #46067 deps: * update ada to 2.0.0 (Node.js GitHub Bot) #47339 dns: * (SEMVER-MINOR) expose getDefaultResultOrder (btea) #46973 doc: * add ovflowd to collaborators (Claudio Wunder) #47844 * add KhafraDev to collaborators (Matthew Aitken) #47510 * events: * (SEMVER-MINOR) add getMaxListeners method (Matthew Aitken) #47039 fs: * (SEMVER-MINOR) add support for mode flag to specify the copy behavior (Tetsuharu Ohzeki) #47084 * (SEMVER-MINOR) add recursive option to readdir and opendir (Ethan Arrowood) #41439 * (SEMVER-MINOR) add support for mode flag to specify the copy behavior (Tetsuharu Ohzeki) #47084 * (SEMVER-MINOR) implement byob mode for readableWebStream() (Debadree Chatterjee) #46933 http: * (SEMVER-MINOR) prevent writing to the body when not allowed by HTTP spec (Gerrard Lindsay) #47732 * (SEMVER-MINOR) remove internal error in assignSocket (Matteo Collina) #47723 * (SEMVER-MINOR) add highWaterMark opt in http.createServer (HinataKah0) #47405 lib: * (SEMVER-MINOR) add webstreams to Duplex.from() (Debadree Chatterjee) #46190 module: * change default resolver to not throw on unknown scheme (Gil Tayar) #47824 node-api: * (SEMVER-MINOR) define version 9 (Chengzhong Wu) #48151 * (SEMVER-MINOR) deprecate napi_module_register (Vladimir Morozov) #46319 stream: * (SEMVER-MINOR) preserve object mode in compose (Raz Luvaton) #47413 * (SEMVER-MINOR) add setter & getter for default highWaterMark (#46929) (Robert Nagy) #46929 test: * unflake test-vm-timeout-escape-nexttick (Santiago Gimeno) #48078 test_runner: * (SEMVER-MINOR) add shorthands to `test` (Chemi Atlow) #47909 * (SEMVER-MINOR) support combining coverage reports (Colin Ihrig) #47686 * (SEMVER-MINOR) execute before hook on test (Chemi Atlow) #47586 * (SEMVER-MINOR) expose reporter for use in run api (Chemi Atlow) #47238 tools: * update LICENSE and license-builder.sh (Santiago Gimeno) #48078 url: * drop ICU requirement for parsing hostnames (Yagiz Nizipli) #47339 * use ada::url_aggregator for parsing urls (Yagiz Nizipli) #47339 * (SEMVER-MINOR) implement URL.canParse (Matthew Aitken) #47179 wasi: * (SEMVER-MINOR) no longer require flag to enable wasi (Michael Dawson) #47286 PR-URL: #48694

Notable changes: Ada 2.0 Node.js v18.17.0 comes with the latest version of the URL parser, Ada. This update brings significant performance improvements to URL parsing, including enhancements to the url.domainToASCII and url.domainToUnicode functions in node:url. Ada 2.0 has been integrated into the Node.js codebase, ensuring that all parts of the application can benefit from the improved performance. Additionally, Ada 2.0 features a significant performance boost over its predecessor, Ada 1.0.4, while also eliminating the need for the ICU requirement for URL hostname parsing. Contributed by Yagiz Nizipli and Daniel Lemire in #47339 Web Crypto API Web Crypto API functions' arguments are now coerced and validated as per their WebIDL definitions like in other Web Crypto API implementations. This further improves interoperability with other implementations of Web Crypto API. Contributed by Filip Skokan in #46067 crypto: * update root certificates to NSS 3.89 (Node.js GitHub Bot) #47659 dns: * (SEMVER-MINOR) expose getDefaultResultOrder (btea) #46973 doc: * add ovflowd to collaborators (Claudio Wunder) #47844 * add KhafraDev to collaborators (Matthew Aitken) #47510 * events: * (SEMVER-MINOR) add getMaxListeners method (Matthew Aitken) #47039 fs: * (SEMVER-MINOR) add support for mode flag to specify the copy behavior (Tetsuharu Ohzeki) #47084 * (SEMVER-MINOR) add recursive option to readdir and opendir (Ethan Arrowood) #41439 * (SEMVER-MINOR) add support for mode flag to specify the copy behavior (Tetsuharu Ohzeki) #47084 * (SEMVER-MINOR) implement byob mode for readableWebStream() (Debadree Chatterjee) #46933 http: * (SEMVER-MINOR) prevent writing to the body when not allowed by HTTP spec (Gerrard Lindsay) #47732 * (SEMVER-MINOR) remove internal error in assignSocket (Matteo Collina) #47723 * (SEMVER-MINOR) add highWaterMark opt in http.createServer (HinataKah0) #47405 lib: * (SEMVER-MINOR) add webstreams to Duplex.from() (Debadree Chatterjee) #46190 module: * change default resolver to not throw on unknown scheme (Gil Tayar) #47824 node-api: * (SEMVER-MINOR) define version 9 (Chengzhong Wu) #48151 * (SEMVER-MINOR) deprecate napi_module_register (Vladimir Morozov) #46319 stream: * (SEMVER-MINOR) preserve object mode in compose (Raz Luvaton) #47413 * (SEMVER-MINOR) add setter & getter for default highWaterMark (#46929) (Robert Nagy) #46929 test: * unflake test-vm-timeout-escape-nexttick (Santiago Gimeno) #48078 test_runner: * (SEMVER-MINOR) add shorthands to `test` (Chemi Atlow) #47909 * (SEMVER-MINOR) support combining coverage reports (Colin Ihrig) #47686 * (SEMVER-MINOR) execute before hook on test (Chemi Atlow) #47586 * (SEMVER-MINOR) expose reporter for use in run api (Chemi Atlow) #47238 tools: * update LICENSE and license-builder.sh (Santiago Gimeno) #48078 url: * (SEMVER-MINOR) implement URL.canParse (Matthew Aitken) #47179 wasi: * (SEMVER-MINOR) no longer require flag to enable wasi (Michael Dawson) #47286 PR-URL: #48694

Notable changes: Ada 2.0 Node.js v18.17.0 comes with the latest version of the URL parser, Ada. This update brings significant performance improvements to URL parsing, including enhancements to the url.domainToASCII and url.domainToUnicode functions in node:url. Ada 2.0 has been integrated into the Node.js codebase, ensuring that all parts of the application can benefit from the improved performance. Additionally, Ada 2.0 features a significant performance boost over its predecessor, Ada 1.0.4, while also eliminating the need for the ICU requirement for URL hostname parsing. Contributed by Yagiz Nizipli and Daniel Lemire in #47339 Web Crypto API Web Crypto API functions' arguments are now coerced and validated as per their WebIDL definitions like in other Web Crypto API implementations. This further improves interoperability with other implementations of Web Crypto API. Contributed by Filip Skokan in #46067 crypto: * update root certificates to NSS 3.89 (Node.js GitHub Bot) #47659 dns: * (SEMVER-MINOR) expose getDefaultResultOrder (btea) #46973 doc: * add ovflowd to collaborators (Claudio Wunder) #47844 * add KhafraDev to collaborators (Matthew Aitken) #47510 * events: * (SEMVER-MINOR) add getMaxListeners method (Matthew Aitken) #47039 fs: * (SEMVER-MINOR) add support for mode flag to specify the copy behavior (Tetsuharu Ohzeki) #47084 * (SEMVER-MINOR) add recursive option to readdir and opendir (Ethan Arrowood) #41439 * (SEMVER-MINOR) add support for mode flag to specify the copy behavior (Tetsuharu Ohzeki) #47084 * (SEMVER-MINOR) implement byob mode for readableWebStream() (Debadree Chatterjee) #46933 http: * (SEMVER-MINOR) prevent writing to the body when not allowed by HTTP spec (Gerrard Lindsay) #47732 * (SEMVER-MINOR) remove internal error in assignSocket (Matteo Collina) #47723 * (SEMVER-MINOR) add highWaterMark opt in http.createServer (HinataKah0) #47405 lib: * (SEMVER-MINOR) add webstreams to Duplex.from() (Debadree Chatterjee) #46190 * (SEMVER-MINOR) implement AbortSignal.any() (Chemi Atlow) #47821 module: * change default resolver to not throw on unknown scheme (Gil Tayar) #47824 node-api: * (SEMVER-MINOR) define version 9 (Chengzhong Wu) #48151 * (SEMVER-MINOR) deprecate napi_module_register (Vladimir Morozov) #46319 stream: * (SEMVER-MINOR) preserve object mode in compose (Raz Luvaton) #47413 * (SEMVER-MINOR) add setter & getter for default highWaterMark (#46929) (Robert Nagy) #46929 test: * unflake test-vm-timeout-escape-nexttick (Santiago Gimeno) #48078 test_runner: * (SEMVER-MINOR) add shorthands to `test` (Chemi Atlow) #47909 * (SEMVER-MINOR) support combining coverage reports (Colin Ihrig) #47686 * (SEMVER-MINOR) execute before hook on test (Chemi Atlow) #47586 * (SEMVER-MINOR) expose reporter for use in run api (Chemi Atlow) #47238 tools: * update LICENSE and license-builder.sh (Santiago Gimeno) #48078 url: * (SEMVER-MINOR) implement URL.canParse (Matthew Aitken) #47179 wasi: * (SEMVER-MINOR) no longer require flag to enable wasi (Michael Dawson) #47286 PR-URL: #48694

JSDoc update was omitted during code implementation Refs: nodejs#46933

JSDoc comment did not match the actual implementation of `FileHandle#readableWebStream`. Refs: #46933 PR-URL: #48853 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Deokjin Kim <deokjin81.kim@gmail.com>

JSDoc comment did not match the actual implementation of `FileHandle#readableWebStream`. Refs: nodejs#46933 PR-URL: nodejs#48853 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Deokjin Kim <deokjin81.kim@gmail.com>

Notable changes: Ada 2.0 Node.js v18.17.0 comes with the latest version of the URL parser, Ada. This update brings significant performance improvements to URL parsing, including enhancements to the url.domainToASCII and url.domainToUnicode functions in node:url. Ada 2.0 has been integrated into the Node.js codebase, ensuring that all parts of the application can benefit from the improved performance. Additionally, Ada 2.0 features a significant performance boost over its predecessor, Ada 1.0.4, while also eliminating the need for the ICU requirement for URL hostname parsing. Contributed by Yagiz Nizipli and Daniel Lemire in nodejs#47339 Web Crypto API Web Crypto API functions' arguments are now coerced and validated as per their WebIDL definitions like in other Web Crypto API implementations. This further improves interoperability with other implementations of Web Crypto API. Contributed by Filip Skokan in nodejs#46067 crypto: * update root certificates to NSS 3.89 (Node.js GitHub Bot) nodejs#47659 dns: * (SEMVER-MINOR) expose getDefaultResultOrder (btea) nodejs#46973 doc: * add ovflowd to collaborators (Claudio Wunder) nodejs#47844 * add KhafraDev to collaborators (Matthew Aitken) nodejs#47510 * events: * (SEMVER-MINOR) add getMaxListeners method (Matthew Aitken) nodejs#47039 fs: * (SEMVER-MINOR) add support for mode flag to specify the copy behavior (Tetsuharu Ohzeki) nodejs#47084 * (SEMVER-MINOR) add recursive option to readdir and opendir (Ethan Arrowood) nodejs#41439 * (SEMVER-MINOR) add support for mode flag to specify the copy behavior (Tetsuharu Ohzeki) nodejs#47084 * (SEMVER-MINOR) implement byob mode for readableWebStream() (Debadree Chatterjee) nodejs#46933 http: * (SEMVER-MINOR) prevent writing to the body when not allowed by HTTP spec (Gerrard Lindsay) nodejs#47732 * (SEMVER-MINOR) remove internal error in assignSocket (Matteo Collina) nodejs#47723 * (SEMVER-MINOR) add highWaterMark opt in http.createServer (HinataKah0) nodejs#47405 lib: * (SEMVER-MINOR) add webstreams to Duplex.from() (Debadree Chatterjee) nodejs#46190 * (SEMVER-MINOR) implement AbortSignal.any() (Chemi Atlow) nodejs#47821 module: * change default resolver to not throw on unknown scheme (Gil Tayar) nodejs#47824 node-api: * (SEMVER-MINOR) define version 9 (Chengzhong Wu) nodejs#48151 * (SEMVER-MINOR) deprecate napi_module_register (Vladimir Morozov) nodejs#46319 stream: * (SEMVER-MINOR) preserve object mode in compose (Raz Luvaton) nodejs#47413 * (SEMVER-MINOR) add setter & getter for default highWaterMark (nodejs#46929) (Robert Nagy) nodejs#46929 test: * unflake test-vm-timeout-escape-nexttick (Santiago Gimeno) nodejs#48078 test_runner: * (SEMVER-MINOR) add shorthands to `test` (Chemi Atlow) nodejs#47909 * (SEMVER-MINOR) support combining coverage reports (Colin Ihrig) nodejs#47686 * (SEMVER-MINOR) execute before hook on test (Chemi Atlow) nodejs#47586 * (SEMVER-MINOR) expose reporter for use in run api (Chemi Atlow) nodejs#47238 tools: * update LICENSE and license-builder.sh (Santiago Gimeno) nodejs#48078 url: * (SEMVER-MINOR) implement URL.canParse (Matthew Aitken) nodejs#47179 wasi: * (SEMVER-MINOR) no longer require flag to enable wasi (Michael Dawson) nodejs#47286 PR-URL: nodejs#48694