Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(@types/node): add stream/consumers #55311

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions types/node/index.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,7 @@
/// <reference path="repl.d.ts" />
/// <reference path="stream.d.ts" />
/// <reference path="stream/promises.d.ts" />
/// <reference path="stream/consumers.d.ts" />
/// <reference path="stream/web.d.ts" />
/// <reference path="string_decoder.d.ts" />
/// <reference path="timers.d.ts" />
Expand Down
2 changes: 2 additions & 0 deletions types/node/stream.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
declare module 'stream' {
import { EventEmitter, Abortable } from 'node:events';
import * as streamPromises from 'node:stream/promises';
import * as streamConsumers from 'node:stream/consumers';
class internal extends EventEmitter {
pipe<T extends NodeJS.WritableStream>(
destination: T,
Expand Down Expand Up @@ -1169,6 +1170,7 @@ declare module 'stream' {
unref(): void;
}
const promises: typeof streamPromises;
const consumers: typeof streamConsumers;
}
export = internal;
}
Expand Down
25 changes: 25 additions & 0 deletions types/node/stream/consumers.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
// Duplicates of interface in lib.dom.ts.
// Duplicated here rather than referencing lib.dom.ts because doing so causes lib.dom.ts to be loaded for "test-all"
// Which in turn causes tests to pass that shouldn't pass.
//
// This interface is not, and should not be, exported.
interface Blob {
readonly size: number;
readonly type: string;
arrayBuffer(): Promise<ArrayBuffer>;
slice(start?: number, end?: number, contentType?: string): Blob;
stream(): NodeJS.ReadableStream;
text(): Promise<string>;
}
Comment on lines +5 to +13
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this comment is accurate? It looks like this interface is leaking into global definitions:

149748342-48857820-a09b-439a-a7c7-f954d02f1aab

I can't remember where I read it (maybe in a linter rule or something?), but I'm pretty sure all interfaces in a .d.ts file are exported, so that consumers have access to references to them. Very happy to be proven wrong on this point, though.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not only is that comment (about not being exported) inaccurate, the comment above it isn't accurate either. The interface is not a "duplicate" of Blob from lib.dom.ts, because the DOM Blob#stream method returns a W3C ReadableStream, which is not the same thing as a NodeJS.ReadableStream.

As you point out, simply having @types/node in your node_modules pollutes frontend code with this Node-specific global type definition, which is merged with the real (DOM) Blob definition. As a result, I've had to any-cast the return value whenever I call .stream().

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good news, the correct types have been added in the interim so this erroneous "duplicate" is no longer needed. I just opened #59905 which I think will resolve this error.

Copy link
Contributor Author

@favna favna Apr 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@thw0rted With the benefit of the doubt I will think you didn't mean it as such but I feel called out here anyway, and so I feel a need to defend myself.

I specifically did not add myself to the contributors list for node types because I have no interest in being pinged for reviews and I have hardly any interest in Node's types to begin with as long as they work, and this has for many months.

Furthermore, even though yes it might be wrong that I made this change, that change was made in a pull request just like your own. That means it was reviewed and it was merged by a DT maintainer. Simply denotsting that I did bad here is therefore coming over as shallow because clearly other people were involved as well.

Lastly, as I have also stated elsewhere, this review comment was made after the pull request was already merged. It was therefore quite obviously never taken into full consideration and mostly completely dismissed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right that I didn't mean it as a personal attack -- reading back through my comment I can definitely see how you'd take it that way, and I'm sorry. (I can sympathize about not signing yourself up as a contributor, the scope is unlike anything else on DT!)

The situation is very confusing, and I don't know that anybody can really be blamed directly -- Node chose to add this partial implementation of some web APIs, and worse they're reusing web class names for incomplete implementations that aren't available at the same scope. When you made your PR you didn't even have a "Node Blob" to reference, so your approach was totally reasonable; bleeding into global was an honest mistake.

To make matters worse, I don't see any way to write a test (or I guess a "meta" test?) that would prevent this. Maybe the package.json or tsconfig file could have a field with a list of globals introduced by the package, and if anything else bleeds out, the test would fail? I don't really know enough about TS internals to suggest how to implement something like that but it might be worth kicking the idea around in Discussions -- I'll take a look.


declare module 'stream/consumers' {
import { Readable } from 'node:stream';
function buffer(stream: NodeJS.ReadableStream | Readable | AsyncIterator<any>): Promise<Buffer>;
function text(stream: NodeJS.ReadableStream | Readable | AsyncIterator<any>): Promise<string>;
function arrayBuffer(stream: NodeJS.ReadableStream | Readable | AsyncIterator<any>): Promise<ArrayBuffer>;
function blob(stream: NodeJS.ReadableStream | Readable | AsyncIterator<any>): Promise<Blob>;
function json(stream: NodeJS.ReadableStream | Readable | AsyncIterator<any>): Promise<unknown>;
}
declare module 'node:stream/consumers' {
export * from 'stream/consumers';
}
22 changes: 22 additions & 0 deletions types/node/test/stream.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import { createReadStream, createWriteStream } from 'node:fs';
import { createGzip, constants } from 'node:zlib';
import assert = require('node:assert');
import { Http2ServerResponse } from 'node:http2';
import { text, json, buffer } from 'node:stream/consumers';
import { pipeline as pipelinePromise } from 'node:stream/promises';
import { stdout } from 'node:process';
import 'node:stream/web';
Expand Down Expand Up @@ -455,6 +456,27 @@ async function streamPipelineAsyncPromiseAbortTransform() {
});
}

async function readableToString() {
const r = createReadStream('file.txt');

// $ExpectType string
await text(r);
}

async function readableToJson() {
const r = createReadStream('file.txt');

// $ExpectType unknown
await json(r);
}

async function readableToBuffer() {
const r = createReadStream('file.txt');

// $ExpectType Buffer
await buffer(r);
}

// http://nodejs.org/api/stream.html#stream_readable_pipe_destination_options
function stream_readable_pipe_test() {
const rs = createReadStream(Buffer.from('file.txt'));
Expand Down