Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

child_process: add subprocess.readLines method #45774

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

aduh95
Copy link
Contributor

@aduh95 aduh95 commented Dec 7, 2022

Similarly to fileHandle.readLines(), iterating line by line over the output of a command is a somewhat regular thing when writing scripts. This PR adds a util method to cover this use case.

@nodejs-github-bot nodejs-github-bot added child_process Issues and PRs related to the child_process subsystem. needs-ci PRs that need a full CI run. readline Issues and PRs related to the built-in readline module. labels Dec 7, 2022
@addaleax
Copy link
Member

addaleax commented Dec 7, 2022

I'm very much surprised to see that #42590 was merged, despite the very valid discussion there ... can we not make the same mistake again?

If a utility method is useful, then I see no reason not to make it composable; e.g., a readLines(stream: Readable): AsyncIterable<string>. This could be used as for await (const line of readLines(subprocess.stdout)) { ... }, but, and this is the important bit, also with any other stream.

@aduh95
Copy link
Contributor Author

aduh95 commented Dec 7, 2022

This could be used as for await (const line of readLines(subprocess.stdout)) { ... }

You can already do that with import { createInterface as readLines } from 'node:readline'. However, with that you are still lacking support for catching an 'error' event on the ChildProcess instance, and the exit code not being 0.
My own opinion is that fileHandle.readLines is a great addition to the Node.js fs API, so I would very much be in favor of making the same mistake again, especially because I can't see a single downside to such mistake. That being said, if we can find a a better API for that, I'm open to suggestions.

@addaleax
Copy link
Member

addaleax commented Dec 8, 2022

However, with that you are still lacking support for catching an 'error' event on the ChildProcess instance,

For errors from spawning, this seems reasonable, for the other causes of ChildProcess errors (kill()ing the child process fails/sending messages to it fails), I would be surprised if a user expected that type of error to cause the async iterable to fail.

and the exit code not being 0.

Again… would you want this? Wouldn’t you be more likely to want to check the exit code yourself after the stream has finished out? It seems like a bit of a footgun that the async iterable can fail at the end of the stream.

making the same mistake again

To be clear, by “mistake” I was referring to merging a PR with outstanding discussion, even if not in the form of “Requested Changes”, in the way #42590 was merged.

this.on('error', () => {});
} else {
const errorPromise = new Promise((_, reject) => this.once('error', reject));
const exitPromise = new Promise((resolve, reject) => this.once('exit', (code) => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You would need to listen for close, not exit here, otherwise you’d run the risk of dropping data.

@aduh95
Copy link
Contributor Author

aduh95 commented Dec 8, 2022

For errors from spawning, this seems reasonable, for the other causes of ChildProcess errors (kill()ing the child process fails/sending messages to it fails), I would be surprised if a user expected that type of error to cause the async iterable to fail.
Again… would you want this? Wouldn’t you be more likely to want to check the exit code yourself after the stream has finished out? It seems like a bit of a footgun that the async iterable can fail at the end of the stream.

I guess in this case, folks should use readLines(cp.stdout) instead of cp.readLines(). The fact that it doesn't cover all the use cases doesn't strike me as a sign we should add this method. Worth noting that execa does consider a non-zero exit code as a failure or if it's killed by a signal, and given the popularity of this package (65M/week on npm), I think it's fair to assume it's a quite common use case and makes sense as a default.
This feature is designed to improve scripts authoring DX (same as fileHandle.readLines()), it may not make sense in a production context, but I think that's OK, people can still use the old APIs and/or use the options to dictate the API to behave appropriately.

To be clear, by “mistake” I was referring to merging a PR with outstanding discussion, even if not in the form of “Requested Changes”, in the way #42590 was merged.

Ah yes, sorry for the misunderstanding, in that case of course I agree.

@targos
Copy link
Member

targos commented Oct 2, 2023

=== release test-child-process-readLines ===
Path: parallel/test-child-process-readLines
Error: --- stderr ---
file:///home/runner/work/node/node/test/parallel/test-child-process-readLines.mjs:18
  for await (const line of spawn(process.execPath, ['-p', 42]).readLines()) {
                   ^

TypeError: Iterator result 42 is not an object
    at file:///home/runner/work/node/node/test/parallel/test-child-process-readLines.mjs:18:20
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Node.js v21.0.0-pre
Command: out/Release/node --test-reporter=spec --test-reporter-destination=stdout --test-reporter=./tools/github_reporter/index.js --test-reporter-destination=stdout /home/runner/work/node/node/test/parallel/test-child-process-readLines.mjs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
child_process Issues and PRs related to the child_process subsystem. needs-ci PRs that need a full CI run. readline Issues and PRs related to the built-in readline module.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants