Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add .all property with interleaved stdout and stderr #171

Merged
merged 19 commits into from Mar 10, 2019
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
6c46865
Added result.all intermixed stdout/stderr (concat for sync)
tomsotte Jan 20, 2019
4069b17
Removed `all` stream from execa.sync(); fix creating mixed stream for…
tomsotte Mar 7, 2019
97cde71
Added `all` stream documentation and why it's not possible in execa.s…
tomsotte Mar 7, 2019
23799a0
More predictable noop-132 fixture for testing `all` stream output
tomsotte Mar 7, 2019
40aa251
Merge with upstream/master; fix rename `m` -> `execa` in test.js
tomsotte Mar 8, 2019
3832328
Merge branch 'master' into iss1
tomsotte Mar 8, 2019
e35dcdf
Fixed trailing space
tomsotte Mar 8, 2019
7f430d9
Using test.serial for testing result.all with lower timeout for noop-…
tomsotte Mar 9, 2019
b500215
Added execa.all(), similar to execa.stdout() for result.all; added test
tomsotte Mar 9, 2019
a419b4a
Fixed link to Node.js docs about process I/O stream
tomsotte Mar 9, 2019
bc1450a
all available on errors, if available in result (ex. no execa.sync())
tomsotte Mar 9, 2019
29f3f12
Added all interleaved stream to readme in Why, example and API
tomsotte Mar 9, 2019
43a43bd
Removed unnecessary comments on noop-132 and test.js
tomsotte Mar 10, 2019
dc68492
Fixed and improved readme about all interleaved stream
tomsotte Mar 10, 2019
360bdb4
Removed execa.all()
tomsotte Mar 10, 2019
06fbfff
Update readme.md
sindresorhus Mar 10, 2019
e128464
Update readme.md
sindresorhus Mar 10, 2019
e842c72
Merge branch 'master' into iss1
sindresorhus Mar 10, 2019
b815a79
Update readme.md
sindresorhus Mar 10, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
14 changes: 14 additions & 0 deletions fixtures/noop-132
@@ -0,0 +1,14 @@
#!/usr/bin/env node
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not test interleaving because stderr is not in-between two stdout calls.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was my mistake, I should have left it in-between and let the test fail. As I said in the comments I was testing various way to implement the test to be correct but I couldn't find one for all (more on this on the next conversation).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also I think it might be a good idea to do a more complex interleaving like stdout > stderr > stderr > stdout > stderr > stdout

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it needs to be more complex, if it works just once we should expect to interleave the rest of the outputs. Otherwise we know it's just concatenating the output strings.
From what I understand by the tests I'm doing is that we cannot guarantee the order of the interleaving.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think @sindresorhus?

'use strict';
// Interleaving the output of stdout and stderr is unpredictable and can't be
// expressed as a test. What I've tried so far to make it predictable:
// - process.stdout._handle.setBlocking(true)
// - process.stdout.once('drain', () => process.stderr.write('2'));
// - process.stdout.write('1', () => process.stderr.write('2'));
// - await delay() between calls; it works but with varying ms, based on load
ehmicky marked this conversation as resolved.
Show resolved Hide resolved
// - console.log/error
tomsotte marked this conversation as resolved.
Show resolved Hide resolved

process.stdout.write('1');
process.stdout.write('3');
// Ideally, to demonstrate the interleaving, the next line should go before '3'
process.stderr.write('2');
35 changes: 34 additions & 1 deletion index.js
Expand Up @@ -6,6 +6,7 @@ const stripFinalNewline = require('strip-final-newline');
const npmRunPath = require('npm-run-path');
const isStream = require('is-stream');
const _getStream = require('get-stream');
const mergeStream = require('merge-stream');
const pFinally = require('p-finally');
const onExit = require('signal-exit');
const errname = require('./lib/errname');
Expand Down Expand Up @@ -122,6 +123,24 @@ function handleShell(fn, command, options) {
return fn(file, args, options);
}

function makeAllStream(spawned) {
const mixed = mergeStream();
tomsotte marked this conversation as resolved.
Show resolved Hide resolved

if (!spawned.stdout && !spawned.stderr) {
return null;
}

if (spawned.stdout) {
mixed.add(spawned.stdout);
}

if (spawned.stderr) {
mixed.add(spawned.stderr);
}

return mixed;
}

function getStream(process, stream, {encoding, buffer, maxBuffer}) {
if (!process[stream]) {
return null;
Expand Down Expand Up @@ -268,16 +287,22 @@ module.exports = (command, args, options) => {
if (spawned.stderr) {
spawned.stderr.destroy();
}

if (spawned.all) {
spawned.all.destroy();
}
}

const handlePromise = () => pFinally(Promise.all([
processDone,
getStream(spawned, 'stdout', {encoding, buffer, maxBuffer}),
getStream(spawned, 'stderr', {encoding, buffer, maxBuffer})
getStream(spawned, 'stderr', {encoding, buffer, maxBuffer}),
getStream(spawned, 'all', {encoding, buffer, maxBuffer: maxBuffer * 2})
]).then(arr => {
const result = arr[0];
result.stdout = arr[1];
result.stderr = arr[2];
result.all = arr[3];

if (result.error || result.code !== 0 || result.signal !== null) {
const error = makeError(result, {
Expand All @@ -301,6 +326,7 @@ module.exports = (command, args, options) => {
return {
stdout: handleOutput(parsed.options, result.stdout),
stderr: handleOutput(parsed.options, result.stderr),
all: handleOutput(parsed.options, result.all),
code: 0,
failed: false,
killed: false,
Expand All @@ -314,6 +340,8 @@ module.exports = (command, args, options) => {

handleInput(spawned, parsed.options.input);

spawned.all = makeAllStream(spawned);

spawned.then = (onfulfilled, onrejected) => handlePromise().then(onfulfilled, onrejected);
spawned.catch = onrejected => handlePromise().catch(onrejected);

Expand All @@ -339,6 +367,10 @@ module.exports.sync = (command, args, options) => {
const result = childProcess.spawnSync(parsed.command, parsed.args, parsed.options);
result.code = result.status;

// `spawnSync` doesn't expose the stdout/stderr before terminating, which means
ehmicky marked this conversation as resolved.
Show resolved Hide resolved
// the streams can't be merged unless proxying on `options.stdio`
result.all = result.stdout + result.stderr;

if (result.error || result.status !== 0 || result.signal !== null) {
const error = makeError(result, {
joinedCommand,
Expand All @@ -355,6 +387,7 @@ module.exports.sync = (command, args, options) => {
return {
stdout: handleOutput(parsed.options, result.stdout),
stderr: handleOutput(parsed.options, result.stderr),
all: handleOutput(parsed.options, result.all),
code: 0,
failed: false,
signal: null,
Expand Down
1 change: 1 addition & 0 deletions package.json
Expand Up @@ -40,6 +40,7 @@
"cross-spawn": "^6.0.0",
"get-stream": "^4.0.0",
"is-stream": "^1.1.0",
"merge-stream": "1.0.1",
"npm-run-path": "^2.0.0",
"p-finally": "^1.0.0",
"signal-exit": "^3.0.0",
Expand Down
19 changes: 17 additions & 2 deletions test.js
Expand Up @@ -41,6 +41,19 @@ test('execa.stderr()', async t => {
t.is(stderr, 'foo');
});

test('result.all shows both `stdout` and `stderr` intermixed', async t => {
const result = await m('noop-132');
// Due to the async nature of process.stdout/stderr on POSIX, this test
// is very unpredictable, although it should interleave the streams
// https://nodejs.org/api/process.html#process_a_note_on_process_i_os
ehmicky marked this conversation as resolved.
Show resolved Hide resolved
t.is(result.all, '132');
});

test('result.all shows both `stdout` and `stderr` concatenated - sync', t => {
const result = m.sync('noop-132');
t.is(result.all, '132');
});

test('stdout/stderr available on errors', async t => {
const err = await t.throws(m('exit', ['2']));
t.is(typeof err.stdout, 'string');
Expand Down Expand Up @@ -243,7 +256,8 @@ test('do not buffer stdout when `buffer` set to `false`', async t => {
const promise = m('max-buffer', ['stdout', '10'], {buffer: false});
const [result, stdout] = await Promise.all([
promise,
getStream(promise.stdout)
getStream(promise.stdout),
getStream(promise.all)
]);

t.is(result.stdout, undefined);
Expand All @@ -254,7 +268,8 @@ test('do not buffer stderr when `buffer` set to `false`', async t => {
const promise = m('max-buffer', ['stderr', '10'], {buffer: false});
const [result, stderr] = await Promise.all([
promise,
getStream(promise.stderr)
getStream(promise.stderr),
getStream(promise.all)
]);

t.is(result.stderr, undefined);
Expand Down