Improve piping processes

[ehmicky]

Fixes #745
Fixes #753

This is a major improvement for piping multiple processes. Currently source.pipe(destination) is mostly just a shortcut to source.stdout.pipe(destination.stdin).

From looking at other projects, piping processes is usually implemented either like the above, or by relying on a shell like Bash. IMHO shells were initially designed to be run in an interactive terminal, not to be used in a programmatic context. They cannot easily provide with many features that are useful in a programmatic context. Such as, in the case of piping:

• full pipe information: what did the first process pass to the second?
• proper error handling: beyond just exit codes and printing stderr
• cancellation and dynamic piping: how to stop piping without termination the processes?

This PR adds the following features.

Bash convention

This follows the Bash convention when it comes to many aspects including: exit code, termination of processes, signals, closing streams, etc.

One important point is: process termination happens by closing stdout and stdin, not by sending termination signals. This is how Bash behaves. This allows processes to exit gracefully.

(Side note: SIGPIPE is not sent by the shell, but by the OS when the source process tries to write to the destination process after its stdin closed)

Error handling

Both processes are awaited. Previously, errors in the first process would crash the parent process.

Full pipe information

.pipe() resolves with the last process' result. Additionally, the result from the previous process in the pipe is available as result.pipedFrom[*]. This is recursive, so this works when piping a series of processes. This is available on errors too, which is useful for debugging.

Cancellation

An AbortSignal can be passed to the signal option. When aborted, the processes are unpiped. They are not terminated. This is useful for:

• Terminating one of the two processes, without terminating the other
• Dynamic piping: piping on/off any process to any other process

This uses merge-streams .remove() method under-the-hood.

1-n and n-1 piping

While piping one process to multiple processes (1-n) is simple, piping multiple processes to one process (n-1) is hard. This is because the last process must close the destination's stdin. There are several things that can happen to both stdout and stdin (ending too early, erroring, not being read/written to, being unpiped) and handling it correctly (propagating the error to users, allowing graceful exits, ensuring processes are not left hanging) is tricky.

With this PR, n-1 piping just works without user having to do anything, as Execa keeps track of which process is being piped, and uses merge-streams under-the-hood.

The tests are trying to run 100 processes at once, for both 1-n and n-1, and this works. I have tried up to 5000 processes at once locally, and it worked. After that, I started hitting the OS limits (pthread_create: Resource temporarily unavailable), although those can be configured. This does not leak resources, e.g. does not emit any maxListeners warnings.

Process cleanup

If the arguments/options are invalid, we close the processes' stdout/stdin to ensure they are not left hanging due to a programming mistake.

[sindresorhus]

This is a really good improvement 👍

[ehmicky]

Ready for another round of review. 👍