Feature request: Allow response streams to be piped manually

[rageshkrishna]

I have a need to grab the passthrough server response stream and write it to a file. The goal is to be able to save matching responses to a file while also streaming it back to the client without adding any delays.

I'd like to propose supporting a new property in PassThroughHandlerOptions to enable this feature. Something like this:

server.forAnyRequest().thenPassThrough({
  onStreaming: (serverRes, clientRes, reqUrl) => {
    // The user _has_ to do this as a minimum. Maybe we could
    // be smarter about this and handle it in the request-handler
    // based on a return value from this callback.
    serverRes.pipe(clientRes);
  }
});

I'm completely new to this project and just starting to understand how the APIs are structured, so please let me know if this violates any of your design principles and if so, I'd appreciate some feedback on how to better design this. The property name and parameters are, of course, open to discussion. This is just something I prototyped in a few minutes to achieve my goal. If there's interest in enabling this feature, I'd be happy to work on a PR.

[pimterry]

As noted in #145, we should think how this combines with those changes. Also, I'm interested in ways we could potentially decouple this from thenPassThrough entirely. I think there's interesting use cases where you might well want to do this even in cases that aren't being proxied upstream.

One thing I've been meaning to look at is a restructing towards a steps-based model, instead of single handlers. In that approach, once a rule was matched (just like now) rather than running a single handler, instead of a series of steps would run, until a final step.

This opens up a lot more composability, to do things like:

server.forAnyRequest()
    .wait(100, 'ms')
    .streamBodyThrough(transformStream)
    .limitBandwidth(1024, 'kbps')
    .thenForwardTo('example.com')

This is particularly relevant to start better supporting streaming APIs, and expanding the websocket API, to also allow defining whole series of actions ('send a chunk of response body containing 'abc', wait 10 seconds, send a chunk with 'xyz', etc). There's a lot of scope here, and potential to add lots of key features I've been interested in for a while, though I haven't explored it in great depth yet. I do have a working implementation of the same model within MockRTC though, defining a series of steps for mock behaviour for WebRTC interactions.

I think ideas like this would fit really well into that model, and because of that I'm cautious to start extending the existing API to fit them instead. But at the same time, that's not going to be ready imminently as I'm not actively working on it right now, so that's a bit inconvenient for you of course if you want to use onStreaming ASAP. What do you think?