Accept streams as body

[petkaantonov]

If I want to send the request body e.g. gzipped, I currently need the return value and pipe the gzipstream there. However I only have access to the return value when using callback API but we are normally using promise API where the return value "slot" is taken by the promise and we cannot interact with the stream.

I propose extending the body option to accept stream which would do this for me so that I don't need to waste the return value "slot". Any errors should be redirected to the callback error argument.

[simov]

@request/promises can you help with this issue?

[petkaantonov]

Note I am using request through promisifying the callback api, so changes to any promise api doesn't address this.

This is also something that could be useful even without using promises, so I don't think the promise label is entirely accurate.

[analog-nico]

Hi @petkaantonov if I understand it correctly you currently use:

gzipStream.pipe(request.post(endpoint));

...whereas this wouldn't work:

gzipStream.pipe(request.post(endpoint, function (err, response, body) {
    // Callback is provided through promisification.
    // Thus cannot have custom code that interacts with the stream here.
}));

...and you propose this:

request.post({
    uri: endpoint,
    body: gzipStream // Passing a stream instead of just JSON
}, function (err, response, body) {
    // err is either an error of the request or an error message of the gzipStream stream
});

Correct?

[petkaantonov]

Yes I want body: gzipStream to work. But gzipstream.pipe(request.postAsync()) doesn't work since postAsync returns a promise not a stream.

[simov]

@petkaantonov can you provide a full pseudo code on what you are trying to achieve.

[petkaantonov]

request({
   ...
   body: gzipStream
   ...
});

Should work the same as if I did gzipStream.pipe(request({...}). Everything else is just reasoning why I need this and not relevant per se.

[simov]

That seems doable. What happens if you want to stream the response as well? In this case your only choice would be to buffer it inside the callback, though that's probably the case when using Promises anyway?

[petkaantonov]

Well in practice when request payload is big, the response payload is small. And vice versa. So it's not a problem to buffer the short acknowledgment response.

[simov]

What I'm trying to say is that the user should be able to pass not only the input stream, but the output one as well. Buffering by default might not be what you want.

So body: stream should be implemented in request, and other_option: output should be implemented in request-promise since it's already a wrapper and it deals with promises.

When the response event is emitted the response should be piped to whatever output stream you pass to request-promise and the promise should be resolved on output stream end event - that's the case when you don't want to buffer the entire response body in a callback.

[petkaantonov]

Yes I am not against that of course.

btw I have no affiliation with request-promise. I am using the vanilla request via bluebird's promisify function which does automagic api translation.

[simov]

@petkaantonov take a look at #2079

[analog-nico]

@simov I covered your suggestion in request/request-promise#90