multipart/related stream body upload

[thomas-riccardi]

Current multipart option for multipart/related requests only works with buffers or string body.
When setting body to a stream (like a fs.createReadStream()) it silently fails and sends an empty part.

This would be a great addition to request to handle this.

As we don't know the stream final size this forces the use of chunk encoding, but it's not an issue for me.

In the meantime we could have a real error when the body type is not supported.

[Philmod]

+1

[mikeal]

can you give me a code sample?

[Philmod]

This doesn't work for me :

fs.createReadStream('test.json').pipe(request.post('http://requestb.in/1aimq981'));

Check http://requestb.in/1aimq981?inspect

[mikeal]

what does this have to do with multipart?

the code sample you just posed is a json object which, by default, we post as content-type: application/json

if you want multipart forms checkout the form documentation https://github.com/mikeal/request#forms

[thomas-riccardi]

var request = require('request');

var source = 'https://registry.npmjs.org/';
var target = 'http://requestb.in/x2qhewx2';

request({ url: source, qs: { test: 1 } }).pipe(request(target));

try {
  request({
    url: target,
    qs: { test: 2 },
    body: request(source)
  }, function (error, response, body) {
    console.log('this never happens because an error is thrown', error);
  }).on('error', function (error) {
    console.log('body stream on error:', error);
  });
} catch (error) {
  console.log('body stream thrown error:', error);
}

request({
  url: target,
  qs: { test: 3 },
  multipart:[
    { body: 'test' },
    { body: request(source) }
  ]
}, function (error, response, body) {
  console.log('this silently fails. error:', error);
}).on('error', function (error) {
  console.log('multipart body stream on error:', error);
});

Result:

body stream thrown error: [Error: Argument error, options.body.]
this silently fails. error: null

Network output:
http://requestb.in/x2qhewx2?inspect

The test 1 works (except that with the piping it seems the source request headers & query string are also forwarded to the target request, ignoring the qs: { test: 1 }... is this intended? or is this an issue?): the json body is streamed.

The test 2 doesn't work, as expected, but indicates the issue clearly: it throws an error. It's strange though that it throws the error even if we have a callback and an event listener on 'error' that can be used to notify the error. Usually the error is thrown only if there is no other way to notify the error (a callback or an error listener). Should I fill a new issue for that?

The test 3 doesn't work, as expected, but silently fails: no error is thrown, no 'error' event is emitted, no error on the callback first argument. The HTTP request just contains an empty part:

--a794f767-b161-4ca9-b2d8-7d545b92d53b

test
--a794f767-b161-4ca9-b2d8-7d545b92d53b


--a794f767-b161-4ca9-b2d8-7d545b92d53b--

It is clearly an issue that it silently fails; but we could to better than throwing an error: we could accept a stream as body, both in multipart and standard body

[cyrusdavid]

+1

[ryanseys]

+1 for this. Need a way to specify the multipart/related body as a stream.

[seanstrom]

@FredKSchott does this feature relate at all to the multipart form stuff you worked on?

[FredKSchott]

Yup, support for this has been added with the formData option. Check the README for information on how to use it.

[thomas-riccardi]

The initial issue is still here.

The feature request for multipart stream support is partly implemented by the new formData option, but it only works with multipart/form-data, not multipart/related content-type. If the server only accepts multipart/related and/or refuses Content-Disposition: form-data in parts headers then we still cannot use the request library.
This case may not be that common though.

However, the bug report of this issue is not fixed at all: if we put a stream object as body in multipart, then request still silently fails and send an empty part without any error.

[thomas-riccardi]

@FredKSchott comments on my last remarks?

Could someone re-open the issue?

[simov]

@triccardi-systran check out the docs, with the latest release it is possible to pass streams as multipart/related body

[FredKSchott]

Ah my bad @triccardi-systran, the notification must have gotten lost in the shuffle.
@simov might have more context into the recent work on multipart/related, and whether this is now possible. Feel free to reopen if this is still a bug/missing feature.

[thomas-riccardi]

Finally fixed in #1271 and #1276.
Thank you all.