Operate on blob parts (byte sequence)

[jimmywarting]

This PR makes fetch-blob extendable to work with any 3th party blobs by still using them as the source

---

I was successful at using this http-blob WebIDL like reader and add it into fetch-blob using the following code

new Blob([ new HttpFileLike(...) ])

No binary have been read, size still reflect correctly (even after using slice)
the http request isn't being made until i actually call blob.arrayBuffer(), text or stream

---

this will also make it possible to later add in File entries that is backed up by the file system later - you won't have to add anything into the memory and it will still be able to slice and read chunks

Doing something like this now won't end up resulting in 4 gib ram being used

var file1 = File.from(path_to_2gib) // blob like backed up by fs
var file2 = File.from(path_to_2gib)

var concated = new Blob([file1, file2])

you will still be able to read both file as one single blob

closes #40

[codecov]

Codecov Report

Merging #44 into master will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##            master       #44   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            1         1           
  Lines           62        70    +8     
  Branches        12        20    +8     
=========================================
+ Hits            62        70    +8     

Impacted Files	Coverage Δ
index.js	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9dc0747...aa801ae. Read the comment docs.

[jimmywarting]

Also included some of #43 jsDoc, and version change

[jimmywarting]

i guess this is more like how browser handles blob parts

[tinovyatkin]

Wouldn't be adding [Symbol.AsyncIterator] directly into the Blob class looks better than using read function in Readable.from?

[tinovyatkin]

I think a little test case, checking the example from #40 should be added:

var b1 = new Blob([new Uint8Array(1000)])
var b2 = b1.slice(500) // uses the same parts with an offset=500
// b2 should not take up 500 byte more ram... 

Maybe using process.memoryUsage() (as Node is single-threaded it should work?)

[jimmywarting]

Symbol.AsyncIterator don't really belong in the blob prototype...

---

i did a manual test on that. it will either use Buffer.slice() which actually means Buffer.subarray() or it will slice a blob instance.

used a 20mb Uint8Array, sliced from start to end and made sure it still used 20mb
fyi, it did also to it before this pr since it was backed up by buffer's way of slicing it like it where subarray

[tinovyatkin]

fyi, it did also to it before this pr since it was backed up by buffer's way of slicing it like it where subarray

So, what we are fixing here if memory usage is the same? The way data is reading?

[jimmywarting]

So, what we are fixing here if memory usage is the same? The way data is reading?

Yup, data is read in another way now.

[jimmywarting]

I want to rather follow the spec.

Also now when you slice a blob you are going to use the same parts but filtering out some part that are not needed and it will also set a offset (on the first and last part) even doe it may not look like i'm doing it (code-wise).

[tinovyatkin]

I think JSDoc should be completed to integrated types building out of it later.

[tinovyatkin]

As I understand from node-fetch/node-fetch#835 merging this will make node-blob more useful in the userland. Maybe it worth adding an example into README of practical use of Blob out of node-fetch, so, this module may seek wider adoptions/contributions? Because the current example is, hmm, strange and useless.

[jimmywarting]

merging this will make node-blob more useful in the userland

yea, i bet https://github.com/octet-stream/form-data would want to use this also. or if developer uses it together with form-data

One wishful thinking would be to add File into this library as well and some utility function that creates a binary container from filesystem. Everything File really is, is this:

class File extends Blob {
  constructor(blobParts, fileName, options = {}) {
    const { lastModified = Date.now(), ...blobPropertyBag } = options;
    super(blobParts, blobPropertyBag);
    this.name = String(fileName).replace(/\u002F/g, "\u003A");
    this.lastModified = lastModified;
  }
}

The rest is handled by blob. I actually found a readme from chrome explaining how blobs are handled https://chromium.googlesource.com/chromium/src/+/master/storage/browser/blob/README.md

• one where blobFrom(path) and back it up with fs
  • then you had to use new File([ blobFrom(path) ], 'filename') or something like that.
• another was just new File([ partFrom(path) ]) (but that would just act more or less like blobFrom(path)
• Another where just file = fileFrom(path)

But this is a bit out of this scope and could be for another issue.

[readeral]

This PR introduced an issue where fetch-blob installation fails when using a Node major version greater than 10. Raised in #48 and fix proposed in #49 .