Question: Who and why uses Blob on Node.JS

[tinovyatkin]

Hello,

I'm trying to find a real-world example of using Blob response type in Node.JS project/library.

Does anybody know any example of existing library or project that benefits of Blob in Node?

[Richienb]

You can, for example, request binary files like images as blobs.

[tinovyatkin]

@Richienb Yeah, I can image some examples, but I'm looking for real-world applications, as an open-source library, etc...

[jimmywarting]

Guess there isn't really any huge real-world use for it today, have seen someone using it doe. Deno have it built in but it is mostly just a wrapper around some arraybuffer i guess

---

The purpose of having a blob is mostly just to

• save RAM.
• creating object urls for media and web workers
• storing data in browser storage (like IndexedDB)
• helping us making binary request

I recommend you to read this
https://stackoverflow.com/questions/38239361/where-is-blob-binary-data-stored

---

There is also some feature request about implementing blob & blobStorage in node. Understanding the usefulness of Blobs and how they really work under the hood can boost your performance

There is already something analogous in the http2 implementation in the form of the respondWithFile() and respondWithFD() APIs in the http2 side. Basically, the idea would be to prepare chunks of allocated memory at the native layer, with data that never passes into the JS layer (unless absolutely necessary to do so), then use those to source the data for responses. In early benchmarking this yields a massive boost in throughput without the usual backpressure control issues.

---

fetch-blob isn't perfect either, there is actually many parts it dose "wrong", if you want to slice a blob then you shouldn't really allocate a new buffer with the new data. the underlying source should not be one buffer, it should really be an array of blob parts with new offsets. more like this File-like http reader

---

having blob's in formdata with a pointer to the filesystem makes a tremendous memory boost and helps us read/serialize data in a "standard" way when you make request.

[jimmywarting]

imagine how you would go about solving this:

const File = require('fetch-file') // creates a FIle-like IDL wrapper around fs
const Blob = require('fetch-blob')

const file = File.from('./package.json')
new Blob([ file ])

the blob's constructor spec says:

1. Let bytes be an empty sequence of bytes. (it's not like a long ArrayBuffer holding all data)
2. For each element in parts:
   3. If element is a Blob, append the bytes it represents to bytes.

2.3 don't say read the the blob and copy the content to a new instance. Beside the fetch-blob can't read the content of the File in a synchronous way to be able to successfully handle the blob parts. so the slicing method is also wrong, it should not slice the Buffer like fetch-blob dose today. it should instead update the byte sequences and and create a new offset of where it should start reading stuff from. This also means that arrayBuffer, text and stream is wrong too. fetch-blob should instead read each blobPart individually at a later stage to successfully be able to handle the File - the constructor should not read the file content.

---

when you take 2x2 GiB of files from a <input type="file" multiple> and concatenate new Blob([fileA, fileB]) then you won't end up with 4 GiB of RAM, it really means that you have a new blob container with two references point to where a snapshot is located

image taken from: https://docs.google.com/presentation/d/1MOm-8kacXAon1L2tF6VthesNjXgx0fp5AP17L7XDPSM/edit

[tinovyatkin]

@jimmywarting That said, current fetch-blob implementation basically renders Blob useless in the userland?

[jimmywarting]

Mostly yes, but not entirely - there is room for improvements to make it more useful but it would be best if we got native support for blob/files instead to get a "single source of truth"

Those who use it now days just want to be Isomorphic

There could be a other blob polyfill that could handle fetch-blob correctly like a blob part but very unlikely. since there haven't been a way to read them upon till recently with the new reading methods (stream, text, arrayBuffer) but some are starting to use it instead of using FileReader

fetch-blob constructor can only handle it's own instances of Blob atm.

There is also the lack of ObjectURLs in node. you can't display them in some image or video tag. but they can still be useful for workers: eg like new Worker(URL.createObjectURL(blob)) but that dose not work in node either,

node-fetch can't read objectURL's either since BlobStore don't exist.

[jimmywarting]

@Gozala @mattlubner @avaer maybe wants to share what they use blobs for, they have asked us to expose the blob class

[LinusU]

I'm using Blob as a standard data structure that hold some bytes and a mime type. I think it's nice since I don't have to invent my own class for it, and potentially other packages could work with the same type...

e.g. we have some functions that returns Blob, and then another function which receives them and responds to an Express request with it, something like:

res.writeHead(200, { 'Content-Type': blob.type })
blob.stream().pipe(res)

[mattlubner]

You can see this comment on #392, which explains my use case.

tl;dr: We need the exact constructor used by res.blob() to be exported so we can assert using isInstanceOf(Blob) on the returned value.

In general, I feel that all libraries which return objects that extend internally-defined classes aught to export those classes in some manner, if for no other reason than to support robust testing.

Btw, I'm pretty sure @Gozala's use case can be inferred from this comment.

[jimmywarting]

if we got FetchEvent implemented (#370) and some kind of simplified server then i would have started to use blob's in node a lite bit more. almost like @LinusU did

something in lines of this

const File = require('fetch-file')

// A reusable blob that don't hold anything in memory
// but is re-readable thanks to the new .stream()
const index = File.from('./static/index.html') 

globalThis.onfetch(event => {
  // content-type, content-length all taken care of by node-fetch
  event.respondWith(new Response(index))
})

http.createServer((req, res) => {
  dispatch_to_nodeFetch_as_fetch_event(req, res)
})

Essentially creating a little mini service worker

[tinovyatkin]

I don't like something sticking with Node.JS globalThis, but I'm seeing FetchEvent to be great together with AsyncLocalStorage and would love to implement that:

const store = asyncLocalStorage.getStore();
store.onfetch(event => {
  /*.. */
});

[jimmywarting]

I don't like something sticking with Node.JS globalThis, but I'm seeing FetchEvent to be great together with AsyncLocalStorage and would love to implement that:

I have no exact idea of how we go about register a listener to the onfetch event, just tried making a rough point with a mix of how native service worker works

not sure how AsyncLocalStorage fits in...?

I'm also wondering a bit about what would happen if two modules uses node-fetch and it dose request, would you want to listen to all request that are going on or would you want to instantiate something by calling fetch = new Fetch() or something like that?

Anyhow, lets move the discussion about FetchEvent into #370