Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

buffer: introduce Blob #36811

Closed
wants to merge 4 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
114 changes: 114 additions & 0 deletions doc/api/buffer.md
Expand Up @@ -279,6 +279,119 @@ for (const b of buf) {
Additionally, the [`buf.values()`][], [`buf.keys()`][], and
[`buf.entries()`][] methods can be used to create iterators.

## Class: `Blob`
<!-- YAML
added: REPLACEME
-->

> Stability: 1 - Experimental

A [`Blob`][] encapsulates immutable, raw data that can be safely shared across
multiple worker threads.

### `new buffer.Blob([sources[, options]])`
<!-- YAML
added: REPLACEME
-->

* `sources` {string[]|ArrayBuffer[]|TypedArray[]|DataView[]|Blob[]} An array
of string, {ArrayBuffer}, {TypedArray}, {DataView}, or {Blob} objects, or
any mix of such objects, that will be stored within the `Blob`.
* `options` {Object}
* `encoding` {string} The character encoding to use for string sources.
**Default**: `'utf8'`.
* `type` {string} The Blob content-type. The intent is for `type` to convey
the MIME media type of the data, however no validation of the type format
is performed.

Creates a new `Blob` object containing a concatenation of the given sources.

{ArrayBuffer}, {TypedArray}, {DataView}, and {Buffer} sources are copied into
the 'Blob' and can therefore be safely modified after the 'Blob' is created.

String sources are also copied into the `Blob`.

### `blob.arrayBuffer()`
<!-- YAML
added: REPLACEME
-->

* Returns: {Promise}
jasnell marked this conversation as resolved.
Show resolved Hide resolved

Returns a promise that fulfills with an {ArrayBuffer} containing a copy of
the `Blob` data.

### `blob.size`
<!-- YAML
added: REPLACEME
-->

The total size of the `Blob` in bytes.

### `blob.slice([start, [end, [type]]])`
<!-- YAML
added: REPLACEME
-->

* `start` {number} The starting index.
* `end` {number} The ending index.
* `type` {string} The content-type for the new `Blob`

Creates and returns a new `Blob` containing a subset of this `Blob` objects
data. The original `Blob` is not alterered.

### `blob.text()`
<!-- YAML
added: REPLACEME
-->

* Returns: {Promise}

Returns a promise that resolves the contents of the `Blob` decoded as a UTF-8
string.

### `blob.type`
<!-- YAML
added: REPLACEME
-->

* Type: {string}

The content-type of the `Blob`.

### `Blob` objects and `MessageChannel`

Once a {Blob} object is created, it can be sent via `MessagePort` to multiple
destinations without transfering or immediately copying the data. The data
contained by the `Blob` is copied only when the `arrayBuffer()` or `text()`
methods are called.

```js
const { Blob } = require('buffer');
const blob = new Blob(['hello there']);
const { setTimeout: delay } = require('timers/promises');

const mc1 = new MessageChannel();
const mc2 = new MessageChannel();

mc1.port1.onmessage = async ({ data }) => {
console.log(await data.arrayBuffer());
mc1.port1.close();
};

mc2.port1.onmessage = async ({ data }) => {
await delay(1000);
console.log(await data.arrayBuffer());
mc2.port1.close();
};

mc1.port2.postMessage(blob);
mc2.port2.postMessage(blob);

// The Blob is still usable after posting.
data.text().then(console.log);
```

## Class: `Buffer`

The `Buffer` class is a global type for dealing with binary data directly.
Expand Down Expand Up @@ -3380,6 +3493,7 @@ introducing security vulnerabilities into an application.
[UTF-8]: https://en.wikipedia.org/wiki/UTF-8
[WHATWG Encoding Standard]: https://encoding.spec.whatwg.org/
[`ArrayBuffer`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/ArrayBuffer
[`Blob`]: https://developer.mozilla.org/en-US/docs/Web/API/Blob
[`Buffer.alloc()`]: #buffer_static_method_buffer_alloc_size_fill_encoding
[`Buffer.allocUnsafe()`]: #buffer_static_method_buffer_allocunsafe_size
[`Buffer.allocUnsafeSlow()`]: #buffer_static_method_buffer_allocunsafeslow_size
Expand Down
5 changes: 5 additions & 0 deletions lib/buffer.js
Expand Up @@ -115,6 +115,10 @@ const {
createUnsafeBuffer
} = require('internal/buffer');

const {
Blob,
} = require('internal/blob');

FastBuffer.prototype.constructor = Buffer;
Buffer.prototype = FastBuffer.prototype;
addBufferPrototypeMethods(Buffer.prototype);
Expand Down Expand Up @@ -1191,6 +1195,7 @@ if (internalBinding('config').hasIntl) {
}

module.exports = {
Blob,
Buffer,
SlowBuffer,
transcode,
Expand Down
238 changes: 238 additions & 0 deletions lib/internal/blob.js
@@ -0,0 +1,238 @@
'use strict';

const {
ArrayFrom,
ObjectSetPrototypeOf,
Promise,
PromiseResolve,
RegExpPrototypeTest,
StringPrototypeToLowerCase,
Symbol,
SymbolIterator,
Uint8Array,
} = primordials;

const {
createBlob,
FixedSizeBlobCopyJob,
} = internalBinding('buffer');

const {
JSTransferable,
kClone,
kDeserialize,
} = require('internal/worker/js_transferable');

const {
isAnyArrayBuffer,
isArrayBufferView,
} = require('internal/util/types');

const {
customInspectSymbol: kInspect,
emitExperimentalWarning,
} = require('internal/util');
const { inspect } = require('internal/util/inspect');

const {
AbortError,
codes: {
ERR_INVALID_ARG_TYPE,
ERR_BUFFER_TOO_LARGE,
ERR_OUT_OF_RANGE,
}
} = require('internal/errors');

const {
validateObject,
validateString,
validateUint32,
isUint32,
} = require('internal/validators');

const kHandle = Symbol('kHandle');
const kType = Symbol('kType');
const kLength = Symbol('kLength');

let Buffer;

function deferred() {
let res, rej;
const promise = new Promise((resolve, reject) => {
res = resolve;
rej = reject;
});
return { promise, resolve: res, reject: rej };
}

function lazyBuffer() {
if (Buffer === undefined)
Buffer = require('buffer').Buffer;
return Buffer;
}

function isBlob(object) {
return object?.[kHandle] !== undefined;
}

function getSource(source, encoding) {
if (isBlob(source))
return [source.size, source[kHandle]];

if (typeof source === 'string') {
source = lazyBuffer().from(source, encoding);
} else if (isAnyArrayBuffer(source)) {
source = new Uint8Array(source);
} else if (!isArrayBufferView(source)) {
throw new ERR_INVALID_ARG_TYPE(
'source',
[
'string',
'ArrayBuffer',
'SharedArrayBuffer',
'Buffer',
'TypedArray',
'DataView'
],
source);
}

// We copy into a new Uint8Array because the underlying
// BackingStores are going to be detached and owned by
// the Blob. We also don't want to have to worry about
// byte offsets.
source = new Uint8Array(source);
return [source.byteLength, source];
}

class InternalBlob extends JSTransferable {
constructor(handle, length, type = '') {
super();
this[kHandle] = handle;
this[kType] = type;
this[kLength] = length;
}
}

class Blob extends JSTransferable {
constructor(sources = [], options) {
emitExperimentalWarning('buffer.Blob');
if (sources === null ||
typeof sources[SymbolIterator] !== 'function' ||
typeof sources === 'string') {
throw new ERR_INVALID_ARG_TYPE('sources', 'Iterable', sources);
}
if (options !== undefined)
validateObject(options, 'options');
const {
encoding = 'utf8',
type = '',
} = { ...options };

let length = 0;
const sources_ = ArrayFrom(sources, (source) => {
const { 0: len, 1: src } = getSource(source, encoding);
length += len;
return src;
});

// This is a MIME media type but we're not actively checking the syntax.
// But, to be fair, neither does Chrome.
validateString(type, 'options.type');
jasnell marked this conversation as resolved.
Show resolved Hide resolved

if (!isUint32(length))
throw new ERR_BUFFER_TOO_LARGE(0xFFFFFFFF);

super();
this[kHandle] = createBlob(sources_, length);
this[kLength] = length;
this[kType] = RegExpPrototypeTest(/[^\u{0020}-\u{007E}]/u, type) ?
'' : StringPrototypeToLowerCase(type);
}

[kInspect](depth, options) {
if (depth < 0)
return this;

const opts = {
...options,
depth: options.depth == null ? null : options.depth - 1
};

return `Blob ${inspect({
size: this.size,
type: this.type,
}, opts)}`;
}

[kClone]() {
const handle = this[kHandle];
const type = this[kType];
const length = this[kLength];
return {
data: { handle, type, length },
deserializeInfo: 'internal/blob:InternalBlob'
};
}

[kDeserialize]({ handle, type, length }) {
this[kHandle] = handle;
this[kType] = type;
this[kLength] = length;
}

get type() { return this[kType]; }

get size() { return this[kLength]; }

slice(start = 0, end = (this[kLength]), type = this[kType]) {
validateUint32(start, 'start');
if (end < 0) end = this[kLength] + end;
validateUint32(end, 'end');
validateString(type, 'type');
if (end < start)
throw new ERR_OUT_OF_RANGE('end', 'greater than start', end);
if (end > this[kLength])
throw new ERR_OUT_OF_RANGE('end', 'less than or equal to length', end);
return new InternalBlob(
this[kHandle].slice(start, end),
end - start, type);
}

async arrayBuffer() {
const job = new FixedSizeBlobCopyJob(this[kHandle]);

const ret = job.run();
if (ret !== undefined)
return PromiseResolve(ret);

const {
promise,
resolve,
reject
} = deferred();
job.ondone = (err, ab) => {
if (err !== undefined)
return reject(new AbortError());
resolve(ab);
};

return promise;
}

async text() {
const dec = new TextDecoder();
return dec.decode(await this.arrayBuffer());
}
}

InternalBlob.prototype.constructor = Blob;
ObjectSetPrototypeOf(
InternalBlob.prototype,
Blob.prototype);

module.exports = {
Blob,
InternalBlob,
isBlob,
};