Use blobs instead of dataurls to pass data between modules

[jywarren]

We spend a lot of compute time encoding images as datauri in order to pass them from one module to the next. It may also be possible to use blobs and save encoding time, although we would definitely want to compare the benchmark tests to see if this is true.

(moving this from this comment thread)

I think this is probably worth it! I think it's possible. We had very very long ago considered different formats for between-module data handoff, including possibly buffers...? i forget, but it was in our very first issue: #1 (comment)

the default is data-urls, because strings are completely compatible, but if two modules can negotiate that both are compatible with blobs, that's OK too. I think originally the data-urls worked especially well because they had to be displayed anyways. But now that I think of it I think it's possible to display a blob as well in the browser.

Also note https://stackoverflow.com/questions/7650587/using-javascript-to-display-a-blob

I wonder if this would just work, but it depends on the run environment, I imagine:

image-sequencer/src/Run.js

Lines 18 to 54 in 5298d1e

	var i = drawarray[pos].i;
	var input = ref.steps[i - 1].output;
	var step = ref.steps[i];
	step.getStep = function getStep(offset) {
	if (i + offset >= ref.steps.length) return { options: { name: undefined } };
	else return ref.steps.slice(i + offset)[0];
	};
	step.getIndex = function getIndex() {
	return i;
	};
	for (var util in getStepUtils) {
	if (getStepUtils.hasOwnProperty(util)) {
	step[util] = getStepUtils[util];
	}
	}
	// Tell UI that a step is being drawn.
	step.UI.onDraw(step.options.step);
	// provides a set of standard tools for each step
	var inputForNextStep = require('./RunToolkit')(ref.copy(input));
	step.draw(
	inputForNextStep,
	function onEachStep() {
	// This output is accessible by UI
	ref.steps[i].options.step.output = ref.steps[i].output.src;
	ref.steps[i].options.step.wasmSuccess = ref.steps[i].output.wasmSuccess || false;
	ref.steps[i].options.step.useWasm = ref.steps[i].output.useWasm || false;
	// Tell UI that step has been drawn.
	ref.steps[i].UI.onComplete(ref.steps[i].options.step);
	drawStep(drawarray, ++pos);

We could add a parameter to drawStep somewhere that evaluates whether the neighboring steps can both handle blobs.

Interestingly i think we may already be doing this in some modules, where interacting with the glfx library:

image-sequencer/src/modules/ColorHalftone/Module.js

Lines 35 to 36 in 3caa033

	var blob = dataURItoBlob(canvas.toDataURL('image/png'));
	return window.URL.createObjectURL(blob);

See this section:

image-sequencer/src/modules/ColorHalftone/Module.js

Lines 71 to 82 in 3caa033

	var burl = canvasToBlobUrl(canvas);
	if (download) {
	window.open(burl);
	} else { // replace the image
	// keep non-blob version in case we have to fall back:
	// image.src = canvas.toDataURL('image/png');
	// window.location = canvas.toDataURL('image/png');
	imageEl.src = burl;

there, we are trying to treat a blob URL just like a url by using blobUrl = window.URL.createObjectURL(blob); !!

[jywarren]

cc @publiclab/is-maintainers

[jywarren]

And noting that this would also require documentation changes to the Module API to make devs aware that this is allowed.