Switch to streams, instead of using fixed-size buffer

[netroy]

Current implementation uses a fixed length buffer, that causes issues with some JPEGs .. we should switch over to streams.. That'll also make detecting image-size on streaming requests a lot easier.

This'll fix #34 & #37

[tonybart1337]

Hey, guys. If you're interested I've made a module here

[netroy]

@AshtonRoute that looks pretty good. I'll look into it, as soon as I get some time.
Thanks & Cheers

[phal0r]

@netroy
thanks for this lib. Any update on supporting streams? I would really like to use this for one of my projects.

[fregante]

For anyone looking for a stream-based solution, get this function:

async function getStreamImageSize(stream) {
	const chunks = []
	for await (const chunk of stream) {
		chunks.push(chunk)
		try {
			return sizeOf(Buffer.concat(chunks));
		} catch (error) {/* Not ready yet */}
	}

	return sizeOf(Buffer.concat(chunks));
}

Here's a full example code that shows how to use it. Put it in a demo.js file and run it:

const url = require('url')
const http = require('https')

const sizeOf = require('image-size')

const imgUrl = 'https://i.imgur.com/O2HfwuZ.jpg'
const options = url.parse(imgUrl)

async function getStreamImageSize(stream) {
	const chunks = []
	for await (const chunk of stream) {
		chunks.push(chunk)
		try {
			return sizeOf(Buffer.concat(chunks));
		} catch (error) {/* Not ready yet */}
	}

	return sizeOf(Buffer.concat(chunks));
}
http.get(options, async stream => {
	console.log(await getStreamImageSize(stream));
});

This is so fast that in my tests it works with the very first chunk.

[phal0r]

@fregante
Nice solution with using async iterator. Meanwhile I was using a util, which uses a passthrough stream and does essentially the same. The only difference is, that there is a max buffersize (default 128kb) to prevent storing the whole binary image into memory in case of malformed header information or other cases, which could prevent proper detection.

[fregante]

Should it stop after the buffer reaches 100kb? Feels like an easy addition in this case

[mnpenner]

I would think so. Image size is usually pretty early in the file no? Ideally it would stop reading the stream as soon as it found it.

[fregante]

Ideally it would stop reading the stream as soon as it found it.

The getStreamImageSize suggested above already does that. I was talking about a possible extra safety check.

[mnpenner]

Oh, so it does. Didn't look closely enough. I imagine there's a better way than a try/catch on every chunk but that's not too bad.

[fregante]

I imagine there's a better way than a try/catch on every chunk

Indeed if image-size were to implement a non-throwing method to check the size it could be used in this scenario.

[phal0r]

@fregante
With my solution it throws after 128kb are read and no information are found, yes. From my understanding only header information is relevant for detecting the image format, so it does not make much sense to buffer the whole image. If 128kb is enough - no idea. I took inspiration from another package and just refactored it a bit to more modern javascript.

I agree, that the try/catch block is the only feasible solution. The only other way would be to buffer to a minimum bytesize, where all information should be available, but I guess this also depends on the image type.