Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MaxBufferSize is not enough. #34

Open
peecky opened this issue Oct 20, 2014 · 13 comments
Open

MaxBufferSize is not enough. #34

peecky opened this issue Oct 20, 2014 · 13 comments
Milestone

Comments

@peecky
Copy link

peecky commented Oct 20, 2014

Hello.

I got an error, TypeError: Corrupt JPG, exceeded buffer limits, when I use image-size with an JPEG image file. But when I increase the MaxBufferSize in lib/index.js, it works well.

Could you increase the MaxBufferSize?

What I changed to test:
var MaxBufferSize = 128*1024 * 100;

This is the sample file: https://www.dropbox.com/s/qvat156gv0nq13b/b95ff27a4b146a0b95cb4d5f58189c4f.jpg?dl=0

@oliverfoster
Copy link

+1

@nickjuntilla
Copy link

Increasing this buffer size by default just like this would be a great temporary fix.

@ajarbol
Copy link

ajarbol commented Jan 27, 2016

+1

@huyinghuan
Copy link

  • 1

@zeke
Copy link
Contributor

zeke commented Apr 1, 2016

In the comment above the MaxBufferSize setting:

// TO-DO: make this adaptive based on the initial signature of the image
var MaxBufferSize = 128*1024;

I'd be happy to quickly patch this by increasing the buffer size, but I'd be curious to hear from @netroy why or how the buffer size should be adaptive? Is it to prevent excessive memory allocation?

@puzrin
Copy link

puzrin commented Apr 2, 2016

@zeke, you will deal with such "adaptive" kludges forever, until add true streaming support.

See https://github.com/nodeca/probe-image-size. We did it because this package was not supported for a long time. Feel free to reuse as you wish.

@oliverfoster
Copy link

Thanks for that link @puzrin

@puzrin
Copy link

puzrin commented Apr 2, 2016

@oliverfoster if you plan to use that package, note that it has no SVG support (dropped until someone has time to do it well). From the other hand, there are no such bugs as here :)

@oliverfoster
Copy link

Cool. Isn't the whole point of svg that they have no defined size?

@puzrin
Copy link

puzrin commented Apr 2, 2016

SVG can be with and without size. Problem is how to terminate early - how to decide with good confidence that input is NOT svg image.

Input data is broadcasted to streaming decoders, and then we wait until all decoders fail or someone return good result. All decoders should terminate as early as possible, to minimize reading from input stream.

@oliverfoster
Copy link

I didn't realise you could size them. Cool. In that case I guess you wouldnt really me able to tell the size unless you had at least all of the opening tags and the last one might be right before the end. The XML document format isn't really good for that case as it doesn't really have a header. Might just be better to parse the whole thing as an exception?

@puzrin
Copy link

puzrin commented Apr 2, 2016

Might just be better to parse the whole thing as an exception?

IMHO that would be the same crap like buffer size tune :)

Quick-check can test that data begins with <. Then it's possible to push reencoded chunks to something like sax (streaming parser). Everything can be done well, without reading full content. Someone just need to spend time to do things right :)

@oliverfoster
Copy link

I look forward to seeing it when it's done. Sounds like I could learn a thing of two.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants