Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

missing image file on disk with Minio will make Imageproxy peak #234

Open
Calx-Takeno opened this issue Jun 3, 2020 · 0 comments
Open

Comments

@Calx-Takeno
Copy link

Calx-Takeno commented Jun 3, 2020

We are using Imageproxy and Minio. They are good partner in most of time.
But we found mass of exceptions from Imageproxy console. So we investigate it.
When image file is missing on disk, Minio will return 'doesn't exist' xml.
So Imageproxy occur transforming error exception. And these exceptions are concurrency.
Traffic is more than 1k requests per second on our production server. Concurrent error exceptions make CPU/RAM/Disk-IO peak.
If error exceptions shot up suddenly. Imageproxy will crash.
We have set memery cache. It still consume hardware resources heavy.
I think error excptions are waiting for handling untill exceed of RAM range, this consume most of hardware resource.
Could you develop feature to ignore Minio 'doesn't exist' xml. Like cache initial params '-minio ignore-missing', Imageproxy will be better.
We have 2 nodes of Imageproxy. But they can not server our traffic well when mass missing image file request. Like missing file DDOS attacks Imageproxy.

Imageproxy version: 0.10.0
Minio version: 2019-08-07T01:59:21Z
Server Machine: windows 2016 x64, 8 cores/16GB RAM/1TB SSD

image
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant