Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding an extremely large file to archive returns an error #816

Open
WillHammer93 opened this issue Jun 10, 2023 · 3 comments
Open

Adding an extremely large file to archive returns an error #816

WillHammer93 opened this issue Jun 10, 2023 · 3 comments

Comments

@WillHammer93
Copy link

LRR Version and OS
0.8.90, Windows 11 Education version 22H2
CPU: i7-12700K
RAM: 32GB

Bug Details
Reproducable error when attempting to add one or more extremely large archives.

The compressed folder that I haven't been able to add by itself is 10GB.
Nipping this problem at the bud would be great since there are plenty of image sets that are over 9GB in size.

I don't have any computers available that would have even more RAM or a more powerful CPU, so I can't test if it is just a matter of hardware not being able to handle it. Honestly I doubt that would be the case, but I am not an expert by any means so that is still an option to consider.

Matching Logs

Error checking Minion job status
Error: Can't open file "/home/koyomi/lanraragi/script/../log/lanraragi.log": Read-only file system at /usr/local/share/perl5/site_perl/Mojo/Log.pm line 19.

This would suggest that this might just be a me problem, but I haven't found the referred file so I could disable the read-only setting.
All signs point to it being in a Linux/Unix location but I am running Windows, so I can't find it.
Sorry for not being a programmer. I literally made a Github profile for this...

Screenshots
Following happens after trying to upload the archive again after failure.
Breaks the installation but is easily fixed with the built in re-installer.
Untitled

@WillHammer93
Copy link
Author

With a suggestion from Discord, I managed to successfully add the file to the archive by dropping the file to the content folder directly.
This means that the issue is with the web transaction step and despite curing the symptom, the core problem remains

@lxrst
Copy link

lxrst commented Jun 23, 2023

Education version

Ha

hardware not being able to handle it

That's possible-- not sure how [whatever library LRR uses] works-- but yeah, probably not the case.

All signs point to it being in a Linux/Unix location but I am running Windows, so I can't find it.

I believe LRR runs inside Docker/WSL on Windows, so that'd be why.

Sorry for not being a programmer.

No worries!

I don't know Perl at all, but I might use this as an opportunity to learn it.

A few questions:

  • What format is your archive and what level was it compressed at (if you know)?
  • How large are the contained images?

Also, it'd be helpful if you could link the file, so I can try to reproduce the issue.

@WillHammer93
Copy link
Author

WillHammer93 commented Jun 23, 2023

What format is your archive and what level was it compressed at (if you know)?

The problem file was a zip file which I compressed at the "Best" compression method of WINRAR with dictionary size of 32KB. I use the same settings for all my archives.

How large are the contained images?

The largest single image is ~29mb and the smallest is just 665kb. Judging by the statistically inaccurate analysis method of eyeballing it, I'd say the vast majority of images are between 5-7mb. In total there are 1653 images.

Also, it'd be helpful if you could link the file, so I can try to reproduce the issue.

The archive is called "toraisix Part 1" and it can be found in sadpanda (/2389953/ddcc780995/).
There is a torrent available but I used GalleryDL to rip the images directly since it was one of multiple archives I was ripping and I had the sufficient quota to handle it.

I haven't ran into similar problems afterwards despite that the second largest archive file I have is just 30mb smaller while the largest single image in it is at a whopping 48mb.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants