Set a sensible default for HSQL defrag #8062
Comments
|
Sounds good to me!
Have a look at that and let us know if you're still on for fixing this 😁 |
|
Could you share the SQL statements? |
|
@psiinon Sure, I have a locally functional working code, provided in this PR #8064 I am noting that I need to cleanup a few things and VSCode autoformatting is giving me issues with linting with ./gradlew build but I will return to this and figure that out. Once that is solve I can then look at further code cleanups but happy to receive feedback at this point if desirable to this team. It will need some bounds checking e.g. 0-99 @thc202 it is |
|
You can use |
I came back to sit down and "learn about java linters" and this made me go from "ugh" to "solved" in one command, thank you so much! |
|
I was asking about this |
Is your feature request related to a problem? Please describe.
While performing scans on a large (to me) GraqhQL interface, I could not complete the initial scan using the GraphQL addon and automation as the database grew to 128Gb and its maximum size. On inspection, it became clear that the params statistics gathering generated DELETE and INSERT statements, leaving a lot of unused space. In this instance, once I issued a SHUTDOWN COMPACT the .data file was reduced from 128Gb to 144Mb.
Creating a session, closing zap, and altering
SET FILES DEFRAG 0to30solved this problem. However, in hindsight, I think this would be better at a much larger figure. HSQL document suggests useful values are between 30 and 60. With the same scan running, the file size is being kept in check, and the graphql scanner can complete.Reference for
SET FILES DEFRAGhttps://www.hsqldb.org/doc/2.0/guide/management-chapt.html
Describe the solution you'd like
Some way to easily set/unset the DEFRAG threshold OR a default where if we are seeing more than 60/70/80% reclaimable so that out of the box, larger scans are likely to work. Of course, if the defragging is a performance problem, users should be able to disable it.
Describe alternatives you've considered
As described above, You can manually edit the .script file for a session and make a specific session defrag at any percentage point you like.
Screenshots
No response
Additional context
The scan filled the disk at around 26,000 requests mark in multiple attempts. The scan is being executed via Automation, there was no feedback in the GUI that the DB is having an issue.
I have said yes to helping fix this issue. I am happy to try and work on other potential fixes, but I don't know Java.
Would you like to help fix this issue?
The text was updated successfully, but these errors were encountered: