-
Notifications
You must be signed in to change notification settings - Fork 209
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Precompile lua functions to prevent wasted CPU cycles #328
Precompile lua functions to prevent wasted CPU cycles #328
Conversation
This reduces queue ticks from 10ms to 500ms in development, lowering the number of checks by 50x. This reduces CPU usage in development, as miniredis is CPU heavy. Also, alicebob/miniredis#328 should be upstreamed when merged, further lowering CPU usage.
Hi, thanks for the clear PR! It looks good, but some error messages changed. The errors can't have the exact same wording as Redis, but they should use the same |
@alicebob thanks for the quick review! believe that i've fixed that error, appreciate the heads up :) |
c207b0a
to
7c30851
Compare
Perf: reduce queue ticks when using dev server to 500ms This reduces queue ticks from 10ms to 500ms in development, lowering the number of checks by 50x. This reduces CPU usage in development, as miniredis is CPU heavy. Also, alicebob/miniredis#328 should be upstreamed when merged, further lowering CPU usage.
Hi, thanks for the fix. Just the test failure left. I see the Lua code eventually does this in
https://github.com/yuin/gopher-lua/blob/v1.1.0/auxlib.go#L387 so maybe do it that way? |
Aha, didn't notice that — sorry! All fixed. Had to change the name of the input script to match before as you mentioned. Thank you! |
`miniredis.runLuaScript` calls lua.*LState.DoString() for every lua invocation, even if scripts are evaluated via checksums. In turn, lua parses and compiles the same scripts on each call, which can cause high CPU usage on script-heavy test instances. This PR uses `sync.Map` to store pre-compiled lua functions for each script, using the same logic as the Lua library itself: tul/gopher-lua@7a6135d This reduces CPU usage by about 50%.
723ab68
to
536e717
Compare
Thanks! It's now in master. |
miniredis.runLuaScript
calls lua.*LState.DoString() for every lua invocation, even if scripts are evaluated via checksums.In turn, lua parses and compiles the same scripts on each call, which can cause high CPU usage on script-heavy test instances.
This PR uses
sync.Map
to store pre-compiled lua functions for each script, using the same logic as the Lua library itself: tul/gopher-lua@7a6135dThis reduces CPU usage by about 50%.
Before:
After:
This takes
runLuaScript
's cumulative time from 1.19s to 0.68s, or about a 57% improvement.