-
Notifications
You must be signed in to change notification settings - Fork 969
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Prefetch entries in Soroban ops #3651
Comments
Setting target milestone to |
It looks like Soroban TXs (Restore, Extend, InvokeHostFunction) don't prefetch any keys. For SQL, this makes sense. We prefetch all keys in a single query, so there is no way for us to enforce byte read limits. We could potentially do something like this but the query performance may be significantly limited by such measures, defeating prefetch gains in the first place. However, for BucketListDB, we can enforce a byte limited prefetch, where we prefetch keys until we reach the given limit. This would provide more efficient disk access but prevent IO DOS attacks, where a malicious TX includes lots of very large keys with a small readBytes value. This will require some plumbing to make sure we propagate the prefetch byte limit correctly since this would have to be a per-tx limit. For SQL, I think it's best that we just permanently disable Soroban TX prefetching since we're shifting to BucketListDB anyway and a SQL solution doesn't seem super clear. |
Agree. I think for BucketsDB the benefit is batching should allow us to take advantage of the order of keys and reduce random seeks (and may also yield benefits down the line with parallelization as well). |
InvokeHostFunctionOp, BumpFootprintExpirationOp, and RestoreFootprintOp does not prefetch any entries, leading to each entry in the footprint being fetched individually.
The text was updated successfully, but these errors were encountered: