-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
erigon-lib/state: inverted index to use elias fano reverse iterator seek #10397
base: main
Are you sure you want to change the base?
Conversation
- Reduces memory allocations for reverse iterator from 49681997 B/op to 144 B/op - Adds support for reverse binary search Seek to reverse iterator (we can use that in `FrozenInvertedIdxIter.advanceInFiles` for more performance - will add in a subsequent [PR](#10397)) ``` ➜ go test -bench=. -benchmem goos: darwin goarch: arm64 pkg: github.com/ledgerwatch/erigon-lib/recsplit/eliasfano32 BenchmarkName/next_to_value_1_000_000-8 1113 1091076 ns/op 144 B/op 1 allocs/op BenchmarkName/seek_to_value_1_000_000-8 2000680 601.1 ns/op 144 B/op 1 allocs/op BenchmarkName/reverse_next_to_value_1_230-8 8 129665120 ns/op 152 B/op 2 allocs/op BenchmarkName/reverse_seek_to_value_1_230-8 1000000 1062 ns/op 152 B/op 2 allocs/op BenchmarkName/naive_reverse_iterator-8 8 135305406 ns/op 49681997 B/op 42 allocs/op BenchmarkName/reverse_iterator-8 8 127542469 ns/op 144 B/op 1 allocs/op PASS ok github.com/ledgerwatch/erigon-lib/recsplit/eliasfano32 12.937s ```
will you take a followup issue on implementing Seek with new descending iterator? |
@awskii I need to play around with unwind first and learn how it works a bit better. |
pending on #10395
Seek is supported for reverse binary search which is a lot more efficient in to getting to the correct iterator start point than using
it.Next
andcontinue