-
Notifications
You must be signed in to change notification settings - Fork 208
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement set reconciliation based sync (syncv2) #5769
base: develop
Are you sure you want to change the base?
Conversation
Parent links make it impossible to implement (semi-)persistent mode
Not tested yet
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #5769 +/- ##
=========================================
- Coverage 80.7% 80.0% -0.8%
=========================================
Files 288 292 +4
Lines 29894 30769 +875
=========================================
+ Hits 24149 24620 +471
- Misses 4158 4434 +276
- Partials 1587 1715 +128 ☔ View full report in Codecov by Sentry. |
@mathcrypto the discussion was not finished yet, but I restarted it #5785 (comment) |
Existing fetcher will be used for blobs
Also, fix a race in syncTree and go version for proper loop var capture handling.
Motivation
Existing sync is causing a lot of resource usage, including memory consumption and network traffic, and can't replace gossip which can also be unreliable at times.
Description
This implements the new sync protocol as described here.
The algorithm is based on Range-Based Set Reconciliation paper by Aljoscha Meyer and also loosely based on SREP: Out-Of-Band Sync of Transaction Pools for Large-Scale Blockchains paper by Novak Boškov, Sevval Simsek, Ari Trachtenberg, and David Starobinski.
Test Plan
Add systests that verify new sync.
Verify on testnet, then on mainnet.
TODO
atxsdata