Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[replay-verify] rebalance the ranges #13272

Merged
merged 1 commit into from
May 15, 2024
Merged

[replay-verify] rebalance the ranges #13272

merged 1 commit into from
May 15, 2024

Conversation

areshand
Copy link
Contributor

Description

Type of Change

  • New feature
  • Bug fix
  • Breaking change
  • Performance improvement
  • Refactoring
  • Dependency update
  • Documentation update
  • Tests

Which Components or Systems Does This Change Impact?

  • Validator Node
  • Full Node (API, Indexer, etc.)
  • Move/Aptos Virtual Machine
  • Aptos Framework
  • Aptos CLI/SDK
  • Developer Infrastructure
  • Other (specify)

How Has This Been Tested?

Key Areas to Review

Checklist

  • I have read and followed the CONTRIBUTING doc
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I identified and added all stakeholders and component owners affected by this change as reviewers
  • I tested both happy and unhappy path of the functionality
  • I have made corresponding changes to the documentation

Copy link

trunk-io bot commented May 13, 2024

⏱️ 96h 35m total CI duration on this PR
Job Cumulative Duration Recent Runs
test-replay / replay-verify (16) 6h 2m 🟥🟥🟥
test-replay / replay-verify (10) 6h 1m 🟥🟥🟥
test-replay / replay-verify (9) 6h 1m 🟥🟥🟥
test-replay / replay-verify (8) 5h 58m 🟩🟥🟥
test-replay / replay-verify (15) 5h 27m 🟩🟩🟥
test-replay / replay-verify (11) 5h 13m 🟩🟩🟥
test-replay / replay-verify (17) 4h 57m 🟥🟥🟥
test-replay / replay-verify (6) 4h 39m 🟩🟩🟥
test-replay / replay-verify (0) 4h 37m 🟩🟩🟥
test-replay / replay-verify (12) 4h 32m 🟩🟩🟥
test-replay / replay-verify (2) 4h 23m 🟩🟩🟥
test-replay / replay-verify (13) 4h 16m 🟩🟩🟥
test-replay / replay-verify (4) 4h 4m 🟩🟩🟥
test-replay / replay-verify (14) 3h 52m 🟩🟩🟥
test-replay / replay-verify (7) 3h 44m 🟩🟩🟥
test-replay / replay-verify (5) 3h 40m 🟩🟩🟥
test-replay / replay-verify (3) 3h 39m 🟩🟩🟥
test-replay / replay-verify (1) 3h 4m 🟩🟩🟩
windows-build 2h 2m 🟩🟩🟩
rust-targeted-unit-tests 1h 58m 🟩🟩🟩🟩
rust-smoke-tests 1h 51m 🟩🟩🟩
forge-framework-upgrade-test / forge 1h 12m 🟩
rust-move-tests 48m 🟩🟩🟩🟩
forge-e2e-test / forge 43m 🟩🟩🟩
forge-compat-test / forge 43m 🟩🟩🟩
rust-images / rust-all 37m 🟩🟩🟩
rust-lints 27m 🟩🟩🟩🟩
execution-performance / single-node-performance 23m 🟩
rust-build-cached-packages 20m 🟩🟩🟩
cli-e2e-tests / run-cli-tests 17m 🟩🟩🟩
run-tests-main-branch 17m 🟩🟩🟩🟩
check 12m 🟩🟩🟩
test-target-determinator 9m 🟥🟥🟩
execution-performance / test-target-determinator 8m 🟥🟥🟩
general-lints 6m 🟥🟥🟩🟩
check-dynamic-deps 5m 🟩🟩🟩🟩
node-api-compatibility-tests / node-api-compatibility-tests 3m 🟩🟩🟩
semgrep/ci 2m 🟩🟩🟩🟩
file_change_determinator 47s 🟩🟩🟩🟩
file_change_determinator 39s 🟩🟩🟩🟩
file_change_determinator 35s 🟩🟩🟩
determine-test-metadata 18s 🟩🟩🟩
permission-check 13s 🟩🟩🟩🟩
permission-check 11s 🟩🟩🟩🟩
permission-check 11s 🟩🟩🟩🟩
permission-check 8s 🟩🟩🟩🟩
permission-check 7s 🟩🟩🟩
determine-docker-build-metadata 7s 🟩🟩🟩

🚨 1 job on the last run was significantly faster/slower than expected

Job Duration vs 7d avg Delta
rust-targeted-unit-tests 42m 19m +120%

settingsfeedbackdocs ⋅ learn more about trunk.io

@areshand areshand marked this pull request as ready for review May 13, 2024 23:27
@areshand areshand requested a review from a team as a code owner May 13, 2024 23:27
@areshand areshand requested review from msmouse and grao1991 May 13, 2024 23:27
@areshand
Copy link
Contributor Author

balance the ranges before adding snapshots. please take a look @msmouse @grao1991

@msmouse
Copy link
Contributor

msmouse commented May 13, 2024

Glad to see we are "sealing" the last range and open a new (open ended) one as we designed 😂

title isn't entirely accurate though

@areshand areshand enabled auto-merge (rebase) May 14, 2024 16:20

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

✅ Forge suite compat success on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 4d55b10f09a10ca63e8f248e4f70decea2cf49ce

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 4d55b10f09a10ca63e8f248e4f70decea2cf49ce (PR)
1. Check liveness of validators at old version: 01b24e7e3548382dd25440b39a0438a993387f12
compatibility::simple-validator-upgrade::liveness-check : committed: 5529.8080730079 txn/s, latency: 5859.631841173496 ms, (p50: 5400 ms, p90: 9300 ms, p99: 13800 ms), latency samples: 197700
2. Upgrading first Validator to new version: 4d55b10f09a10ca63e8f248e4f70decea2cf49ce
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1748.249708370791 txn/s, latency: 15943.281890019485 ms, (p50: 18600 ms, p90: 22000 ms, p99: 22800 ms), latency samples: 92380
3. Upgrading rest of first batch to new version: 4d55b10f09a10ca63e8f248e4f70decea2cf49ce
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1830.9654849557526 txn/s, latency: 15604.54144693833 ms, (p50: 19100 ms, p90: 22000 ms, p99: 22600 ms), latency samples: 91780
4. upgrading second batch to new version: 4d55b10f09a10ca63e8f248e4f70decea2cf49ce
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 2835.1713715348565 txn/s, latency: 10571.597453703704 ms, (p50: 9900 ms, p90: 12700 ms, p99: 13900 ms), latency samples: 120960
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 4d55b10f09a10ca63e8f248e4f70decea2cf49ce passed
Test Ok

Copy link
Contributor

✅ Forge suite realistic_env_max_load success on 4d55b10f09a10ca63e8f248e4f70decea2cf49ce

two traffics test: inner traffic : committed: 8254.235839053596 txn/s, latency: 4748.316605714734 ms, (p50: 4500 ms, p90: 5600 ms, p99: 10800 ms), latency samples: 3571120
two traffics test : committed: 99.93864929799486 txn/s, latency: 1852.4272222222223 ms, (p50: 1800 ms, p90: 2100 ms, p99: 4700 ms), latency samples: 1800
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.210, avg: 0.203", "QsPosToProposal: max: 0.229, avg: 0.209", "ConsensusProposalToOrdered: max: 0.473, avg: 0.415", "ConsensusOrderedToCommit: max: 0.392, avg: 0.372", "ConsensusProposalToCommit: max: 0.800, avg: 0.787"]
Max round gap was 1 [limit 4] at version 1336022. Max no progress secs was 4.926443 [limit 15] at version 1336022.
Test Ok

Copy link
Contributor

✅ Forge suite framework_upgrade success on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 4d55b10f09a10ca63e8f248e4f70decea2cf49ce

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 4d55b10f09a10ca63e8f248e4f70decea2cf49ce (PR)
Upgrade the nodes to version: 4d55b10f09a10ca63e8f248e4f70decea2cf49ce
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1301.4242753996984 txn/s, submitted: 1303.0124849408069 txn/s, failed submission: 1.5882095411084187 txn/s, expired: 1.5882095411084187 txn/s, latency: 2367.3992764993027 ms, (p50: 1800 ms, p90: 4200 ms, p99: 6000 ms), latency samples: 114720
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 928.0385791404296 txn/s, submitted: 930.5862268495749 txn/s, failed submission: 2.5476477091451777 txn/s, expired: 2.5476477091451777 txn/s, latency: 3397.773321687048 ms, (p50: 2400 ms, p90: 7400 ms, p99: 11800 ms), latency samples: 80140
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 4d55b10f09a10ca63e8f248e4f70decea2cf49ce passed
Upgrade the remaining nodes to version: 4d55b10f09a10ca63e8f248e4f70decea2cf49ce
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1241.0882407640981 txn/s, submitted: 1242.9862919830073 txn/s, failed submission: 1.8980512189089631 txn/s, expired: 1.8980512189089631 txn/s, latency: 2698.5305390938634 ms, (p50: 2400 ms, p90: 4500 ms, p99: 6300 ms), latency samples: 104620
Test Ok

@areshand areshand merged commit 921b829 into main May 15, 2024
55 of 73 checks passed
@areshand areshand deleted the rebalance_ranges branch May 15, 2024 23:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants