[Fuzzing] Do publish with official flow #13282

khokho · 2024-05-14T21:56:52Z

Description

Updated the fuzzer to use official publish flow for deploying module
Additional improvements which improve performance by a good margin at an expense of small patch to FakeExecutor
Verify the modules with production config to reject bad modules quicker

Type of Change

Which Components or Systems Does This Change Impact?

How Has This Been Tested?

N/A

Key Areas to Review

The patch to FakeExecutor and it's feature gating.

Checklist

I have read and followed the CONTRIBUTING doc
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I identified and added all stakeholders and component owners affected by this change as reviewers
I tested both happy and unhappy path of the functionality
I have made corresponding changes to the documentation

trunk-io · 2024-05-14T21:56:56Z

⏱️ 12h 55m total CI duration on this PR

Job	Cumulative Duration	Recent Runs
forge-framework-upgrade-test / forge	2h 48m	🟥 🟩
execution-performance / single-node-performance	1h 42m	🟩 🟩 🟩 🟩
rust-smoke-tests	1h 38m	🟩 ⬜ 🟩 🟩
rust-targeted-unit-tests	52m	⬜ 🟩 ⬜ 🟩 🟩
windows-build	45m	🟩
rust-move-tests	44m	⬜ 🟩 ⬜ 🟩 🟩
forge-e2e-test / forge	42m	🟩 🟩 🟩
forge-compat-test / forge	39m	🟩 🟩 🟩
rust-images / rust-all	37m	🟩 🟩 🟩
cli-e2e-tests / run-cli-tests	24m	🟥 🟥 🟩
rust-lints	24m	⬜ 🟩 ⬜ 🟩 🟩
execution-performance / test-target-determinator	17m	🟩 🟩 🟩 🟩
rust-build-cached-packages	14m	🟩 ⬜ 🟩 🟩
check	13m	🟩 ⬜ 🟩 🟩
test-target-determinator	13m	🟩 🟩 🟩
run-tests-main-branch	12m	🟩 🟩 🟩
general-lints	9m	🟩 🟩 ⬜ 🟩 🟩
check-dynamic-deps	7m	🟩 🟩 🟩 🟩 🟩
indexer-grpc-e2e-tests / test-indexer-grpc-docker-compose	7m	🟩 🟩 🟥 🟩
node-api-compatibility-tests / node-api-compatibility-tests	3m	🟩 🟩 🟩
semgrep/ci	2m	🟩 🟩 🟩 🟩 🟩
file_change_determinator	59s	🟩 🟩 🟩 🟩 🟩
file_change_determinator	35s	🟩 🟩 🟩
file_change_determinator	33s	🟩 🟩 🟩
permission-check	15s	🟥 🟩 🟥 🟩 🟩
permission-check	12s	🟥 🟩 🟥 🟩 🟩
permission-check	11s	🟥 🟩 🟥 🟩 🟩
permission-check	11s	🟥 🟩 🟥 🟩 🟩
permission-check	9s	🟩 🟥 🟩 🟩
determine-docker-build-metadata	8s	🟩 🟩 🟩

🚨 2 jobs on the last run were significantly faster/slower than expected

Job	Duration	vs 7d avg	Delta
rust-build-cached-packages	4m	5m
rust-targeted-unit-tests	13m	19m

_{settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io}

georgemitenkov

Probably module deserialization needs fixing, but otherwise looks good, thanks!

testsuite/fuzzer/fuzz/fuzz_targets/move/aptosvm_publish_and_run.rs

testsuite/fuzzer/fuzz/Cargo.toml

testsuite/fuzzer/fuzz/fuzz_targets/move/aptosvm_publish_and_run.rs

github-actions · 2024-05-15T14:04:30Z

✅ Forge suite `compat` success on `01b24e7e3548382dd25440b39a0438a993387f12` ==> `537572776f5dfe03a485c1053345a8e7150d310b`

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 537572776f5dfe03a485c1053345a8e7150d310b (PR)
1. Check liveness of validators at old version: 01b24e7e3548382dd25440b39a0438a993387f12
compatibility::simple-validator-upgrade::liveness-check : committed: 6546.1876728767875 txn/s, latency: 5003.063015231047 ms, (p50: 4800 ms, p90: 8600 ms, p99: 9900 ms), latency samples: 232420
2. Upgrading first Validator to new version: 537572776f5dfe03a485c1053345a8e7150d310b
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1814.843001650385 txn/s, latency: 15797.582871703966 ms, (p50: 18700 ms, p90: 21400 ms, p99: 22600 ms), latency samples: 90260
3. Upgrading rest of first batch to new version: 537572776f5dfe03a485c1053345a8e7150d310b
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1837.6555863153417 txn/s, latency: 15694.792847840243 ms, (p50: 18700 ms, p90: 21900 ms, p99: 22300 ms), latency samples: 92140
4. upgrading second batch to new version: 537572776f5dfe03a485c1053345a8e7150d310b
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3529.2642656590024 txn/s, latency: 8850.64761871894 ms, (p50: 9600 ms, p90: 12600 ms, p99: 12800 ms), latency samples: 144880
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 537572776f5dfe03a485c1053345a8e7150d310b passed
Test Ok

github-actions · 2024-05-15T14:05:43Z

✅ Forge suite `realistic_env_max_load` success on `537572776f5dfe03a485c1053345a8e7150d310b`

two traffics test: inner traffic : committed: 7648.645232402264 txn/s, latency: 5118.158863161202 ms, (p50: 4800 ms, p90: 6000 ms, p99: 11100 ms), latency samples: 3309880
two traffics test : committed: 99.94462019769514 txn/s, latency: 1850.5362637362637 ms, (p50: 1800 ms, p90: 2000 ms, p99: 3100 ms), latency samples: 1820
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.214, avg: 0.204", "QsPosToProposal: max: 0.338, avg: 0.244", "ConsensusProposalToOrdered: max: 0.458, avg: 0.420", "ConsensusOrderedToCommit: max: 0.398, avg: 0.352", "ConsensusProposalToCommit: max: 0.837, avg: 0.771"]
Max round gap was 1 [limit 4] at version 27141. Max no progress secs was 4.489168 [limit 15] at version 1650456.
Test Ok

github-actions · 2024-05-15T19:00:35Z

✅ Forge suite `framework_upgrade` success on `01b24e7e3548382dd25440b39a0438a993387f12` ==> `537572776f5dfe03a485c1053345a8e7150d310b`

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 537572776f5dfe03a485c1053345a8e7150d310b (PR)
Upgrade the nodes to version: 537572776f5dfe03a485c1053345a8e7150d310b
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1411.187824501395 txn/s, submitted: 1413.2862450582745 txn/s, failed submission: 2.0984205568793977 txn/s, expired: 2.0984205568793977 txn/s, latency: 2542.548596654275 ms, (p50: 2100 ms, p90: 4500 ms, p99: 6500 ms), latency samples: 107600
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1161.5043679569064 txn/s, submitted: 1164.2065882778609 txn/s, failed submission: 2.702220320954416 txn/s, expired: 2.702220320954416 txn/s, latency: 2624.4089860411013 ms, (p50: 2300 ms, p90: 4400 ms, p99: 6000 ms), latency samples: 103160
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 537572776f5dfe03a485c1053345a8e7150d310b passed
Upgrade the remaining nodes to version: 537572776f5dfe03a485c1053345a8e7150d310b
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1143.1570478470153 txn/s, submitted: 1145.8290160149475 txn/s, failed submission: 2.6719681679322522 txn/s, expired: 2.6719681679322522 txn/s, latency: 2769.6712991819245 ms, (p50: 2400 ms, p90: 4600 ms, p99: 6900 ms), latency samples: 102680
Test Ok

gedigi · 2024-05-15T23:46:07Z

testsuite/fuzzer/fuzz/fuzz_targets/move/aptosvm_publish_and_run.rs

+        move_bytecode_verifier::verify_module_with_config(&vm_config.verifier, &m_de).map_err(
+            |e| {
+                check_for_invariant_violation_vmerror(e);
+                Corpus::Keep


why do we keep the corpus data if it produces invalid modules?

[Fuzzing] Publish with official flow; Minor improvements;

f369680

zi0Black approved these changes May 15, 2024

View reviewed changes

zi0Black requested review from gedigi and georgemitenkov May 15, 2024 03:41

zi0Black added the CICD:run-e2e-tests when this label is present github actions will run all land-blocking e2e tests from the PR label May 15, 2024

This comment has been minimized.

Sign in to view

georgemitenkov approved these changes May 15, 2024

View reviewed changes

testsuite/fuzzer/fuzz/fuzz_targets/move/aptosvm_publish_and_run.rs Outdated Show resolved Hide resolved

testsuite/fuzzer/fuzz/fuzz_targets/move/aptosvm_publish_and_run.rs Outdated Show resolved Hide resolved