Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[gas] allow higher limits for approved governance proposals #13244

Merged
merged 2 commits into from
May 15, 2024

Conversation

vgao1996
Copy link
Contributor

@vgao1996 vgao1996 commented May 10, 2024

Description

This adds separate limits that applies to approved governance proposals in all gas categories (exec, io & storage), eliminating the need to relax these limits during releases, which is error-prone.

Additionally, this also makes the transaction size limit for governance proposals configurable.

Type of Change

  • New feature
  • Bug fix
  • Breaking change
  • Performance improvement
  • Refactoring
  • Dependency update
  • Documentation update
  • Tests

Which Components or Systems Does This Change Impact?

  • Validator Node
  • Full Node (API, Indexer, etc.)
  • Move/Aptos Virtual Machine
  • Aptos Framework
  • Aptos CLI/SDK
  • Developer Infrastructure
  • Other (specify)

How Has This Been Tested?

E2E test added.

Key Areas to Review

  • aptos-gas-schedule
    • Adding new gas parameters, bumping the gas feature version
  • aptos-vm
    • Checking whether a transaction is a governance script
    • A bit more refactoring to eliminate duplicate logic

Checklist

  • I have read and followed the CONTRIBUTING doc
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I identified and added all stakeholders and component owners affected by this change as reviewers
  • I tested both happy and unhappy path of the functionality
  • I have made corresponding changes to the documentation

Copy link

trunk-io bot commented May 10, 2024

⏱️ 9h total CI duration on this PR
Job Cumulative Duration Recent Runs
rust-targeted-unit-tests 1h 35m 🟥🟩🟩🟩 (+2 more)
forge-framework-upgrade-test / forge 1h 19m 🟩
rust-move-tests 1h 4m 🟥🟩🟩🟩 (+2 more)
rust-move-unit-coverage 42m 🟩🟩
windows-build 41m 🟩
rust-smoke-tests 34m 🟩
run-tests-main-branch 30m 🟩🟩🟩🟩🟩 (+2 more)
rust-lints 29m 🟥🟥🟩🟩 (+2 more)
execution-performance / single-node-performance 25m 🟩
forge-compat-test / forge 18m 🟩
forge-e2e-test / forge 16m 🟩
rust-images / rust-all 13m 🟩
general-lints 11m 🟩🟩🟩🟩 (+2 more)
check-dynamic-deps 10m 🟩🟩🟩🟩🟩 (+2 more)
cli-e2e-tests / run-cli-tests 9m 🟥
rust-build-cached-packages 5m 🟩
test-target-determinator 4m 🟩
execution-performance / test-target-determinator 4m 🟩
check 4m 🟩
semgrep/ci 3m 🟩🟩🟩🟩🟩 (+2 more)
file_change_determinator 1m 🟩🟩🟩🟩🟩 (+2 more)
file_change_determinator 1m 🟩🟩🟩🟩🟩 (+2 more)
node-api-compatibility-tests / node-api-compatibility-tests 49s 🟩
permission-check 24s 🟩🟩🟩🟩🟩 (+2 more)
permission-check 21s 🟩🟩🟩🟩🟩 (+2 more)
permission-check 18s 🟩🟩🟩🟩🟩 (+2 more)
permission-check 17s 🟩🟩🟩🟩🟩 (+2 more)
file_change_determinator 9s 🟩
determine-docker-build-metadata 8s 🟩
permission-check 2s 🟩

🚨 3 jobs on the last run were significantly faster/slower than expected

Job Duration vs 7d avg Delta
rust-move-tests 17m 12m +41%
cli-e2e-tests / run-cli-tests 9m 7m +34%
forge-compat-test / forge 18m 14m +33%

settingsfeedbackdocs ⋅ learn more about trunk.io

@vgao1996 vgao1996 force-pushed the relax-gov-exec-limit branch 3 times, most recently from e7fd32a to dc61d12 Compare May 10, 2024 18:52
@vgao1996 vgao1996 marked this pull request as ready for review May 13, 2024 18:42
Copy link
Contributor

@georgemitenkov georgemitenkov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

aptos-move/e2e-tests/src/executor.rs Show resolved Hide resolved
aptos-move/e2e-move-tests/src/tests/governance_updates.rs Outdated Show resolved Hide resolved
aptos-move/e2e-move-tests/src/tests/governance_updates.rs Outdated Show resolved Hide resolved
))),
);
let gas_used = output.gas_used();
assert!(max_gas_regular - 1 <= gas_used && gas_used <= max_gas_regular + 1);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not gas_used == max_gas_regular? I do not understand why we allow off by 1 here...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. I was originally thinking that there was an off by 1 error caused by rounding, but that was the wrong reason.

The reason why we're having this is because when a transaction gets terminated due to hitting execution gas limit, there could an overshoot, as long as the overall gas budget hasn't been depleted.

I've changed the formula to let overshoot = (max_gas_regular.min(max_gas_gov) / 5).max(1);, which should be a bit more robust than a constant value 1.

aptos-move/aptos-vm/src/gas.rs Outdated Show resolved Hide resolved
aptos-move/aptos-vm/src/aptos_vm.rs Outdated Show resolved Hide resolved
aptos-move/aptos-vm/src/aptos_vm.rs Outdated Show resolved Hide resolved
fn validate_signed_transaction(
&self,
session: &mut SessionExt,
resolver: &impl AptosMoveResolver,
transaction: &SignedTransaction,
transaction_data: &TransactionMetadata,
log_context: &AdapterLogSchema,
is_governance_proposal: bool,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be part of TransactionMetadata?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, because in order to determine if a transaction is an approved governance proposal, you need to check whether its hash exists in ApprovedExecutionHashes, which is not something you can derive from the transaction itself like other transaction metadata does.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All usages of is_governance_proposal() function are where we create the metadata, e.g.

let txn_metadata = TransactionMetadata::new(txn);
let is_governance_proposal = is_governance_proposal(resolver, txn, &txn_metadata);

This is true for exactly 2 places where we create metadata. It seems cleaner to have this as a field, instead of passing a flag everywhere, no?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, again I guess the real deciding factor is how we want to define transaction metadata.

  • If we want it to contain info strictly derivable from the transaction itself, then the is_governance_proposal flag has to be passed around
  • Otherwise, we may include it in the txn metadata, and deal with the implication of requiring a resolver to compute metadata.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tbh, I am not exactly sure what this metadata is, I think the idea was to have transaction-representation agnostic data structure for the VM, which also caches some information. We already store keyless/randomness related info there:

pub required_deposit: Option<u64>,
pub is_keyless: bool,

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, is_keyless is information you can derive from the txn itself. is_governance_proposal depends on the on-chain state.

aptos-move/aptos-vm/src/aptos_vm.rs Outdated Show resolved Hide resolved
@vgao1996
Copy link
Contributor Author

@georgemitenkov I've addressed pretty much all of your comments. The new changes are in a separate commit. Please take another look!

@vgao1996 vgao1996 enabled auto-merge (squash) May 15, 2024 21:35

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

✅ Forge suite realistic_env_max_load success on 639d2b97e1939bd6f809d1f94b4f645c272a58e2

two traffics test: inner traffic : committed: 7328.896468660417 txn/s, latency: 5345.417676003028 ms, (p50: 5100 ms, p90: 6500 ms, p99: 10900 ms), latency samples: 3170400
two traffics test : committed: 100.0904278559291 txn/s, latency: 2070.687078651685 ms, (p50: 1900 ms, p90: 2300 ms, p99: 5300 ms), latency samples: 1780
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.210, avg: 0.206", "QsPosToProposal: max: 0.382, avg: 0.264", "ConsensusProposalToOrdered: max: 0.500, avg: 0.456", "ConsensusOrderedToCommit: max: 0.426, avg: 0.405", "ConsensusProposalToCommit: max: 0.912, avg: 0.861"]
Max round gap was 1 [limit 4] at version 1283308. Max no progress secs was 4.640541 [limit 15] at version 1283308.
Test Ok

Copy link
Contributor

✅ Forge suite compat success on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 639d2b97e1939bd6f809d1f94b4f645c272a58e2

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 639d2b97e1939bd6f809d1f94b4f645c272a58e2 (PR)
1. Check liveness of validators at old version: 01b24e7e3548382dd25440b39a0438a993387f12
compatibility::simple-validator-upgrade::liveness-check : committed: 4313.722846131892 txn/s, latency: 5765.568030526834 ms, (p50: 5100 ms, p90: 7700 ms, p99: 12400 ms), latency samples: 203100
2. Upgrading first Validator to new version: 639d2b97e1939bd6f809d1f94b4f645c272a58e2
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1702.6971992388494 txn/s, latency: 17263.97914322804 ms, (p50: 19000 ms, p90: 24400 ms, p99: 24700 ms), latency samples: 92440
3. Upgrading rest of first batch to new version: 639d2b97e1939bd6f809d1f94b4f645c272a58e2
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1266.7673294967906 txn/s, latency: 21662.192310855262 ms, (p50: 25000 ms, p90: 29000 ms, p99: 29700 ms), latency samples: 72960
4. upgrading second batch to new version: 639d2b97e1939bd6f809d1f94b4f645c272a58e2
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3538.0358513777555 txn/s, latency: 8829.80456118494 ms, (p50: 9600 ms, p90: 12600 ms, p99: 12900 ms), latency samples: 144480
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 639d2b97e1939bd6f809d1f94b4f645c272a58e2 passed
Test Ok

Copy link
Contributor

✅ Forge suite framework_upgrade success on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 639d2b97e1939bd6f809d1f94b4f645c272a58e2

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 639d2b97e1939bd6f809d1f94b4f645c272a58e2 (PR)
Upgrade the nodes to version: 639d2b97e1939bd6f809d1f94b4f645c272a58e2
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1291.9685275710865 txn/s, submitted: 1294.6392635453935 txn/s, failed submission: 2.6707359743071555 txn/s, expired: 2.6707359743071555 txn/s, latency: 2316.791912144703 ms, (p50: 1800 ms, p90: 3900 ms, p99: 5700 ms), latency samples: 116100
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 519.2960382303402 txn/s, submitted: 520.1882994644129 txn/s, failed submission: 0.8922612340727495 txn/s, expired: 0.8922612340727495 txn/s, latency: 6018.648217353952 ms, (p50: 6500 ms, p90: 9800 ms, p99: 17300 ms), latency samples: 46560
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 639d2b97e1939bd6f809d1f94b4f645c272a58e2 passed
Upgrade the remaining nodes to version: 639d2b97e1939bd6f809d1f94b4f645c272a58e2
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1291.9240149130133 txn/s, submitted: 1294.8586667029608 txn/s, failed submission: 2.9346517899474356 txn/s, expired: 2.9346517899474356 txn/s, latency: 2507.6755460422855 ms, (p50: 2100 ms, p90: 4500 ms, p99: 6700 ms), latency samples: 114460
Test Ok

@vgao1996 vgao1996 merged commit e54d82f into aptos-labs:main May 15, 2024
98 of 99 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants