Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add aggregator_v2::is_at_least API #13246

Merged
merged 2 commits into from
May 28, 2024
Merged

Add aggregator_v2::is_at_least API #13246

merged 2 commits into from
May 28, 2024

Conversation

igor-aptos
Copy link
Contributor

We can have an efficient aggregator_v2::is_at_least call (and is cheaper than equivalent try_sub() + add() calls), which can then be used to check for minimal balance, or other things.

we don't need really is_at_most(value), as that is really equivalent to !is_at_least(value + 1).

Adding also constructors with value, for now just as move utility, we can see if we want to make them a single native call (instead of currently two). but creation of aggregators is much less frequent, so should be fine.

Description

Type of Change

  • New feature
  • Bug fix
  • Breaking change
  • Performance improvement
  • Refactoring
  • Dependency update
  • Documentation update
  • Tests

Which Components or Systems Does This Change Impact?

  • Validator Node
  • Full Node (API, Indexer, etc.)
  • Move/Aptos Virtual Machine
  • Aptos Framework
  • Aptos CLI/SDK
  • Developer Infrastructure
  • Other (specify)

How Has This Been Tested?

Key Areas to Review

Checklist

  • I have read and followed the CONTRIBUTING doc
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I identified and added all stakeholders and component owners affected by this change as reviewers
  • I tested both happy and unhappy path of the functionality
  • I have made corresponding changes to the documentation

Copy link

trunk-io bot commented May 10, 2024

⏱️ 1h 2m total CI duration on this PR
Job Cumulative Duration Recent Runs
rust-targeted-unit-tests 21m 🟥
rust-move-unit-coverage 16m 🟩
rust-move-tests 10m 🟥
rust-lints 5m 🟩
run-tests-main-branch 4m 🟩
general-lints 2m 🟩
check-dynamic-deps 1m 🟩
semgrep/ci 33s 🟩
file_change_determinator 10s 🟩
file_change_determinator 9s 🟩
permission-check 3s 🟩
permission-check 3s 🟩
permission-check 3s 🟩
permission-check 2s 🟩

🚨 1 job on the last run was significantly faster/slower than expected

Job Duration vs 7d avg Delta
rust-move-unit-coverage 16m 22m -24%

settingsfeedbackdocs ⋅ learn more about trunk.io

Copy link

codecov bot commented May 10, 2024

Codecov Report

Attention: Patch coverage is 24.32432% with 56 lines in your changes are missing coverage. Please review.

Project coverage is 33.1%. Comparing base (d972e92) to head (9f49c9c).
Report is 6 commits behind head on main.

Files Patch % Lines
...rk/src/natives/aggregator_natives/aggregator_v2.rs 4.5% 42 Missing ⚠️
...s-gas-schedule/src/gas_schedule/aptos_framework.rs 0.0% 14 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##             main   #13246    +/-   ##
========================================
  Coverage    33.0%    33.1%            
========================================
  Files        1753     1753            
  Lines      337797   337871    +74     
========================================
+ Hits       111802   111928   +126     
+ Misses     225995   225943    -52     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@georgemitenkov georgemitenkov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, can we add tests though?


debug_assert_eq!(args.len(), 2);
debug_assert_eq!(ty_args.len(), 1);
context.charge(AGGREGATOR_V2_TRY_SUB_BASE)?;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is re-used because the cost is the same as sub?

@@ -68,7 +79,9 @@ impl DelayedFieldData {
DelayedChange::Create(DelayedFieldValue::Aggregator(value)) => {
match math.unsigned_add_delta(*value, &input) {
Ok(new_value) => {
*value = new_value;
if apply_delta {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Ok(new_value) if apply_delta => {}, _ => {}?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return is different?

@igor-aptos igor-aptos changed the base branch from main to igor/optimize_fa_apt May 22, 2024 21:56
/// which has no parallelism impact.
/// If called in a transaction that also modifies the aggregator, or has other read/write conflicts,
/// it will sequentialize that transaction. (i.e. up to concurrency_level times slower)
/// If called in a separate transaction (i.e. after transaction that modifies aggregator), it might be
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this based on measurements, or what's the reasoning here?

@@ -55,7 +66,7 @@ impl DelayedFieldData {
&input,
max_value,
)?;
if result {
if result && apply_delta {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume apply_delta = false is the new case, in which case is the behavior supposed to mirror apply_delta = true but without modifying self? Can w add one-liner comment?

I think apply_delta can and should be statically known parameter: every call is actually statically set true or false, and that way, with generics or macros we should be able to force self to be &mut or & based on the apply_delta value. Then the compiler will prove that when apply_delta = false, there can be no side effects, and it won't be error prone in the future to refactor this function and accidentally forget to guard mutations with apply_delta. It will also avoid the need of tests on apply_delta.

Of course we can also have separate methods, but that'd require duplication significant code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

synced up to try and do this in followup, if reasonably readable

add<Element>(addr, use_type, i, b);
}

// issue with type inference of lambda or downcasting from mut to non-mut?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lambda doesn't seem to work when first instruction infers &T, and second infers &mut T, it doesn't compile

max_value,
SignedU128::Negative(rhs),
resolver,
false,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this is the only use of 'false', and since here we only need to check if the value of the aggregator is at least rhs, wouldn't a simpler implementation that avoids modifying try_add_delta be to read lhs, and then share the code below with BoundedMath.unsigned_subtract? (the computation of lhs would be conditional - and that can be isolated and tested separately).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need to call resolver.delayed_field_try_add_delta_outcome to capture the restriction

@@ -270,6 +282,42 @@ fn native_try_sub(
Ok(smallvec![Value::bool(success)])
}

fn native_is_at_least_impl(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this (and can it, if not) have a simple unit test?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is unit tests on the delayed_field_extension.rs, and there are move e2e tests for native calls.

@igor-aptos igor-aptos force-pushed the igor/optimize_fa_apt branch 3 times, most recently from b2a904d to 4a19bb1 Compare May 28, 2024 14:37
Base automatically changed from igor/optimize_fa_apt to main May 28, 2024 18:03
@igor-aptos igor-aptos force-pushed the igor/is_at_least_api branch 2 times, most recently from 5e4724f to 386060c Compare May 28, 2024 21:36
@igor-aptos igor-aptos enabled auto-merge (squash) May 28, 2024 21:51

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

✅ Forge suite compat success on 3ffe0986b5fe4acb76544ae7ae85d73b91a6a411 ==> 9f49c9c3a0ec8b1c6b4188f5b8764d596aeee575

Compatibility test results for 3ffe0986b5fe4acb76544ae7ae85d73b91a6a411 ==> 9f49c9c3a0ec8b1c6b4188f5b8764d596aeee575 (PR)
1. Check liveness of validators at old version: 3ffe0986b5fe4acb76544ae7ae85d73b91a6a411
compatibility::simple-validator-upgrade::liveness-check : committed: 6860.744549413718 txn/s, latency: 4808.498442482217 ms, (p50: 4800 ms, p90: 7800 ms, p99: 8400 ms), latency samples: 244620
2. Upgrading first Validator to new version: 9f49c9c3a0ec8b1c6b4188f5b8764d596aeee575
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 2290.0810678330176 txn/s, latency: 13523.978279883382 ms, (p50: 14800 ms, p90: 17300 ms, p99: 19500 ms), latency samples: 96040
3. Upgrading rest of first batch to new version: 9f49c9c3a0ec8b1c6b4188f5b8764d596aeee575
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 3345.8645405707034 txn/s, latency: 9293.406157528285 ms, (p50: 9300 ms, p90: 13900 ms, p99: 14200 ms), latency samples: 137880
4. upgrading second batch to new version: 9f49c9c3a0ec8b1c6b4188f5b8764d596aeee575
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 4648.171510491932 txn/s, latency: 6959.451743162343 ms, (p50: 5100 ms, p90: 10800 ms, p99: 15800 ms), latency samples: 170380
5. check swarm health
Compatibility test for 3ffe0986b5fe4acb76544ae7ae85d73b91a6a411 ==> 9f49c9c3a0ec8b1c6b4188f5b8764d596aeee575 passed
Test Ok

Copy link
Contributor

✅ Forge suite realistic_env_max_load success on 9f49c9c3a0ec8b1c6b4188f5b8764d596aeee575

two traffics test: inner traffic : committed: 7911.17163078533 txn/s, latency: 4949.317853343154 ms, (p50: 4800 ms, p90: 6500 ms, p99: 10300 ms), latency samples: 3421320
two traffics test : committed: 99.88711593230741 txn/s, latency: 1892.5372093023257 ms, (p50: 1800 ms, p90: 2100 ms, p99: 4700 ms), latency samples: 1720
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.216, avg: 0.205", "QsPosToProposal: max: 0.303, avg: 0.243", "ConsensusProposalToOrdered: max: 0.453, avg: 0.415", "ConsensusOrderedToCommit: max: 0.405, avg: 0.389", "ConsensusProposalToCommit: max: 0.817, avg: 0.805"]
Max round gap was 1 [limit 4] at version 1765255. Max no progress secs was 4.675459 [limit 15] at version 1765255.
Test Ok

Copy link
Contributor

✅ Forge suite framework_upgrade success on 3ffe0986b5fe4acb76544ae7ae85d73b91a6a411 ==> 9f49c9c3a0ec8b1c6b4188f5b8764d596aeee575

Compatibility test results for 3ffe0986b5fe4acb76544ae7ae85d73b91a6a411 ==> 9f49c9c3a0ec8b1c6b4188f5b8764d596aeee575 (PR)
Upgrade the nodes to version: 9f49c9c3a0ec8b1c6b4188f5b8764d596aeee575
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1180.1167757448459 txn/s, submitted: 1182.8458952879057 txn/s, failed submission: 2.729119543059963 txn/s, expired: 2.729119543059963 txn/s, latency: 2751.4211986895357 ms, (p50: 2100 ms, p90: 4500 ms, p99: 8700 ms), latency samples: 103780
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1158.081371845732 txn/s, submitted: 1160.288502342865 txn/s, failed submission: 2.207130497133089 txn/s, expired: 2.207130497133089 txn/s, latency: 2578.8498761196875 ms, (p50: 2100 ms, p90: 4500 ms, p99: 8300 ms), latency samples: 104940
5. check swarm health
Compatibility test for 3ffe0986b5fe4acb76544ae7ae85d73b91a6a411 ==> 9f49c9c3a0ec8b1c6b4188f5b8764d596aeee575 passed
Upgrade the remaining nodes to version: 9f49c9c3a0ec8b1c6b4188f5b8764d596aeee575
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1168.3012156875072 txn/s, submitted: 1169.436149839137 txn/s, failed submission: 1.1349341516295972 txn/s, expired: 1.1349341516295972 txn/s, latency: 2635.2754711482416 ms, (p50: 2100 ms, p90: 4500 ms, p99: 7800 ms), latency samples: 102940
Test Ok

@igor-aptos igor-aptos merged commit d3bb8f6 into main May 28, 2024
53 of 54 checks passed
@igor-aptos igor-aptos deleted the igor/is_at_least_api branch May 28, 2024 23:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants