Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scrypto compiler default options for tests #1807

Merged

Conversation

mstrug-rdx
Copy link
Contributor

@mstrug-rdx mstrug-rdx commented May 10, 2024

Summary

Added CompilerProfile to LedgerSimulator.

Details

Adding new scrypto-compiler crate introduced divergence between default scrypto build command compilation options and compilation options used by LedgerSimulator and SDK in scrypto-test project. This PR provides option for the users of LedgerSimulator to choose if they want to use standard scrypto build compilation options or faster compilation options (WASM optimizations are disabled in that case).
Also refactored PackageFactory to use same compiler object as LedgerSimulator.

Testing

Added new tests which verifies if blueprint compiled using scrypto build and LedgerSimulator with Standard compilation options are binary same. Also added other validation tests.

Copy link

github-actions bot commented May 10, 2024

Docker tags
docker.io/radixdlt/private-scrypto-builder:3e28794154

Copy link

github-actions bot commented May 10, 2024

Benchmark for 3e28794

Click to view benchmark
Test Base PR %
costing::bench_prepare_wasm 65.6±0.49ms 65.6±0.15ms 0.00%
costing::decode_sbor 10.8±0.05µs 10.9±0.02µs +0.93%
costing::decode_sbor_bytes 29.1±0.02µs 29.3±0.07µs +0.69%
costing::deserialize_wasm 1280.1±4.01µs 1287.9±2.36µs +0.61%
costing::instantiate_flash_loan 3.9±0.62ms 3.9±0.66ms 0.00%
costing::instantiate_radiswap 5.6±0.05ms 5.8±0.06ms +3.57%
costing::spin_loop 22.0±0.04ms 21.7±0.06ms -1.36%
costing::validate_sbor_payload 30.0±0.17µs 30.1±0.04µs +0.33%
costing::validate_sbor_payload_bytes 259.1±1.23ns 273.3±1.09ns +5.48%
costing::validate_secp256k1 76.3±0.05µs 76.4±0.80µs +0.13%
costing::validate_wasm 36.6±0.05ms 37.3±0.06ms +1.91%
decimal::add/0 8.4±0.00ns 8.4±0.06ns 0.00%
decimal::add/rust-native 9.8±0.01ns 9.8±0.01ns 0.00%
decimal::add/wasmer 119.2±0.07ns 119.4±0.22ns +0.17%
decimal::add/wasmer-call-native 448.8±0.85ns 450.3±0.66ns +0.33%
decimal::add/wasmi 670.0±1.20ns 638.6±2.58ns -4.69%
decimal::add/wasmi-call-native 5.4±0.01µs 5.4±0.01µs 0.00%
decimal::div/0 190.1±0.12ns 188.9±0.12ns -0.63%
decimal::from_string/0 157.2±0.33ns 156.4±0.27ns -0.51%
decimal::mul/0 142.0±0.07ns 139.8±0.22ns -1.55%
decimal::mul/rust-native 137.6±0.06ns 136.8±0.14ns -0.58%
decimal::mul/wasmer 1503.3±0.97ns 1525.3±1.27ns +1.46%
decimal::mul/wasmer-call-native 582.9±0.51ns 579.3±2.86ns -0.62%
decimal::mul/wasmi 41.0±0.12µs 41.6±0.22µs +1.46%
decimal::mul/wasmi-call-native 5.6±0.02µs 5.6±0.01µs 0.00%
decimal::pow/0 651.9±0.65ns 656.0±0.59ns +0.63%
decimal::pow/rust-native 631.0±0.36ns 630.7±0.41ns -0.05%
decimal::pow/wasmer 6.6±0.00µs 6.6±0.01µs 0.00%
decimal::pow/wasmer-call-native 1023.3±0.79ns 1024.4±1.19ns +0.11%
decimal::pow/wasmi 194.2±0.26µs 195.0±0.59µs +0.41%
decimal::pow/wasmi-call-native 5.3±0.01µs 5.3±0.01µs 0.00%
decimal::root/0 7.9±0.01µs 8.1±0.01µs +2.53%
decimal::sub/0 8.5±0.01ns 8.5±0.01ns 0.00%
decimal::to_string/0 438.9±0.16ns 439.8±0.27ns +0.21%
precise_decimal::add/0 10.0±0.01ns 9.4±0.06ns -6.00%
precise_decimal::add/rust-native 11.4±0.00ns 11.4±0.00ns 0.00%
precise_decimal::add/wasmer 117.3±0.15ns 121.6±0.27ns +3.67%
precise_decimal::add/wasmer-call-native 501.6±0.55ns 510.8±2.13ns +1.83%
precise_decimal::add/wasmi 799.7±2.24ns 806.2±2.57ns +0.81%
precise_decimal::add/wasmi-call-native 7.0±0.02µs 7.0±0.02µs 0.00%
precise_decimal::div/0 301.2±0.33ns 303.2±0.34ns +0.66%
precise_decimal::from_string/0 195.0±0.09ns 193.0±0.33ns -1.03%
precise_decimal::mul/0 345.2±2.33ns 349.4±2.51ns +1.22%
precise_decimal::mul/rust-native 308.7±3.22ns 309.0±2.94ns +0.10%
precise_decimal::mul/wasmer 3.4±0.00µs 3.4±0.00µs 0.00%
precise_decimal::mul/wasmer-call-native 870.8±0.78ns 826.6±1.21ns -5.08%
precise_decimal::mul/wasmi 106.5±0.13µs 105.5±0.29µs -0.94%
precise_decimal::mul/wasmi-call-native 7.4±0.01µs 7.4±0.05µs 0.00%
precise_decimal::pow/0 1859.0±3.76ns 1858.3±3.15ns -0.04%
precise_decimal::pow/rust-native 1490.1±4.29ns 1487.8±3.64ns -0.15%
precise_decimal::pow/wasmer 16.1±0.01µs 16.1±0.01µs 0.00%
precise_decimal::pow/wasmer-call-native 2.2±0.00µs 2.1±0.00µs -4.55%
precise_decimal::pow/wasmi 516.6±1.62µs 509.5±1.37µs -1.37%
precise_decimal::pow/wasmi-call-native 13.1±0.03µs 13.2±0.03µs +0.76%
precise_decimal::root/0 57.4±0.17µs 56.9±0.02µs -0.87%
precise_decimal::sub/0 9.6±0.02ns 9.5±0.00ns -1.04%
precise_decimal::to_string/0 728.8±1.60ns 722.6±1.09ns -0.85%
schema::validate_payload 344.3±0.40µs 342.2±0.39µs -0.61%
transaction::radiswap 5.4±0.03ms 5.4±0.03ms 0.00%
transaction::transfer 1764.7±4.16µs 1771.5±9.34µs +0.39%
transaction_processing::prepare 2.2±0.00ms 2.2±0.00ms 0.00%
transaction_processing::prepare_and_decompile 6.1±0.02ms 6.2±0.01ms +1.64%
transaction_processing::prepare_and_decompile_and_recompile 24.2±0.12ms 28.1±3.12ms +16.12%
transaction_validation::validate_manifest 42.1±0.05µs 42.1±0.04µs 0.00%
transaction_validation::verify_bls_2KB 1013.1±65.85µs 1002.9±64.98µs -1.01%
transaction_validation::verify_bls_32B 967.1±12.96µs 961.7±7.82µs -0.56%
transaction_validation::verify_ecdsa 74.2±0.08µs 74.2±0.12µs 0.00%
transaction_validation::verify_ed25519 55.0±0.24µs 55.0±0.05µs 0.00%

@mstrug-rdx mstrug-rdx changed the base branch from develop to release/bottlenose May 13, 2024 10:28
@mstrug-rdx mstrug-rdx marked this pull request as ready for review May 13, 2024 11:19
radix-engine-tests/src/common.rs Outdated Show resolved Hide resolved
radix-engine-tests/src/common.rs Outdated Show resolved Hide resolved

env_vars.iter().for_each(|(name, value)| {
compiler_builder.env(name, EnvironmentVariableAction::Set(value.clone()));
});

#[cfg(feature = "coverage")]
{
if _use_coverage {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need to feature flag?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, because it changes some limits to larger values in transaction executor and wasm executor. Also there was missing coverage feature in scrypto-test project.
Additionally I've fixed scrypto coverage command memory error which was caused by using coverage feature and Rustflags on 1st compilation stage (schema extract).

scrypto-test/src/ledger_simulator/ledger_simulator.rs Outdated Show resolved Hide resolved
@mstrug-rdx mstrug-rdx force-pushed the refactor/test-package-compile-default-opt branch from f97fe99 to 6457b8e Compare May 13, 2024 13:34
@@ -1218,4 +1262,36 @@ mod tests {
assert_eq!(cmd_to_string(&cmd_phase_2),
format!("cargo build --target wasm32-unknown-unknown --target-dir {} --manifest-path {} --features scrypto/log-error --features scrypto/log-warn --features scrypto/log-info --features scrypto/no-schema --profile release", default_target_path.display(), manifest_path.display()));
}

#[test]
fn test_command_coverage() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding a coverage test as well!

@mstrug-rdx mstrug-rdx merged commit 5595a9a into release/bottlenose May 14, 2024
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants