Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use node 18 #4779

Merged
merged 9 commits into from Dec 12, 2022
Merged

Use node 18 #4779

merged 9 commits into from Dec 12, 2022

Conversation

wemeetagain
Copy link
Member

Motivation

Resolve #4768

Description

Minimal diff to test/build/recommend using node 18

@wemeetagain wemeetagain requested a review from a team as a code owner November 17, 2022 17:10
@github-actions
Copy link
Contributor

github-actions bot commented Nov 17, 2022

⚠️ Performance Alert ⚠️

Possible performance regression was detected for some benchmarks.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold.

Benchmark suite Current: a0891b8 Previous: d1bef46 Ratio
Buffer.concat 32 items 4.7920 us/op 2.1840 ns/op 2194.14
computeProposerBoostScoreFromBalances 2.5012 ms/op 805.93 us/op 3.10
Full benchmark results
Benchmark suite Current: a0891b8 Previous: d1bef46 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.0557 ms/op 2.3389 ms/op 0.45
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 61.153 us/op 70.807 us/op 0.86
BLS verify - blst-native 2.1739 ms/op 2.1701 ms/op 1.00
BLS verifyMultipleSignatures 3 - blst-native 4.4901 ms/op 4.4931 ms/op 1.00
BLS verifyMultipleSignatures 8 - blst-native 9.7012 ms/op 9.7348 ms/op 1.00
BLS verifyMultipleSignatures 32 - blst-native 35.222 ms/op 35.292 ms/op 1.00
BLS aggregatePubkeys 32 - blst-native 46.901 us/op 46.491 us/op 1.01
BLS aggregatePubkeys 128 - blst-native 182.96 us/op 181.84 us/op 1.01
getAttestationsForBlock 78.571 ms/op 87.213 ms/op 0.90
isKnown best case - 1 super set check 470.00 ns/op 480.00 ns/op 0.98
isKnown normal case - 2 super set checks 466.00 ns/op 478.00 ns/op 0.97
isKnown worse case - 16 super set checks 468.00 ns/op 479.00 ns/op 0.98
CheckpointStateCache - add get delete 8.8040 us/op 9.4680 us/op 0.93
validate gossip signedAggregateAndProof - struct 5.0221 ms/op 5.0310 ms/op 1.00
validate gossip attestation - struct 2.3672 ms/op 2.3746 ms/op 1.00
pickEth1Vote - no votes 2.1686 ms/op 2.2144 ms/op 0.98
pickEth1Vote - max votes 16.266 ms/op 21.688 ms/op 0.75
pickEth1Vote - Eth1Data hashTreeRoot value x2048 12.954 ms/op 12.977 ms/op 1.00
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 20.620 ms/op 22.398 ms/op 0.92
pickEth1Vote - Eth1Data fastSerialize value x2048 1.0973 ms/op 1.5644 ms/op 0.70
pickEth1Vote - Eth1Data fastSerialize tree x2048 9.7882 ms/op 15.161 ms/op 0.65
bytes32 toHexString 971.00 ns/op 1.1230 us/op 0.86
bytes32 Buffer.toString(hex) 638.00 ns/op 785.00 ns/op 0.81
bytes32 Buffer.toString(hex) from Uint8Array 1.0210 us/op 995.00 ns/op 1.03
bytes32 Buffer.toString(hex) + 0x 671.00 ns/op 773.00 ns/op 0.87
Object access 1 prop 0.28500 ns/op 0.40300 ns/op 0.71
Map access 1 prop 0.27700 ns/op 0.31000 ns/op 0.89
Object get x1000 10.915 ns/op 11.146 ns/op 0.98
Map get x1000 0.92200 ns/op 1.0900 ns/op 0.85
Object set x1000 72.178 ns/op 89.944 ns/op 0.80
Map set x1000 42.038 ns/op 60.123 ns/op 0.70
Return object 10000 times 0.43290 ns/op 0.44070 ns/op 0.98
Throw Error 10000 times 6.5630 us/op 6.1105 us/op 1.07
fastMsgIdFn sha256 / 200 bytes 4.7660 us/op 4.9000 us/op 0.97
fastMsgIdFn h32 xxhash / 200 bytes 466.00 ns/op 601.00 ns/op 0.78
fastMsgIdFn h64 xxhash / 200 bytes 643.00 ns/op 804.00 ns/op 0.80
fastMsgIdFn sha256 / 1000 bytes 15.713 us/op 15.337 us/op 1.02
fastMsgIdFn h32 xxhash / 1000 bytes 651.00 ns/op 785.00 ns/op 0.83
fastMsgIdFn h64 xxhash / 1000 bytes 751.00 ns/op 902.00 ns/op 0.83
fastMsgIdFn sha256 / 10000 bytes 136.66 us/op 133.60 us/op 1.02
fastMsgIdFn h32 xxhash / 10000 bytes 2.5200 us/op 2.6670 us/op 0.94
fastMsgIdFn h64 xxhash / 10000 bytes 1.7470 us/op 1.9600 us/op 0.89
enrSubnets - fastDeserialize 64 bits 2.2240 us/op 3.0660 us/op 0.73
enrSubnets - ssz BitVector 64 bits 784.00 ns/op 855.00 ns/op 0.92
enrSubnets - fastDeserialize 4 bits 307.00 ns/op 420.00 ns/op 0.73
enrSubnets - ssz BitVector 4 bits 821.00 ns/op 873.00 ns/op 0.94
prioritizePeers score -10:0 att 32-0.1 sync 2-0 77.625 us/op 92.505 us/op 0.84
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 111.79 us/op 134.11 us/op 0.83
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 196.74 us/op 233.90 us/op 0.84
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 321.31 us/op 347.07 us/op 0.93
prioritizePeers score 0:0 att 64-1 sync 4-1 401.79 us/op 435.95 us/op 0.92
RateTracker 1000000 limit, 1 obj count per request 174.59 ns/op 198.19 ns/op 0.88
RateTracker 1000000 limit, 2 obj count per request 130.13 ns/op 149.35 ns/op 0.87
RateTracker 1000000 limit, 4 obj count per request 105.67 ns/op 123.83 ns/op 0.85
RateTracker 1000000 limit, 8 obj count per request 93.044 ns/op 109.77 ns/op 0.85
RateTracker with prune 3.6200 us/op 5.0920 us/op 0.71
array of 16000 items push then shift 51.620 us/op 51.621 us/op 1.00
LinkedList of 16000 items push then shift 12.502 ns/op 12.837 ns/op 0.97
array of 16000 items push then pop 188.32 ns/op 215.65 ns/op 0.87
LinkedList of 16000 items push then pop 12.118 ns/op 12.578 ns/op 0.96
array of 24000 items push then shift 77.397 us/op 77.506 us/op 1.00
LinkedList of 24000 items push then shift 13.111 ns/op 14.074 ns/op 0.93
array of 24000 items push then pop 193.13 ns/op 203.93 ns/op 0.95
LinkedList of 24000 items push then pop 12.197 ns/op 12.608 ns/op 0.97
intersect bitArray bitLen 8 21.649 ns/op 10.610 ns/op 2.04
intersect array and set length 8 125.75 ns/op 163.51 ns/op 0.77
intersect bitArray bitLen 128 70.970 ns/op 58.037 ns/op 1.22
intersect array and set length 128 1.6782 us/op 1.9575 us/op 0.86
Buffer.concat 32 items 4.7920 us/op 2.1840 ns/op 2194.14
Uint8Array.set 32 items 3.7780 us/op
pass gossip attestations to forkchoice per slot 3.7652 ms/op 6.2022 ms/op 0.61
computeDeltas 4.4664 ms/op 4.5502 ms/op 0.98
computeProposerBoostScoreFromBalances 2.5012 ms/op 805.93 us/op 3.10
altair processAttestation - 250000 vs - 7PWei normalcase 3.4044 ms/op 3.7423 ms/op 0.91
altair processAttestation - 250000 vs - 7PWei worstcase 5.3979 ms/op 6.4435 ms/op 0.84
altair processAttestation - setStatus - 1/6 committees join 178.95 us/op 179.63 us/op 1.00
altair processAttestation - setStatus - 1/3 committees join 359.71 us/op 356.23 us/op 1.01
altair processAttestation - setStatus - 1/2 committees join 518.42 us/op 512.51 us/op 1.01
altair processAttestation - setStatus - 2/3 committees join 675.91 us/op 670.51 us/op 1.01
altair processAttestation - setStatus - 4/5 committees join 943.09 us/op 934.65 us/op 1.01
altair processAttestation - setStatus - 100% committees join 1.1246 ms/op 1.1202 ms/op 1.00
altair processBlock - 250000 vs - 7PWei normalcase 22.828 ms/op 29.253 ms/op 0.78
altair processBlock - 250000 vs - 7PWei normalcase hashState 39.749 ms/op 37.087 ms/op 1.07
altair processBlock - 250000 vs - 7PWei worstcase 73.625 ms/op 84.915 ms/op 0.87
altair processBlock - 250000 vs - 7PWei worstcase hashState 101.71 ms/op 103.08 ms/op 0.99
phase0 processBlock - 250000 vs - 7PWei normalcase 3.1679 ms/op 3.6666 ms/op 0.86
phase0 processBlock - 250000 vs - 7PWei worstcase 50.991 ms/op 52.243 ms/op 0.98
altair processEth1Data - 250000 vs - 7PWei normalcase 628.40 us/op 885.90 us/op 0.71
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 7.6740 us/op 6.7420 us/op 1.14
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 22.066 us/op 20.935 us/op 1.05
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 9.9530 us/op 9.1670 us/op 1.09
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 7.6320 us/op 6.9880 us/op 1.09
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 124.20 us/op 93.224 us/op 1.33
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 1.0238 ms/op 930.04 us/op 1.10
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 141069 11.181 ms/op 12.921 ms/op 0.87
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 250000 19.261 ms/op 22.278 ms/op 0.86
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 250000 70.308 ms/op 72.133 ms/op 0.97
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 250000 37.268 ms/op 44.463 ms/op 0.84
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 250000 120.33 ms/op 120.57 ms/op 1.00
Tree 40 250000 create 585.31 ms/op 919.71 ms/op 0.64
Tree 40 250000 get(125000) 236.85 ns/op 236.42 ns/op 1.00
Tree 40 250000 set(125000) 1.8509 us/op 2.4218 us/op 0.76
Tree 40 250000 toArray() 26.788 ms/op 28.623 ms/op 0.94
Tree 40 250000 iterate all - toArray() + loop 26.701 ms/op 28.895 ms/op 0.92
Tree 40 250000 iterate all - get(i) 106.65 ms/op 111.61 ms/op 0.96
MutableVector 250000 create 13.081 ms/op 14.304 ms/op 0.91
MutableVector 250000 get(125000) 10.755 ns/op 11.187 ns/op 0.96
MutableVector 250000 set(125000) 488.07 ns/op 696.88 ns/op 0.70
MutableVector 250000 toArray() 5.8183 ms/op 6.6983 ms/op 0.87
MutableVector 250000 iterate all - toArray() + loop 6.0912 ms/op 6.9121 ms/op 0.88
MutableVector 250000 iterate all - get(i) 2.8855 ms/op 2.6070 ms/op 1.11
Array 250000 create 5.9150 ms/op 6.7280 ms/op 0.88
Array 250000 clone - spread 2.4249 ms/op 3.5050 ms/op 0.69
Array 250000 get(125000) 1.1190 ns/op 1.6100 ns/op 0.70
Array 250000 set(125000) 1.0830 ns/op 1.6320 ns/op 0.66
Array 250000 iterate all - loop 150.91 us/op 150.98 us/op 1.00
effectiveBalanceIncrements clone Uint8Array 300000 43.561 us/op 178.99 us/op 0.24
effectiveBalanceIncrements clone MutableVector 300000 708.00 ns/op 693.00 ns/op 1.02
effectiveBalanceIncrements rw all Uint8Array 300000 243.91 us/op 247.64 us/op 0.98
effectiveBalanceIncrements rw all MutableVector 300000 159.04 ms/op 165.85 ms/op 0.96
phase0 afterProcessEpoch - 250000 vs - 7PWei 189.19 ms/op 188.97 ms/op 1.00
phase0 beforeProcessEpoch - 250000 vs - 7PWei 77.796 ms/op 59.665 ms/op 1.30
altair processEpoch - mainnet_e81889 532.98 ms/op 571.44 ms/op 0.93
mainnet_e81889 - altair beforeProcessEpoch 113.63 ms/op 116.40 ms/op 0.98
mainnet_e81889 - altair processJustificationAndFinalization 31.678 us/op 16.394 us/op 1.93
mainnet_e81889 - altair processInactivityUpdates 8.6296 ms/op 9.2064 ms/op 0.94
mainnet_e81889 - altair processRewardsAndPenalties 75.375 ms/op 120.28 ms/op 0.63
mainnet_e81889 - altair processRegistryUpdates 2.4280 us/op 2.4650 us/op 0.98
mainnet_e81889 - altair processSlashings 1.1880 us/op 517.00 ns/op 2.30
mainnet_e81889 - altair processEth1DataReset 1.1000 us/op 591.00 ns/op 1.86
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.1300 ms/op 1.9240 ms/op 1.11
mainnet_e81889 - altair processSlashingsReset 7.2390 us/op 4.3140 us/op 1.68
mainnet_e81889 - altair processRandaoMixesReset 9.4110 us/op 3.9430 us/op 2.39
mainnet_e81889 - altair processHistoricalRootsUpdate 1.3630 us/op 758.00 ns/op 1.80
mainnet_e81889 - altair processParticipationFlagUpdates 2.3250 us/op 2.5730 us/op 0.90
mainnet_e81889 - altair processSyncCommitteeUpdates 590.00 ns/op 695.00 ns/op 0.85
mainnet_e81889 - altair afterProcessEpoch 196.10 ms/op 221.17 ms/op 0.89
phase0 processEpoch - mainnet_e58758 473.39 ms/op 639.48 ms/op 0.74
mainnet_e58758 - phase0 beforeProcessEpoch 183.02 ms/op 237.26 ms/op 0.77
mainnet_e58758 - phase0 processJustificationAndFinalization 19.168 us/op 16.442 us/op 1.17
mainnet_e58758 - phase0 processRewardsAndPenalties 98.573 ms/op 139.16 ms/op 0.71
mainnet_e58758 - phase0 processRegistryUpdates 16.050 us/op 7.9650 us/op 2.02
mainnet_e58758 - phase0 processSlashings 1.1230 us/op 635.00 ns/op 1.77
mainnet_e58758 - phase0 processEth1DataReset 1.1750 us/op 582.00 ns/op 2.02
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.1875 ms/op 1.8409 ms/op 1.19
mainnet_e58758 - phase0 processSlashingsReset 5.9990 us/op 4.1210 us/op 1.46
mainnet_e58758 - phase0 processRandaoMixesReset 8.3110 us/op 4.5020 us/op 1.85
mainnet_e58758 - phase0 processHistoricalRootsUpdate 846.00 ns/op 628.00 ns/op 1.35
mainnet_e58758 - phase0 processParticipationRecordUpdates 3.8530 us/op 3.9080 us/op 0.99
mainnet_e58758 - phase0 afterProcessEpoch 161.61 ms/op 164.90 ms/op 0.98
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.9324 ms/op 1.9880 ms/op 0.97
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.3929 ms/op 2.2656 ms/op 1.06
altair processInactivityUpdates - 250000 normalcase 38.786 ms/op 50.521 ms/op 0.77
altair processInactivityUpdates - 250000 worstcase 32.016 ms/op 53.890 ms/op 0.59
phase0 processRegistryUpdates - 250000 normalcase 12.687 us/op 6.5580 us/op 1.93
phase0 processRegistryUpdates - 250000 badcase_full_deposits 645.74 us/op 371.21 us/op 1.74
phase0 processRegistryUpdates - 250000 worstcase 0.5 160.22 ms/op 232.91 ms/op 0.69
altair processRewardsAndPenalties - 250000 normalcase 74.319 ms/op 142.08 ms/op 0.52
altair processRewardsAndPenalties - 250000 worstcase 102.23 ms/op 140.30 ms/op 0.73
phase0 getAttestationDeltas - 250000 normalcase 11.137 ms/op 12.276 ms/op 0.91
phase0 getAttestationDeltas - 250000 worstcase 11.299 ms/op 12.832 ms/op 0.88
phase0 processSlashings - 250000 worstcase 6.0623 ms/op 5.5686 ms/op 1.09
altair processSyncCommitteeUpdates - 250000 288.03 ms/op 298.51 ms/op 0.96
BeaconState.hashTreeRoot - No change 488.00 ns/op 568.00 ns/op 0.86
BeaconState.hashTreeRoot - 1 full validator 71.077 us/op 70.261 us/op 1.01
BeaconState.hashTreeRoot - 32 full validator 651.15 us/op 650.42 us/op 1.00
BeaconState.hashTreeRoot - 512 full validator 6.8524 ms/op 9.2545 ms/op 0.74
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 85.665 us/op 85.699 us/op 1.00
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.2990 ms/op 1.4703 ms/op 0.88
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 16.351 ms/op 17.258 ms/op 0.95
BeaconState.hashTreeRoot - 1 balances 69.162 us/op 77.483 us/op 0.89
BeaconState.hashTreeRoot - 32 balances 634.69 us/op 773.35 us/op 0.82
BeaconState.hashTreeRoot - 512 balances 6.1341 ms/op 6.3290 ms/op 0.97
BeaconState.hashTreeRoot - 250000 balances 112.94 ms/op 109.32 ms/op 1.03
aggregationBits - 2048 els - zipIndexesInBitList 21.691 us/op 24.742 us/op 0.88
regular array get 100000 times 60.504 us/op 60.562 us/op 1.00
wrappedArray get 100000 times 60.512 us/op 60.594 us/op 1.00
arrayWithProxy get 100000 times 26.129 ms/op 26.535 ms/op 0.98
ssz.Root.equals 914.00 ns/op 465.00 ns/op 1.97
byteArrayEquals 899.00 ns/op 474.00 ns/op 1.90
shuffle list - 16384 els 11.409 ms/op 11.535 ms/op 0.99
shuffle list - 250000 els 167.42 ms/op 168.82 ms/op 0.99
processSlot - 1 slots 12.652 us/op 15.142 us/op 0.84
processSlot - 32 slots 1.8916 ms/op 1.9592 ms/op 0.97
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 316.53 us/op 388.88 us/op 0.81
getCommitteeAssignments - req 1 vs - 250000 vc 5.4466 ms/op 5.4212 ms/op 1.00
getCommitteeAssignments - req 100 vs - 250000 vc 7.9605 ms/op 8.0440 ms/op 0.99
getCommitteeAssignments - req 1000 vs - 250000 vc 8.5282 ms/op 8.6036 ms/op 0.99
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 7.5300 ns/op 8.1600 ns/op 0.92
state getBlockRootAtSlot - 250000 vs - 7PWei 1.1919 us/op 1.1225 us/op 1.06
computeProposers - vc 250000 16.871 ms/op 18.368 ms/op 0.92
computeEpochShuffling - vc 250000 171.76 ms/op 171.85 ms/op 1.00
getNextSyncCommittee - vc 250000 287.06 ms/op 303.33 ms/op 0.95

by benchmarkbot/action

@twoeths
Copy link
Contributor

twoeths commented Nov 18, 2022

not sure why computeProposerBoostScoreFromBalances benchmark test is 3x slower than NodeJS 16

@dapplion
Copy link
Contributor

not sure why computeProposerBoostScoreFromBalances benchmark test is 3x slower than NodeJS 16

Can you retry multiple times and double check in local to make sure it's not a false positive?

@wemeetagain
Copy link
Member Author

not sure why computeProposerBoostScoreFromBalances benchmark test is 3x slower than NodeJS 16

Can you retry multiple times and double check in local to make sure it's not a false positive?

confirmed locally and also in CI multiple times.

@wemeetagain
Copy link
Member Author

also e2e tests consistently timing out in a few places 👀

@wemeetagain
Copy link
Member Author

tests fixed, ready for re-review

@twoeths
Copy link
Contributor

twoeths commented Dec 9, 2022

  • Mesh peers and gossip peer scores are great (while rpc count is the same)

Screen Shot 2022-12-09 at 10 09 38

  • EventLoop lag is better than Nodejs 16

Screen Shot 2022-12-09 at 10 10 29

  • That's confirmed by attestation produce/publish delay

Screen Shot 2022-12-09 at 10 11 09

However:

  • Heap memory (and gc) is not stable so we need to monitor it for a long time

Screen Shot 2022-12-09 at 10 12 37

  • And some benchmark tests failed, especially Buffer.concat() is >2000x slower is a concern

@dapplion
Copy link
Contributor

dapplion commented Dec 9, 2022

Where are all the places we use Buffer.concat in Lodestar's regular runtime?

Copy link
Contributor

@twoeths twoeths left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heap memory is an issue of unstable, not this branch. As discussed, we'd merge this anyway as the metrics look great. Some more things we need to do:

  • Create 2 issues for the 2 failed performance tests
    • Find some alternatives
    • Look for same usages in our dependencies
  • Open a performance issue of Buffer.concat for NodeJS team, really hope they'd tackle this soon

@wemeetagain
Copy link
Member Author

Where are all the places we use Buffer.concat in Lodestar's regular runtime?

Probably the most run:

return Buffer.from(digest(Buffer.concat(vec))).subarray(0, 20);

Others:

return Buffer.concat([Buffer.from(graffiti, "utf8"), Buffer.alloc(GRAFFITI_SIZE, 0)], GRAFFITI_SIZE);

A few in these files:

digest(Buffer.concat([epochSeed, intToBytes(slot, 8)]))

return digest(Buffer.concat(inputs));

@wemeetagain
Copy link
Member Author

opened issue upstream in node repo:

@wemeetagain wemeetagain merged commit 7d662d7 into unstable Dec 12, 2022
@wemeetagain wemeetagain deleted the cayman/node-18 branch December 12, 2022 15:05
@dapplion
Copy link
Contributor

Ok to me to defer issue for latter and unblock v18

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Upgrade to node 18
3 participants