New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use node 18 #4779
Use node 18 #4779
Conversation
|
Benchmark suite | Current: a0891b8 | Previous: d1bef46 | Ratio |
---|---|---|---|
Buffer.concat 32 items | 4.7920 us/op | 2.1840 ns/op | 2194.14 |
computeProposerBoostScoreFromBalances | 2.5012 ms/op | 805.93 us/op | 3.10 |
Full benchmark results
Benchmark suite | Current: a0891b8 | Previous: d1bef46 | Ratio |
---|---|---|---|
getPubkeys - index2pubkey - req 1000 vs - 250000 vc | 1.0557 ms/op | 2.3389 ms/op | 0.45 |
getPubkeys - validatorsArr - req 1000 vs - 250000 vc | 61.153 us/op | 70.807 us/op | 0.86 |
BLS verify - blst-native | 2.1739 ms/op | 2.1701 ms/op | 1.00 |
BLS verifyMultipleSignatures 3 - blst-native | 4.4901 ms/op | 4.4931 ms/op | 1.00 |
BLS verifyMultipleSignatures 8 - blst-native | 9.7012 ms/op | 9.7348 ms/op | 1.00 |
BLS verifyMultipleSignatures 32 - blst-native | 35.222 ms/op | 35.292 ms/op | 1.00 |
BLS aggregatePubkeys 32 - blst-native | 46.901 us/op | 46.491 us/op | 1.01 |
BLS aggregatePubkeys 128 - blst-native | 182.96 us/op | 181.84 us/op | 1.01 |
getAttestationsForBlock | 78.571 ms/op | 87.213 ms/op | 0.90 |
isKnown best case - 1 super set check | 470.00 ns/op | 480.00 ns/op | 0.98 |
isKnown normal case - 2 super set checks | 466.00 ns/op | 478.00 ns/op | 0.97 |
isKnown worse case - 16 super set checks | 468.00 ns/op | 479.00 ns/op | 0.98 |
CheckpointStateCache - add get delete | 8.8040 us/op | 9.4680 us/op | 0.93 |
validate gossip signedAggregateAndProof - struct | 5.0221 ms/op | 5.0310 ms/op | 1.00 |
validate gossip attestation - struct | 2.3672 ms/op | 2.3746 ms/op | 1.00 |
pickEth1Vote - no votes | 2.1686 ms/op | 2.2144 ms/op | 0.98 |
pickEth1Vote - max votes | 16.266 ms/op | 21.688 ms/op | 0.75 |
pickEth1Vote - Eth1Data hashTreeRoot value x2048 | 12.954 ms/op | 12.977 ms/op | 1.00 |
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 | 20.620 ms/op | 22.398 ms/op | 0.92 |
pickEth1Vote - Eth1Data fastSerialize value x2048 | 1.0973 ms/op | 1.5644 ms/op | 0.70 |
pickEth1Vote - Eth1Data fastSerialize tree x2048 | 9.7882 ms/op | 15.161 ms/op | 0.65 |
bytes32 toHexString | 971.00 ns/op | 1.1230 us/op | 0.86 |
bytes32 Buffer.toString(hex) | 638.00 ns/op | 785.00 ns/op | 0.81 |
bytes32 Buffer.toString(hex) from Uint8Array | 1.0210 us/op | 995.00 ns/op | 1.03 |
bytes32 Buffer.toString(hex) + 0x | 671.00 ns/op | 773.00 ns/op | 0.87 |
Object access 1 prop | 0.28500 ns/op | 0.40300 ns/op | 0.71 |
Map access 1 prop | 0.27700 ns/op | 0.31000 ns/op | 0.89 |
Object get x1000 | 10.915 ns/op | 11.146 ns/op | 0.98 |
Map get x1000 | 0.92200 ns/op | 1.0900 ns/op | 0.85 |
Object set x1000 | 72.178 ns/op | 89.944 ns/op | 0.80 |
Map set x1000 | 42.038 ns/op | 60.123 ns/op | 0.70 |
Return object 10000 times | 0.43290 ns/op | 0.44070 ns/op | 0.98 |
Throw Error 10000 times | 6.5630 us/op | 6.1105 us/op | 1.07 |
fastMsgIdFn sha256 / 200 bytes | 4.7660 us/op | 4.9000 us/op | 0.97 |
fastMsgIdFn h32 xxhash / 200 bytes | 466.00 ns/op | 601.00 ns/op | 0.78 |
fastMsgIdFn h64 xxhash / 200 bytes | 643.00 ns/op | 804.00 ns/op | 0.80 |
fastMsgIdFn sha256 / 1000 bytes | 15.713 us/op | 15.337 us/op | 1.02 |
fastMsgIdFn h32 xxhash / 1000 bytes | 651.00 ns/op | 785.00 ns/op | 0.83 |
fastMsgIdFn h64 xxhash / 1000 bytes | 751.00 ns/op | 902.00 ns/op | 0.83 |
fastMsgIdFn sha256 / 10000 bytes | 136.66 us/op | 133.60 us/op | 1.02 |
fastMsgIdFn h32 xxhash / 10000 bytes | 2.5200 us/op | 2.6670 us/op | 0.94 |
fastMsgIdFn h64 xxhash / 10000 bytes | 1.7470 us/op | 1.9600 us/op | 0.89 |
enrSubnets - fastDeserialize 64 bits | 2.2240 us/op | 3.0660 us/op | 0.73 |
enrSubnets - ssz BitVector 64 bits | 784.00 ns/op | 855.00 ns/op | 0.92 |
enrSubnets - fastDeserialize 4 bits | 307.00 ns/op | 420.00 ns/op | 0.73 |
enrSubnets - ssz BitVector 4 bits | 821.00 ns/op | 873.00 ns/op | 0.94 |
prioritizePeers score -10:0 att 32-0.1 sync 2-0 | 77.625 us/op | 92.505 us/op | 0.84 |
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 | 111.79 us/op | 134.11 us/op | 0.83 |
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 | 196.74 us/op | 233.90 us/op | 0.84 |
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 | 321.31 us/op | 347.07 us/op | 0.93 |
prioritizePeers score 0:0 att 64-1 sync 4-1 | 401.79 us/op | 435.95 us/op | 0.92 |
RateTracker 1000000 limit, 1 obj count per request | 174.59 ns/op | 198.19 ns/op | 0.88 |
RateTracker 1000000 limit, 2 obj count per request | 130.13 ns/op | 149.35 ns/op | 0.87 |
RateTracker 1000000 limit, 4 obj count per request | 105.67 ns/op | 123.83 ns/op | 0.85 |
RateTracker 1000000 limit, 8 obj count per request | 93.044 ns/op | 109.77 ns/op | 0.85 |
RateTracker with prune | 3.6200 us/op | 5.0920 us/op | 0.71 |
array of 16000 items push then shift | 51.620 us/op | 51.621 us/op | 1.00 |
LinkedList of 16000 items push then shift | 12.502 ns/op | 12.837 ns/op | 0.97 |
array of 16000 items push then pop | 188.32 ns/op | 215.65 ns/op | 0.87 |
LinkedList of 16000 items push then pop | 12.118 ns/op | 12.578 ns/op | 0.96 |
array of 24000 items push then shift | 77.397 us/op | 77.506 us/op | 1.00 |
LinkedList of 24000 items push then shift | 13.111 ns/op | 14.074 ns/op | 0.93 |
array of 24000 items push then pop | 193.13 ns/op | 203.93 ns/op | 0.95 |
LinkedList of 24000 items push then pop | 12.197 ns/op | 12.608 ns/op | 0.97 |
intersect bitArray bitLen 8 | 21.649 ns/op | 10.610 ns/op | 2.04 |
intersect array and set length 8 | 125.75 ns/op | 163.51 ns/op | 0.77 |
intersect bitArray bitLen 128 | 70.970 ns/op | 58.037 ns/op | 1.22 |
intersect array and set length 128 | 1.6782 us/op | 1.9575 us/op | 0.86 |
Buffer.concat 32 items | 4.7920 us/op | 2.1840 ns/op | 2194.14 |
Uint8Array.set 32 items | 3.7780 us/op | ||
pass gossip attestations to forkchoice per slot | 3.7652 ms/op | 6.2022 ms/op | 0.61 |
computeDeltas | 4.4664 ms/op | 4.5502 ms/op | 0.98 |
computeProposerBoostScoreFromBalances | 2.5012 ms/op | 805.93 us/op | 3.10 |
altair processAttestation - 250000 vs - 7PWei normalcase | 3.4044 ms/op | 3.7423 ms/op | 0.91 |
altair processAttestation - 250000 vs - 7PWei worstcase | 5.3979 ms/op | 6.4435 ms/op | 0.84 |
altair processAttestation - setStatus - 1/6 committees join | 178.95 us/op | 179.63 us/op | 1.00 |
altair processAttestation - setStatus - 1/3 committees join | 359.71 us/op | 356.23 us/op | 1.01 |
altair processAttestation - setStatus - 1/2 committees join | 518.42 us/op | 512.51 us/op | 1.01 |
altair processAttestation - setStatus - 2/3 committees join | 675.91 us/op | 670.51 us/op | 1.01 |
altair processAttestation - setStatus - 4/5 committees join | 943.09 us/op | 934.65 us/op | 1.01 |
altair processAttestation - setStatus - 100% committees join | 1.1246 ms/op | 1.1202 ms/op | 1.00 |
altair processBlock - 250000 vs - 7PWei normalcase | 22.828 ms/op | 29.253 ms/op | 0.78 |
altair processBlock - 250000 vs - 7PWei normalcase hashState | 39.749 ms/op | 37.087 ms/op | 1.07 |
altair processBlock - 250000 vs - 7PWei worstcase | 73.625 ms/op | 84.915 ms/op | 0.87 |
altair processBlock - 250000 vs - 7PWei worstcase hashState | 101.71 ms/op | 103.08 ms/op | 0.99 |
phase0 processBlock - 250000 vs - 7PWei normalcase | 3.1679 ms/op | 3.6666 ms/op | 0.86 |
phase0 processBlock - 250000 vs - 7PWei worstcase | 50.991 ms/op | 52.243 ms/op | 0.98 |
altair processEth1Data - 250000 vs - 7PWei normalcase | 628.40 us/op | 885.90 us/op | 0.71 |
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 | 7.6740 us/op | 6.7420 us/op | 1.14 |
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 | 22.066 us/op | 20.935 us/op | 1.05 |
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 | 9.9530 us/op | 9.1670 us/op | 1.09 |
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 | 7.6320 us/op | 6.9880 us/op | 1.09 |
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 | 124.20 us/op | 93.224 us/op | 1.33 |
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 | 1.0238 ms/op | 930.04 us/op | 1.10 |
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 141069 | 11.181 ms/op | 12.921 ms/op | 0.87 |
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 250000 | 19.261 ms/op | 22.278 ms/op | 0.86 |
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 250000 | 70.308 ms/op | 72.133 ms/op | 0.97 |
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 250000 | 37.268 ms/op | 44.463 ms/op | 0.84 |
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 250000 | 120.33 ms/op | 120.57 ms/op | 1.00 |
Tree 40 250000 create | 585.31 ms/op | 919.71 ms/op | 0.64 |
Tree 40 250000 get(125000) | 236.85 ns/op | 236.42 ns/op | 1.00 |
Tree 40 250000 set(125000) | 1.8509 us/op | 2.4218 us/op | 0.76 |
Tree 40 250000 toArray() | 26.788 ms/op | 28.623 ms/op | 0.94 |
Tree 40 250000 iterate all - toArray() + loop | 26.701 ms/op | 28.895 ms/op | 0.92 |
Tree 40 250000 iterate all - get(i) | 106.65 ms/op | 111.61 ms/op | 0.96 |
MutableVector 250000 create | 13.081 ms/op | 14.304 ms/op | 0.91 |
MutableVector 250000 get(125000) | 10.755 ns/op | 11.187 ns/op | 0.96 |
MutableVector 250000 set(125000) | 488.07 ns/op | 696.88 ns/op | 0.70 |
MutableVector 250000 toArray() | 5.8183 ms/op | 6.6983 ms/op | 0.87 |
MutableVector 250000 iterate all - toArray() + loop | 6.0912 ms/op | 6.9121 ms/op | 0.88 |
MutableVector 250000 iterate all - get(i) | 2.8855 ms/op | 2.6070 ms/op | 1.11 |
Array 250000 create | 5.9150 ms/op | 6.7280 ms/op | 0.88 |
Array 250000 clone - spread | 2.4249 ms/op | 3.5050 ms/op | 0.69 |
Array 250000 get(125000) | 1.1190 ns/op | 1.6100 ns/op | 0.70 |
Array 250000 set(125000) | 1.0830 ns/op | 1.6320 ns/op | 0.66 |
Array 250000 iterate all - loop | 150.91 us/op | 150.98 us/op | 1.00 |
effectiveBalanceIncrements clone Uint8Array 300000 | 43.561 us/op | 178.99 us/op | 0.24 |
effectiveBalanceIncrements clone MutableVector 300000 | 708.00 ns/op | 693.00 ns/op | 1.02 |
effectiveBalanceIncrements rw all Uint8Array 300000 | 243.91 us/op | 247.64 us/op | 0.98 |
effectiveBalanceIncrements rw all MutableVector 300000 | 159.04 ms/op | 165.85 ms/op | 0.96 |
phase0 afterProcessEpoch - 250000 vs - 7PWei | 189.19 ms/op | 188.97 ms/op | 1.00 |
phase0 beforeProcessEpoch - 250000 vs - 7PWei | 77.796 ms/op | 59.665 ms/op | 1.30 |
altair processEpoch - mainnet_e81889 | 532.98 ms/op | 571.44 ms/op | 0.93 |
mainnet_e81889 - altair beforeProcessEpoch | 113.63 ms/op | 116.40 ms/op | 0.98 |
mainnet_e81889 - altair processJustificationAndFinalization | 31.678 us/op | 16.394 us/op | 1.93 |
mainnet_e81889 - altair processInactivityUpdates | 8.6296 ms/op | 9.2064 ms/op | 0.94 |
mainnet_e81889 - altair processRewardsAndPenalties | 75.375 ms/op | 120.28 ms/op | 0.63 |
mainnet_e81889 - altair processRegistryUpdates | 2.4280 us/op | 2.4650 us/op | 0.98 |
mainnet_e81889 - altair processSlashings | 1.1880 us/op | 517.00 ns/op | 2.30 |
mainnet_e81889 - altair processEth1DataReset | 1.1000 us/op | 591.00 ns/op | 1.86 |
mainnet_e81889 - altair processEffectiveBalanceUpdates | 2.1300 ms/op | 1.9240 ms/op | 1.11 |
mainnet_e81889 - altair processSlashingsReset | 7.2390 us/op | 4.3140 us/op | 1.68 |
mainnet_e81889 - altair processRandaoMixesReset | 9.4110 us/op | 3.9430 us/op | 2.39 |
mainnet_e81889 - altair processHistoricalRootsUpdate | 1.3630 us/op | 758.00 ns/op | 1.80 |
mainnet_e81889 - altair processParticipationFlagUpdates | 2.3250 us/op | 2.5730 us/op | 0.90 |
mainnet_e81889 - altair processSyncCommitteeUpdates | 590.00 ns/op | 695.00 ns/op | 0.85 |
mainnet_e81889 - altair afterProcessEpoch | 196.10 ms/op | 221.17 ms/op | 0.89 |
phase0 processEpoch - mainnet_e58758 | 473.39 ms/op | 639.48 ms/op | 0.74 |
mainnet_e58758 - phase0 beforeProcessEpoch | 183.02 ms/op | 237.26 ms/op | 0.77 |
mainnet_e58758 - phase0 processJustificationAndFinalization | 19.168 us/op | 16.442 us/op | 1.17 |
mainnet_e58758 - phase0 processRewardsAndPenalties | 98.573 ms/op | 139.16 ms/op | 0.71 |
mainnet_e58758 - phase0 processRegistryUpdates | 16.050 us/op | 7.9650 us/op | 2.02 |
mainnet_e58758 - phase0 processSlashings | 1.1230 us/op | 635.00 ns/op | 1.77 |
mainnet_e58758 - phase0 processEth1DataReset | 1.1750 us/op | 582.00 ns/op | 2.02 |
mainnet_e58758 - phase0 processEffectiveBalanceUpdates | 2.1875 ms/op | 1.8409 ms/op | 1.19 |
mainnet_e58758 - phase0 processSlashingsReset | 5.9990 us/op | 4.1210 us/op | 1.46 |
mainnet_e58758 - phase0 processRandaoMixesReset | 8.3110 us/op | 4.5020 us/op | 1.85 |
mainnet_e58758 - phase0 processHistoricalRootsUpdate | 846.00 ns/op | 628.00 ns/op | 1.35 |
mainnet_e58758 - phase0 processParticipationRecordUpdates | 3.8530 us/op | 3.9080 us/op | 0.99 |
mainnet_e58758 - phase0 afterProcessEpoch | 161.61 ms/op | 164.90 ms/op | 0.98 |
phase0 processEffectiveBalanceUpdates - 250000 normalcase | 1.9324 ms/op | 1.9880 ms/op | 0.97 |
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 | 2.3929 ms/op | 2.2656 ms/op | 1.06 |
altair processInactivityUpdates - 250000 normalcase | 38.786 ms/op | 50.521 ms/op | 0.77 |
altair processInactivityUpdates - 250000 worstcase | 32.016 ms/op | 53.890 ms/op | 0.59 |
phase0 processRegistryUpdates - 250000 normalcase | 12.687 us/op | 6.5580 us/op | 1.93 |
phase0 processRegistryUpdates - 250000 badcase_full_deposits | 645.74 us/op | 371.21 us/op | 1.74 |
phase0 processRegistryUpdates - 250000 worstcase 0.5 | 160.22 ms/op | 232.91 ms/op | 0.69 |
altair processRewardsAndPenalties - 250000 normalcase | 74.319 ms/op | 142.08 ms/op | 0.52 |
altair processRewardsAndPenalties - 250000 worstcase | 102.23 ms/op | 140.30 ms/op | 0.73 |
phase0 getAttestationDeltas - 250000 normalcase | 11.137 ms/op | 12.276 ms/op | 0.91 |
phase0 getAttestationDeltas - 250000 worstcase | 11.299 ms/op | 12.832 ms/op | 0.88 |
phase0 processSlashings - 250000 worstcase | 6.0623 ms/op | 5.5686 ms/op | 1.09 |
altair processSyncCommitteeUpdates - 250000 | 288.03 ms/op | 298.51 ms/op | 0.96 |
BeaconState.hashTreeRoot - No change | 488.00 ns/op | 568.00 ns/op | 0.86 |
BeaconState.hashTreeRoot - 1 full validator | 71.077 us/op | 70.261 us/op | 1.01 |
BeaconState.hashTreeRoot - 32 full validator | 651.15 us/op | 650.42 us/op | 1.00 |
BeaconState.hashTreeRoot - 512 full validator | 6.8524 ms/op | 9.2545 ms/op | 0.74 |
BeaconState.hashTreeRoot - 1 validator.effectiveBalance | 85.665 us/op | 85.699 us/op | 1.00 |
BeaconState.hashTreeRoot - 32 validator.effectiveBalance | 1.2990 ms/op | 1.4703 ms/op | 0.88 |
BeaconState.hashTreeRoot - 512 validator.effectiveBalance | 16.351 ms/op | 17.258 ms/op | 0.95 |
BeaconState.hashTreeRoot - 1 balances | 69.162 us/op | 77.483 us/op | 0.89 |
BeaconState.hashTreeRoot - 32 balances | 634.69 us/op | 773.35 us/op | 0.82 |
BeaconState.hashTreeRoot - 512 balances | 6.1341 ms/op | 6.3290 ms/op | 0.97 |
BeaconState.hashTreeRoot - 250000 balances | 112.94 ms/op | 109.32 ms/op | 1.03 |
aggregationBits - 2048 els - zipIndexesInBitList | 21.691 us/op | 24.742 us/op | 0.88 |
regular array get 100000 times | 60.504 us/op | 60.562 us/op | 1.00 |
wrappedArray get 100000 times | 60.512 us/op | 60.594 us/op | 1.00 |
arrayWithProxy get 100000 times | 26.129 ms/op | 26.535 ms/op | 0.98 |
ssz.Root.equals | 914.00 ns/op | 465.00 ns/op | 1.97 |
byteArrayEquals | 899.00 ns/op | 474.00 ns/op | 1.90 |
shuffle list - 16384 els | 11.409 ms/op | 11.535 ms/op | 0.99 |
shuffle list - 250000 els | 167.42 ms/op | 168.82 ms/op | 0.99 |
processSlot - 1 slots | 12.652 us/op | 15.142 us/op | 0.84 |
processSlot - 32 slots | 1.8916 ms/op | 1.9592 ms/op | 0.97 |
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei | 316.53 us/op | 388.88 us/op | 0.81 |
getCommitteeAssignments - req 1 vs - 250000 vc | 5.4466 ms/op | 5.4212 ms/op | 1.00 |
getCommitteeAssignments - req 100 vs - 250000 vc | 7.9605 ms/op | 8.0440 ms/op | 0.99 |
getCommitteeAssignments - req 1000 vs - 250000 vc | 8.5282 ms/op | 8.6036 ms/op | 0.99 |
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei | 7.5300 ns/op | 8.1600 ns/op | 0.92 |
state getBlockRootAtSlot - 250000 vs - 7PWei | 1.1919 us/op | 1.1225 us/op | 1.06 |
computeProposers - vc 250000 | 16.871 ms/op | 18.368 ms/op | 0.92 |
computeEpochShuffling - vc 250000 | 171.76 ms/op | 171.85 ms/op | 1.00 |
getNextSyncCommittee - vc 250000 | 287.06 ms/op | 303.33 ms/op | 0.95 |
by benchmarkbot/action
not sure why |
Can you retry multiple times and double check in local to make sure it's not a false positive? |
confirmed locally and also in CI multiple times. |
also e2e tests consistently timing out in a few places 👀 |
9eaeaf2
to
f9d856f
Compare
298d051
to
93d66f3
Compare
93d66f3
to
b28a7b3
Compare
tests fixed, ready for re-review |
However:
|
Where are all the places we use Buffer.concat in Lodestar's regular runtime? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Heap memory is an issue of unstable, not this branch. As discussed, we'd merge this anyway as the metrics look great. Some more things we need to do:
- Create 2 issues for the 2 failed performance tests
- Find some alternatives
- Look for same usages in our dependencies
- Open a performance issue of
Buffer.concat
for NodeJS team, really hope they'd tackle this soon
Probably the most run:
Others:
A few in these files:
|
opened issue upstream in node repo: |
Ok to me to defer issue for latter and unblock v18 |
Motivation
Resolve #4768
Description
Minimal diff to test/build/recommend using node 18