v0.93.0

[steven-sheehy]

This release deduplicates balance history resulting in a major reduction in database size with no loss in balance granularity. The mainnet database saw a 45% reduction going from 50 TB to 28TB! This deduplication process works by not storing balance history if the account did not experience a balance change since the last snapshot. A migration to deduplicate historical balances runs asynchronously in the background and against mainnet state took about 24 hours to complete. Because the index was changed to reverse the order from (timestamp, account_id) to (account_id, timestamp), this required a large effort to rework queries in multiple REST APIs. Also, the balance tables are now partitioned and this meant changes in our database metrics to properly aggregate child tables on their parent name.

HIP-584 continues to chug along with multiple bug fixes and optimizations. Changing per request objects to be singletons resulted in a large decrease in memory and CPU usage, allowing more concurrent requests to be handled. Web3 k6 tests were hooked into our automated performance testing to ensure they run every release to ensure no regressions. Finally, support for historical blocks made progress with more of the plumbing put in place to process non-latest blocks.

A new cluster database health check was added to the monitor to provide proper failover in multi-cluster deployments. The local file stream provider now allows for input files to be grouped by date for faster processing when the directory contains millions of files. This is a step towards a faster historical syncing mode. Finally, REST API queries were optimized for Citus deployments so that they could reach parity with PostgreSQL.

Enhancements

• Use deduped token balance data in historical token balance distribution REST API (0.93) #7286
• Add metrics for partitioned tables (0.93) #7267
• Support deduplicated balances in accounts rest endpoints #7234
• HIP-584: Extract MirrorEvmTxProcessor as a singleton bean #7211
• Add web3 k6 tests to testkube #7205
• HIP-584 Historical: Set historical block values #7185
• Add Health check on Rest Network Stake #7182
• Time partition and deduplicate existing balance data in v1 #7181
• Support local files grouped by day #7180
• Improve MirrorEvmTxProcessor execute method #7179
• Use extensionless/register #7175
• HIP-584: Token transfer does not fail after dissociate #7146
• HIP-584 Historical: Extend nft, nft allowance and token account DB queries to filter by timestamp #7096
• HIP-584 Historical: Map block number to timestamp #7090
• Remove not cached tokens and fix flaky tests #6997

Bug Fixes

• HIP-584: Throw exception only on populated data and missing bytecode (0.93) #7291
• Fix incorrect usage of k6 VU execution context in web3 tests #7244
• Fix incorrect value for flyway placeholder partitionStartDate #7238
• Fix incorrect key to look up entity in entity state #7212
• Bump io.projectreactor.netty:reactor-netty-http #7208
• HIP-584: 0x returned when contract's runtime bytecode is not available yet #7203
• HIP-584: Set correct sender account in TransferPrecompile #7194
• Fix flaky RedisEntityListenerIntegrationTest #7190
• Fix flaky SyntheticCryptoTransferApprovalsMigrationTest #7189
• Fix log rate dashboard #7183
• HIP-584: Account balance cache delays is reflecting to proper eth_call balance update #7178
• HIP-584: Move setting of recipient to be Address.ZERO outside of validations #7158
• HIP-584: Update the blockhash operation to return value for all blocks #7156
• Teardown REST API test resources #7152
• HIP-584: Fix nested contract deploy fail #7148
• Fix balances API with timestamp performance in v2 #7087
• Fix contract result endpoints performance in v2 #7049

Dependency Upgrades

• Bump logback from 1.4.11 to 1.4.13 (0.93) #7290
• Bump github.com/ethereum/go-ethereum from 1.13.4 to 1.13.5 in /hedera-mirror-rosetta #7229
• Bump mathjs from 12.0.0 to 12.1.0 in /hedera-mirror-rest/monitoring/monitor_apis #7228
• Bump org.testcontainers:junit-jupiter from 1.19.1 to 1.19.2 #7227
• Bump io.projectreactor:reactor-core-micrometer from 1.0.11 to 1.1.0 #7226
• Bump org.owasp:dependency-check-gradle from 8.4.2 to 8.4.3 #7225
• Bump io.spring.gradle:dependency-management-plugin from 1.1.3 to 1.1.4 #7224
• Bump software.amazon.awssdk:bom from 2.21.21 to 2.21.26 #7223
• Bump org.openapitools:openapi-generator-gradle-plugin from 7.0.1 to 7.1.0 #7222
• Bump vertxVersion from 4.4.6 to 4.5.0 #7221
• Bump protobufVersion from 3.25.0 to 3.25.1 #7220
• Bump org.jetbrains:annotations from 24.0.1 to 24.1.0 #7219
• Bump org.apache.commons:commons-compress from 1.24.0 to 1.25.0 #7218
• Bump mem from 9.0.2 to 10.0.0 in /hedera-mirror-rest #7217
• Bump @aws-sdk/client-s3 from 3.449.0 to 3.454.0 in /hedera-mirror-rest #7216
• Bump mathjs from 12.0.0 to 12.1.0 in /hedera-mirror-rest #7215
• Bump @aws-sdk/client-s3 from 3.441.0 to 3.449.0 in /hedera-mirror-rest #7201
• Bump express-openapi-validator from 5.0.6 to 5.1.0 in /hedera-mirror-rest #7200
• Bump io.hypersistence:hypersistence-utils-hibernate-62 from 3.6.0 to 3.6.1 #7199
• Bump software.amazon.awssdk:bom from 2.21.15 to 2.21.21 #7198
• Bump uk.org.webcompere:system-stubs-jupiter from 2.1.3 to 2.1.5 #7197
• Bump com.bmuschko:gradle-docker-plugin from 9.3.6 to 9.3.7 #7196
• Bump axios from 1.4.0 to 1.6.1 in /hedera-mirror-rest #7195
• Bump extensionless from 1.7.0 to 1.7.3 in /hedera-mirror-rest/monitoring/monitor_apis #7174
• Bump com.esaulpaugh:headlong from 10.0.0 to 10.0.1 #7173
• Bump testcontainersSpringBootVersion from 3.0.5 to 3.0.6 #7172
• Bump protobufVersion from 3.24.4 to 3.25.0 #7171
• Bump software.amazon.awssdk:bom from 2.21.10 to 2.21.15 #7169
• Bump helm/chart-testing-action from 2.6.0 to 2.6.1 #7168
• Bump github.com/go-playground/validator/v10 from 10.15.5 to 10.16.0 in /hedera-mirror-rosetta #7167
• Bump github.com/hashgraph/hedera-sdk-go/v2 from 2.31.0 to 2.32.0 in /hedera-mirror-rosetta #7166
• Bump @aws-sdk/client-s3 from 3.438.0 to 3.441.0 in /hedera-mirror-rest #7165
• Bump extensionless from 1.7.0 to 1.7.3 in /hedera-mirror-rest #7164
• Bump sinon from 17.0.0 to 17.0.1 in /hedera-mirror-rest #7163
• Bump extensionless from 1.7.0 to 1.7.3 in /hedera-mirror-rest/check-state-proof #7162
• Bump Node.js from 18.18.2 to 20.9.0 #7161
• Bump chart-testing-action from 2.4.0 to 2.6.0 #7150
• Bump versions for v0.93.0-SNAPSHOT #7149

Contributors

We'd like to thank all the contributors who worked on this release!

• @bilyana-gospodinova
• @IvanKavaldzhiev
• @xin-hedera
• @jnels124
• @yiliev0
• @0xivanov
• @edwin-greene
• @kselveliev
• @steven-sheehy

---

This discussion was created from the release v0.93.0.