{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":349172057,"defaultBranch":"main","name":"venice","ownerLogin":"linkedin","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2021-03-18T18:04:30.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/357098?v=4","public":true,"private":false,"isOrgOwned":true},"refInfo":{"name":"","listCacheKey":"v0:1716496301.0","currentOid":""},"activityList":{"items":[{"before":"1f3c01727cdbb7ac68f46cb125233b6b35454d8f","after":"d46fd2be158f2848a76a3904576267495ec8c8c5","ref":"refs/heads/javadoc","pushedAt":"2024-05-24T21:15:12.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"[server] clean up DIV metrics and fix broken ones (#993)\n\n* [server] clean up ingestion metrics and fix broken ones","shortMessageHtmlLink":"[server] clean up DIV metrics and fix broken ones (#993)"}},{"before":"1f3c01727cdbb7ac68f46cb125233b6b35454d8f","after":"d46fd2be158f2848a76a3904576267495ec8c8c5","ref":"refs/heads/main","pushedAt":"2024-05-24T21:14:50.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"adamxchen","name":"Xin(Adam) Chen","path":"/adamxchen","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/25371490?s=80&v=4"},"commit":{"message":"[server] clean up DIV metrics and fix broken ones (#993)\n\n* [server] clean up ingestion metrics and fix broken ones","shortMessageHtmlLink":"[server] clean up DIV metrics and fix broken ones (#993)"}},{"before":"d9cc66bfac38ce2428847c2d8c6b513c5e8b2135","after":"1f3c01727cdbb7ac68f46cb125233b6b35454d8f","ref":"refs/heads/javadoc","pushedAt":"2024-05-23T22:07:10.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"[internal][client] Support GRPC port discovery for non-local environment (#987)\n\n- [Functional] Introduce logic to use fallback Grpc port for netty server address -> Grpc server mapping\r\n- [Refactor] Modularize request handling and remove boiler plate in GrpcTransportClient\r\n- [Bug fix] Fix integration tests that was failing due to [fast-client][server] Throw 403 for metadata request that does not have storage read quota enabled #975\r\n- [Clean up] Remove some validations, added few invariants check and reorganize some depending on the mode (local vs integration/production)\r\n- [Clean up] Rename fields to remove type information\r\n- [Test] Add unit tests for GrpcTransportClient class\r\n\r\nCo-authored-by: Bharath Kumarasubramanian ","shortMessageHtmlLink":"[internal][client] Support GRPC port discovery for non-local environm…"}},{"before":"d9cc66bfac38ce2428847c2d8c6b513c5e8b2135","after":"1f3c01727cdbb7ac68f46cb125233b6b35454d8f","ref":"refs/heads/main","pushedAt":"2024-05-23T22:06:48.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"mynameborat","name":"Bharath Kumarasubramanian","path":"/mynameborat","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/46942335?s=80&v=4"},"commit":{"message":"[internal][client] Support GRPC port discovery for non-local environment (#987)\n\n- [Functional] Introduce logic to use fallback Grpc port for netty server address -> Grpc server mapping\r\n- [Refactor] Modularize request handling and remove boiler plate in GrpcTransportClient\r\n- [Bug fix] Fix integration tests that was failing due to [fast-client][server] Throw 403 for metadata request that does not have storage read quota enabled #975\r\n- [Clean up] Remove some validations, added few invariants check and reorganize some depending on the mode (local vs integration/production)\r\n- [Clean up] Rename fields to remove type information\r\n- [Test] Add unit tests for GrpcTransportClient class\r\n\r\nCo-authored-by: Bharath Kumarasubramanian ","shortMessageHtmlLink":"[internal][client] Support GRPC port discovery for non-local environm…"}},{"before":"acbf04aa3ce476bdaaef0839905d83937b4ce12c","after":"d9cc66bfac38ce2428847c2d8c6b513c5e8b2135","ref":"refs/heads/javadoc","pushedAt":"2024-05-23T19:52:06.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"[davinci] PartitionTracker code cleanup (#996)\n\nThis change cleans up the code in PartitionTracker based on IDE warnings and\r\nthere should have no semantic or functionality modifications.","shortMessageHtmlLink":"[davinci] PartitionTracker code cleanup (#996)"}},{"before":"acbf04aa3ce476bdaaef0839905d83937b4ce12c","after":"d9cc66bfac38ce2428847c2d8c6b513c5e8b2135","ref":"refs/heads/main","pushedAt":"2024-05-23T19:51:45.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"lluwm","name":"Lei Lu","path":"/lluwm","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/17652526?s=80&v=4"},"commit":{"message":"[davinci] PartitionTracker code cleanup (#996)\n\nThis change cleans up the code in PartitionTracker based on IDE warnings and\r\nthere should have no semantic or functionality modifications.","shortMessageHtmlLink":"[davinci] PartitionTracker code cleanup (#996)"}},{"before":"2720a8396a8a269c4f14d31fe81aef831cf747fd","after":"acbf04aa3ce476bdaaef0839905d83937b4ce12c","ref":"refs/heads/javadoc","pushedAt":"2024-05-22T11:35:08.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"[vpj] Fix file system on spark push job (#999)\n\nFix file system on spark push job, to enable S3 compatibility.","shortMessageHtmlLink":"[vpj] Fix file system on spark push job (#999)"}},{"before":"2720a8396a8a269c4f14d31fe81aef831cf747fd","after":"acbf04aa3ce476bdaaef0839905d83937b4ce12c","ref":"refs/heads/main","pushedAt":"2024-05-22T11:34:50.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"FelixGV","name":"Felix GV","path":"/FelixGV","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1248632?s=80&v=4"},"commit":{"message":"[vpj] Fix file system on spark push job (#999)\n\nFix file system on spark push job, to enable S3 compatibility.","shortMessageHtmlLink":"[vpj] Fix file system on spark push job (#999)"}},{"before":"0a64bf4c94dfec8baad91ac4289779b59048bcaf","after":"2720a8396a8a269c4f14d31fe81aef831cf747fd","ref":"refs/heads/javadoc","pushedAt":"2024-05-22T00:11:51.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"[common][server] Bug fixes and clean of ACL handlers for GRPC (#982)\n\n- Consolidate boiler plate code in ACL handlers for access validations\r\n- Fix code path that throws exception in GRPC and close the calls with the appropriate error messages\r\n- Fix the flow that invokes onMessage twice in GRPC call intercept to prevent exceptions\r\n- Rename GrpcSSLUtils to GrpcUtils to accommodate general utility methods\r\n- Add reflection service to GRPC server to interact with Grpcurl\r\n- Reuse the header for skipping ACLs from router since the store ACL handling is performed at the router for non FC clients\r\n- [code readability] Move public methods together followed by overridable and protected methods followed by package private and private methods\r\n\r\nCo-authored-by: Bharath Kumarasubramanian ","shortMessageHtmlLink":"[common][server] Bug fixes and clean of ACL handlers for GRPC (#982)"}},{"before":"0a64bf4c94dfec8baad91ac4289779b59048bcaf","after":"2720a8396a8a269c4f14d31fe81aef831cf747fd","ref":"refs/heads/main","pushedAt":"2024-05-22T00:11:35.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"mynameborat","name":"Bharath Kumarasubramanian","path":"/mynameborat","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/46942335?s=80&v=4"},"commit":{"message":"[common][server] Bug fixes and clean of ACL handlers for GRPC (#982)\n\n- Consolidate boiler plate code in ACL handlers for access validations\r\n- Fix code path that throws exception in GRPC and close the calls with the appropriate error messages\r\n- Fix the flow that invokes onMessage twice in GRPC call intercept to prevent exceptions\r\n- Rename GrpcSSLUtils to GrpcUtils to accommodate general utility methods\r\n- Add reflection service to GRPC server to interact with Grpcurl\r\n- Reuse the header for skipping ACLs from router since the store ACL handling is performed at the router for non FC clients\r\n- [code readability] Move public methods together followed by overridable and protected methods followed by package private and private methods\r\n\r\nCo-authored-by: Bharath Kumarasubramanian ","shortMessageHtmlLink":"[common][server] Bug fixes and clean of ACL handlers for GRPC (#982)"}},{"before":"d0f931b275fc57963305f441ee144f8796f69a01","after":"0a64bf4c94dfec8baad91ac4289779b59048bcaf","ref":"refs/heads/javadoc","pushedAt":"2024-05-21T23:00:43.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"[controller] CI build fix (#997)","shortMessageHtmlLink":"[controller] CI build fix (#997)"}},{"before":"d0f931b275fc57963305f441ee144f8796f69a01","after":"0a64bf4c94dfec8baad91ac4289779b59048bcaf","ref":"refs/heads/main","pushedAt":"2024-05-21T23:00:25.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"adamxchen","name":"Xin(Adam) Chen","path":"/adamxchen","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/25371490?s=80&v=4"},"commit":{"message":"[controller] CI build fix (#997)","shortMessageHtmlLink":"[controller] CI build fix (#997)"}},{"before":"2ecfa83811dac3528f39ece18e993603867e2824","after":"d0f931b275fc57963305f441ee144f8796f69a01","ref":"refs/heads/javadoc","pushedAt":"2024-05-21T21:43:09.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"[controller] Check only parent colo store defered swap config (#994)\n\nDuring batch pushes controller queries all child controller to fetch store config to validate the deferred version swap flag. This could be flaky if the some colo fails to respond in timely manner (it does 5 retries). Since the parent colo has the required store config, this PR checks only that instead of doing a expensive remote call to the child controller.","shortMessageHtmlLink":"[controller] Check only parent colo store defered swap config (#994)"}},{"before":"2ecfa83811dac3528f39ece18e993603867e2824","after":"d0f931b275fc57963305f441ee144f8796f69a01","ref":"refs/heads/main","pushedAt":"2024-05-21T21:42:47.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"majisourav99","name":"Sourav Maji","path":"/majisourav99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/62683561?s=80&v=4"},"commit":{"message":"[controller] Check only parent colo store defered swap config (#994)\n\nDuring batch pushes controller queries all child controller to fetch store config to validate the deferred version swap flag. This could be flaky if the some colo fails to respond in timely manner (it does 5 retries). Since the parent colo has the required store config, this PR checks only that instead of doing a expensive remote call to the child controller.","shortMessageHtmlLink":"[controller] Check only parent colo store defered swap config (#994)"}},{"before":"bf00dac1091096d33532f7a44cdddb7e993e5dfd","after":"2ecfa83811dac3528f39ece18e993603867e2824","ref":"refs/heads/javadoc","pushedAt":"2024-05-21T19:22:18.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"[dvc][server][controller][router] Removed Optional from Store API (#990)\n\nThe Store::getVersion API used to return an Optional, now it\r\ninstead returns a @Nullable Version. Also added a getVersionOrThrow\r\nAPI which can throw StoreVersionNotFoundException, a new exception\r\ntype.\r\n\r\nLots of refactoring to adapt to the change, though these should be\r\nnon-functional changes.\r\n\r\nSignificant changes:\r\n\r\n- Some usages of getVersion in the consumer client were doing naked\r\n .get() so these have been replaced by getVersionOrThrow.\r\n\r\n- Refactored the HelixCustomizedViewOfflinePushRepository to make\r\n all changes to the resourceToPartitionCountMap go through fewer\r\n private functions, and fixed some naked .get() bug there as well.\r\n\r\nMiscellaneous main code changes:\r\n\r\n- Deleted the ExpiringSet and its usage in ReadQuotaEnforcementHandler.\r\n\r\n- Removed the dependency on Kafka (the broker) from most modules, since \r\n this should be a test-only dependency now.\r\n\r\nMiscellaneous test code changes:\r\n\r\n- Removed nearly all mocking of the deprecated function\r\n PubSubProducerAdapter::getNumberOfPartitions and replaced it with\r\n VeniceWriterOptions::setPartitionCount.\r\n\r\n- Added timeouts to all TestVeniceHelixAdminWithSharedEnvironment\r\n tests.\r\n\r\n- Added the ability to override the duration of some long operations\r\n in HelixReadOnlySchemaRepository and PubSubConstants, only for\r\n tests. This reduces venice-common unit tests by ~6 minutes.","shortMessageHtmlLink":"[dvc][server][controller][router] Removed Optional from Store API (#990)"}},{"before":"bf00dac1091096d33532f7a44cdddb7e993e5dfd","after":"2ecfa83811dac3528f39ece18e993603867e2824","ref":"refs/heads/main","pushedAt":"2024-05-21T19:21:56.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"FelixGV","name":"Felix GV","path":"/FelixGV","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1248632?s=80&v=4"},"commit":{"message":"[dvc][server][controller][router] Removed Optional from Store API (#990)\n\nThe Store::getVersion API used to return an Optional, now it\r\ninstead returns a @Nullable Version. Also added a getVersionOrThrow\r\nAPI which can throw StoreVersionNotFoundException, a new exception\r\ntype.\r\n\r\nLots of refactoring to adapt to the change, though these should be\r\nnon-functional changes.\r\n\r\nSignificant changes:\r\n\r\n- Some usages of getVersion in the consumer client were doing naked\r\n .get() so these have been replaced by getVersionOrThrow.\r\n\r\n- Refactored the HelixCustomizedViewOfflinePushRepository to make\r\n all changes to the resourceToPartitionCountMap go through fewer\r\n private functions, and fixed some naked .get() bug there as well.\r\n\r\nMiscellaneous main code changes:\r\n\r\n- Deleted the ExpiringSet and its usage in ReadQuotaEnforcementHandler.\r\n\r\n- Removed the dependency on Kafka (the broker) from most modules, since \r\n this should be a test-only dependency now.\r\n\r\nMiscellaneous test code changes:\r\n\r\n- Removed nearly all mocking of the deprecated function\r\n PubSubProducerAdapter::getNumberOfPartitions and replaced it with\r\n VeniceWriterOptions::setPartitionCount.\r\n\r\n- Added timeouts to all TestVeniceHelixAdminWithSharedEnvironment\r\n tests.\r\n\r\n- Added the ability to override the duration of some long operations\r\n in HelixReadOnlySchemaRepository and PubSubConstants, only for\r\n tests. This reduces venice-common unit tests by ~6 minutes.","shortMessageHtmlLink":"[dvc][server][controller][router] Removed Optional from Store API (#990)"}},{"before":"9bf42604c34dbfdcbb7d0d2f39ab9f92302482a0","after":"bf00dac1091096d33532f7a44cdddb7e993e5dfd","ref":"refs/heads/javadoc","pushedAt":"2024-05-21T18:10:25.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"[client][internal] Fix tests related to PR#975 (#995)\n\n- Fix tests failures due to [fast-client][server] Throw 403 for metadata request that does not have storage read quota enabled #975\r\n- Extracting the test fixes from [internal][client] Support GRPC port discovery for non-local environment #987\r\n- Reset storage node read quota enabled flag prior to each test in FastClientIndividualFeatureConfigurationTest\r\n\r\nCo-authored-by: Bharath Kumarasubramanian ","shortMessageHtmlLink":"[client][internal] Fix tests related to PR#975 (#995)"}},{"before":"9bf42604c34dbfdcbb7d0d2f39ab9f92302482a0","after":"bf00dac1091096d33532f7a44cdddb7e993e5dfd","ref":"refs/heads/main","pushedAt":"2024-05-21T18:10:03.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"mynameborat","name":"Bharath Kumarasubramanian","path":"/mynameborat","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/46942335?s=80&v=4"},"commit":{"message":"[client][internal] Fix tests related to PR#975 (#995)\n\n- Fix tests failures due to [fast-client][server] Throw 403 for metadata request that does not have storage read quota enabled #975\r\n- Extracting the test fixes from [internal][client] Support GRPC port discovery for non-local environment #987\r\n- Reset storage node read quota enabled flag prior to each test in FastClientIndividualFeatureConfigurationTest\r\n\r\nCo-authored-by: Bharath Kumarasubramanian ","shortMessageHtmlLink":"[client][internal] Fix tests related to PR#975 (#995)"}},{"before":"c778d635b639f83e0c9bfeb5ad44431c24024bb2","after":"9bf42604c34dbfdcbb7d0d2f39ab9f92302482a0","ref":"refs/heads/javadoc","pushedAt":"2024-05-20T22:39:53.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"[server] Feature to tune level0 compaction tuning for read-write leader (#985)\n\n* [server] Feature to tune level0 compaction tuning for read-write leader\r\n\r\nIn the ingestion path, we often tune level0 compaction to optimize\r\nread/write/space optimization.\r\nWhen AA/WC is enabled, the leader replica will read the same database\r\nwhile ingesting, which is different from standby replicas.\r\nTo optimize the read perf of leader replica, this PR exposes the following\r\nconfig:\r\nrocksdb.level0.compaction.tuning.for.read.write.leader.enabled: default false\r\nrocksdb.level0.file.num.compaction.trigger.for.read.write.leader: default 10\r\nrocksdb.level0.slowdown.writes.trigger.for.read.write.leader: default 20\r\nrocksdb.level0.stops.writes.trigger.for.read.write.leader: default 40\r\n\r\nEssentially, there are 3 sets of level0 compaction tunings:\r\n1. Leader replicas for AA/WC stores.\r\n2. Future version.\r\n3. Current version.\r\n\r\nThe main reason we would like to only optimize the leader replica for AA/WC stores:\r\n1. Standby replicas don't read at ingestion path.\r\n2. Try to keep the write/space amplication factor for standby replicas as more aggressive\r\n level0 compaction tuning is expensive.\r\n\r\n* Fixed logging\r\n\r\n* Addressed comments","shortMessageHtmlLink":"[server] Feature to tune level0 compaction tuning for read-write lead…"}},{"before":"c778d635b639f83e0c9bfeb5ad44431c24024bb2","after":"9bf42604c34dbfdcbb7d0d2f39ab9f92302482a0","ref":"refs/heads/main","pushedAt":"2024-05-20T22:39:36.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"gaojieliu","name":"Gaojie Liu","path":"/gaojieliu","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/17712420?s=80&v=4"},"commit":{"message":"[server] Feature to tune level0 compaction tuning for read-write leader (#985)\n\n* [server] Feature to tune level0 compaction tuning for read-write leader\r\n\r\nIn the ingestion path, we often tune level0 compaction to optimize\r\nread/write/space optimization.\r\nWhen AA/WC is enabled, the leader replica will read the same database\r\nwhile ingesting, which is different from standby replicas.\r\nTo optimize the read perf of leader replica, this PR exposes the following\r\nconfig:\r\nrocksdb.level0.compaction.tuning.for.read.write.leader.enabled: default false\r\nrocksdb.level0.file.num.compaction.trigger.for.read.write.leader: default 10\r\nrocksdb.level0.slowdown.writes.trigger.for.read.write.leader: default 20\r\nrocksdb.level0.stops.writes.trigger.for.read.write.leader: default 40\r\n\r\nEssentially, there are 3 sets of level0 compaction tunings:\r\n1. Leader replicas for AA/WC stores.\r\n2. Future version.\r\n3. Current version.\r\n\r\nThe main reason we would like to only optimize the leader replica for AA/WC stores:\r\n1. Standby replicas don't read at ingestion path.\r\n2. Try to keep the write/space amplication factor for standby replicas as more aggressive\r\n level0 compaction tuning is expensive.\r\n\r\n* Fixed logging\r\n\r\n* Addressed comments","shortMessageHtmlLink":"[server] Feature to tune level0 compaction tuning for read-write lead…"}},{"before":"ce94ee4f621a1185ec74e344722f4916f13940a6","after":"c778d635b639f83e0c9bfeb5ad44431c24024bb2","ref":"refs/heads/javadoc","pushedAt":"2024-05-17T23:27:28.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"[controller] Do not delete true backup version during Repush (#945)\n\nDuring repush Venice deletes the actual backup version instead of current version.\r\nFor example some store has versions: v9, v10. When a repush starts it creates v11 and it deletes v9 assuming it is a backup version, where as repush actually copies data from v10 to v11.\r\nNow later if use wants to rollback to backup version they can only rollback to v10 which is exactly same as v11 as v9 was deleted.\r\n\r\nThis PR fixes that. It relies on StoreBackupVersionCleanupService to delete the backup version v10 asynchronously. Currently it checks controller.backup.version.metadata.fetch.cleanup.enabled to do a safer delete of previous current version which validates there are no reads to v10 version.","shortMessageHtmlLink":"[controller] Do not delete true backup version during Repush (#945)"}},{"before":"53b39bf0c8e6b78d3afa47f0393c3421ef7cbbe1","after":"c778d635b639f83e0c9bfeb5ad44431c24024bb2","ref":"refs/heads/main","pushedAt":"2024-05-17T23:27:06.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"majisourav99","name":"Sourav Maji","path":"/majisourav99","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/62683561?s=80&v=4"},"commit":{"message":"[controller] Do not delete true backup version during Repush (#945)\n\nDuring repush Venice deletes the actual backup version instead of current version.\r\nFor example some store has versions: v9, v10. When a repush starts it creates v11 and it deletes v9 assuming it is a backup version, where as repush actually copies data from v10 to v11.\r\nNow later if use wants to rollback to backup version they can only rollback to v10 which is exactly same as v11 as v9 was deleted.\r\n\r\nThis PR fixes that. It relies on StoreBackupVersionCleanupService to delete the backup version v10 asynchronously. Currently it checks controller.backup.version.metadata.fetch.cleanup.enabled to do a safer delete of previous current version which validates there are no reads to v10 version.","shortMessageHtmlLink":"[controller] Do not delete true backup version during Repush (#945)"}},{"before":"53b39bf0c8e6b78d3afa47f0393c3421ef7cbbe1","after":"ce94ee4f621a1185ec74e344722f4916f13940a6","ref":"refs/heads/javadoc","pushedAt":"2024-05-16T19:21:00.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"Deploying to javadoc from @ linkedin/venice@53b39bf0c8e6b78d3afa47f0393c3421ef7cbbe1 🚀","shortMessageHtmlLink":"Deploying to javadoc from @ 53b39bf 🚀"}},{"before":"b7aac31b61385f903e4f13fb32f7519eebfb945e","after":"53b39bf0c8e6b78d3afa47f0393c3421ef7cbbe1","ref":"refs/heads/javadoc","pushedAt":"2024-05-16T19:19:36.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"[server] Revert lossy rewind criterion to EOP from completion reporting (#988)\n\nThe PR: #846 modified the method responsible for handling lossy rewind checks: it updated\r\nthe function to suppress the lossy rewind issue only after reporting completion, rather\r\nthan upon receiving the EOP. It is important to note that the status of reported completion\r\nis not permanent and can change during state transitions. As a result of this change, there\r\nis an increased likelihood of ingestion stalls / failures due to lossy rewinds, compared\r\nto the previous method based on the EOP signal.\r\n\r\nThis PR reverts the check function to its original state, where the lossy rewind issue is\r\nsuppressed after reporting the EOP signal, as it was before pull request #846.\r\nThis fix is intended to avoid serving stale data. The only downside is that data\r\nfor a few keys might be inconsistent, but this issue also exists with the completion-based\r\ncheck. Since this is a short-term solution, this is okay. We're working on a solution\r\nthat allows us to ignore data from illegitimate topic writers and avoid data loss or divergence.","shortMessageHtmlLink":"[server] Revert lossy rewind criterion to EOP from completion reporti…"}},{"before":"0a1983921e6187bfc0d38f8b858be35ecfb571e3","after":"53b39bf0c8e6b78d3afa47f0393c3421ef7cbbe1","ref":"refs/heads/main","pushedAt":"2024-05-16T19:19:19.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"sushantmane","name":"Sushant Mane","path":"/sushantmane","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/7715725?s=80&v=4"},"commit":{"message":"[server] Revert lossy rewind criterion to EOP from completion reporting (#988)\n\nThe PR: #846 modified the method responsible for handling lossy rewind checks: it updated\r\nthe function to suppress the lossy rewind issue only after reporting completion, rather\r\nthan upon receiving the EOP. It is important to note that the status of reported completion\r\nis not permanent and can change during state transitions. As a result of this change, there\r\nis an increased likelihood of ingestion stalls / failures due to lossy rewinds, compared\r\nto the previous method based on the EOP signal.\r\n\r\nThis PR reverts the check function to its original state, where the lossy rewind issue is\r\nsuppressed after reporting the EOP signal, as it was before pull request #846.\r\nThis fix is intended to avoid serving stale data. The only downside is that data\r\nfor a few keys might be inconsistent, but this issue also exists with the completion-based\r\ncheck. Since this is a short-term solution, this is okay. We're working on a solution\r\nthat allows us to ignore data from illegitimate topic writers and avoid data loss or divergence.","shortMessageHtmlLink":"[server] Revert lossy rewind criterion to EOP from completion reporti…"}},{"before":"0a1983921e6187bfc0d38f8b858be35ecfb571e3","after":"b7aac31b61385f903e4f13fb32f7519eebfb945e","ref":"refs/heads/javadoc","pushedAt":"2024-05-16T18:23:35.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"Deploying to javadoc from @ linkedin/venice@0a1983921e6187bfc0d38f8b858be35ecfb571e3 🚀","shortMessageHtmlLink":"Deploying to javadoc from @ 0a19839 🚀"}},{"before":"43c62a69d0a74809e8fb2ca01d69e32237ab17b6","after":"0a1983921e6187bfc0d38f8b858be35ecfb571e3","ref":"refs/heads/javadoc","pushedAt":"2024-05-16T18:21:30.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"[server] Lag monitor should not expect corresponding version always exists in ZK (#984)\n\n* [server] Lag monitor should not expect corresponding version always exists in ZK\r\n\r\n1. updateLagMonitor was implemented to expect the version object to never be null. This assumption\r\nis wrong because when we are deleting a version the version object is removed from ZK first before\r\nthe state transition occurs on the server from STANDBY->OFFLINE. Add the option for updateLagMonitor\r\nto accept null version if it's called for removeLagMonitor.\r\n\r\n2. Separated the logging for null store and null version to be explicit and not logging store objects.","shortMessageHtmlLink":"[server] Lag monitor should not expect corresponding version always e…"}},{"before":"5c0e66a203be1e9cfd0c6cad27597035543f06a9","after":"0a1983921e6187bfc0d38f8b858be35ecfb571e3","ref":"refs/heads/main","pushedAt":"2024-05-16T18:21:10.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"xunyin8","name":"Xun Yin","path":"/xunyin8","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/7265832?s=80&v=4"},"commit":{"message":"[server] Lag monitor should not expect corresponding version always exists in ZK (#984)\n\n* [server] Lag monitor should not expect corresponding version always exists in ZK\r\n\r\n1. updateLagMonitor was implemented to expect the version object to never be null. This assumption\r\nis wrong because when we are deleting a version the version object is removed from ZK first before\r\nthe state transition occurs on the server from STANDBY->OFFLINE. Add the option for updateLagMonitor\r\nto accept null version if it's called for removeLagMonitor.\r\n\r\n2. Separated the logging for null store and null version to be explicit and not logging store objects.","shortMessageHtmlLink":"[server] Lag monitor should not expect corresponding version always e…"}},{"before":"5c0e66a203be1e9cfd0c6cad27597035543f06a9","after":"43c62a69d0a74809e8fb2ca01d69e32237ab17b6","ref":"refs/heads/javadoc","pushedAt":"2024-05-15T20:38:14.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"Deploying to javadoc from @ linkedin/venice@5c0e66a203be1e9cfd0c6cad27597035543f06a9 🚀","shortMessageHtmlLink":"Deploying to javadoc from @ 5c0e66a 🚀"}},{"before":"58100d4e648c3ae609c62865eb154d59fa0e7ded","after":"5c0e66a203be1e9cfd0c6cad27597035543f06a9","ref":"refs/heads/javadoc","pushedAt":"2024-05-15T20:36:29.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"[test] Fix for cross-test interrupt contamination (#986)\n\nThere is an issue either in Gradle or TestNG where a test which\r\nsets the interrupt flag on the current thread can cause a later\r\ntest to fail, due to checking for the interrupt flag (e.g. in\r\nsleep and syncrhonization-related primitives).\r\n\r\nThis commit fixes this by clearing the interrupt flag via the\r\nVeniceTestListener::onTestStart listener. Also registered this\r\nlistener to unit tests (it was previously used only for integ\r\ntests), and added a minimal unit test which deterministically\r\nreproduces this issue (in the absence of the listener).\r\n\r\nMiscellaneous:\r\n\r\n- Added calls to super in all functions of VeniceTestListener,\r\n as recommended in the parent class' JavaDoc.","shortMessageHtmlLink":"[test] Fix for cross-test interrupt contamination (#986)"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAEU2j8PwA","startCursor":null,"endCursor":null}},"title":"Activity · linkedin/venice"}