{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":105944401,"defaultBranch":"master","name":"yugabyte-db","ownerLogin":"yugabyte","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2017-10-05T21:56:00.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/17074854?v=4","public":true,"private":false,"isOrgOwned":true},"refInfo":{"name":"","listCacheKey":"v0:1717486682.0","currentOid":""},"activityList":{"items":[{"before":"7525a0d87e196a569ef7e71750bd9cc625458b82","after":"39d442a0be0e58e5d37b6389f666115edd06d226","ref":"refs/heads/master","pushedAt":"2024-06-04T06:42:39.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"chidmuthu","name":"Muthu Chidambaram","path":"/chidmuthu","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/86024920?s=80&v=4"},"commit":{"message":"[PLAT-14161] Removing pexlock during activation\n\nSummary: Issue with pexlock not being deleted because the trap never fired. Adds an explicit call to cleanup the pexlock after we use it.\n\nTest Plan:\nActivate pex with ybcloud.sh and ensure that pexlock is cleaned up after change.\nInterrupt while activating pex and ensure pexlock is also cleaned up.\n\nReviewers: dshubin, sanketh\n\nReviewed By: dshubin\n\nSubscribers: yugaware\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35464","shortMessageHtmlLink":"[PLAT-14161] Removing pexlock during activation"}},{"before":"d35ececc302d17b86d23e16f93cbf2fb470a1a25","after":"7525a0d87e196a569ef7e71750bd9cc625458b82","ref":"refs/heads/master","pushedAt":"2024-06-04T05:56:49.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"Sahith02","name":"Sahith Kurapati","path":"/Sahith02","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/52966918?s=80&v=4"},"commit":{"message":"[PLAT-14050] Add DB audit log runtime flag for the API disabled by default\n\nSummary: Added DB audit log runtime flag `yb.universe.audit_logging_enabled` for the telemetry provider and modify audit logging config APIs. Kept it disabled by default.\n\nTest Plan:\nManually tested.\nRun UTs.\nRun itests.\n\nReviewers: amalyshev\n\nReviewed By: amalyshev\n\nSubscribers: yugaware\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35516","shortMessageHtmlLink":"[PLAT-14050] Add DB audit log runtime flag for the API disabled by de…"}},{"before":"d449d150d7b3c27a11d6e9c2e1a9e413591f0283","after":"4430f3e01d62407d3993c74a088bde03d7853ba0","ref":"refs/heads/2024.1","pushedAt":"2024-06-04T05:44:32.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"Sahith02","name":"Sahith Kurapati","path":"/Sahith02","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/52966918?s=80&v=4"},"commit":{"message":"[BACKPORT 2024.1][PLAT-13650] Add multiline support for ycql DB audit logs\n\nSummary:\nOriginal commit: e04627188a8e82ed6fa22b01661a15b0ac9638dd / D34425\nThis PR fixes the DB audit log capture for multi-line YCQL queries. We now use the multiline config in the Filelog Receiver to split based on the audit log regex pattern prefix.\n\nWhen we run the following YCQL query before this PR:\n```\nCREATE TABLE emp5(\n id int primary key);\n```\n\nWe get output log in datadog like:\n```\nI0423 15:09:47.628911 16863 audit_logger.cc:518] AUDIT: user:anonymous|host:10.9.74.144:9042|source:10.9.74.144|port:43754|timestamp:1713884987628|type:CREATE_TABLE|category:DDL|ks:mydatabase|scope:emp5|operation:create table emp5(\n```\nNotice the truncated query.\n\nTest Plan:\nManually tested the following scenario for YCQL before and after this change:\nFlow:\n\n1. Create universe\n2. Enable DB audit logging\n3. Run single line YCQL query:\n```\ncreate table emp12( id int PRIMARY KEY );\n```\n4. Run multi line YCQL query:\n```\ncreate table emp13(\n id int PRIMARY KEY\n);\n```\nStep 3 datadog output:\n```\nI0423 15:48:50.103888 33418 audit_logger.cc:518] AUDIT: user:anonymous|host:10.9.74.144:9042|source:10.9.74.144|port:36596|timestamp:1713887330103|type:CREATE_TABLE|category:DDL|ks:mydatabase|scope:emp12|operation:create table emp12( id int PRIMARY KEY );\n```\n\nStep 4 datadog output:\n```\nI0423 15:55:22.701594 33417 audit_logger.cc:518] AUDIT: user:anonymous|host:10.9.74.144:9042|source:10.9.74.144|port:36596|timestamp:1713887722701|type:CREATE_TABLE|category:DDL|ks:mydatabase|scope:emp13|operation:create table emp13(\nid int PRIMARY KEY\n);\n```\n\nReviewers: amalyshev\n\nReviewed By: amalyshev\n\nSubscribers: yugaware\n\nTags: #jenkins-ready\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35524","shortMessageHtmlLink":"[BACKPORT 2024.1][PLAT-13650] Add multiline support for ycql DB audit…"}},{"before":"c3771b7184d8b6757b8afd7b500984494f9f11af","after":"d35ececc302d17b86d23e16f93cbf2fb470a1a25","ref":"refs/heads/master","pushedAt":"2024-06-04T04:29:45.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"Vars-07","name":"Shubham Varshney","path":"/Vars-07","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19248012?s=80&v=4"},"commit":{"message":"[PLAT-13938] Use hash for ansible/ssh control path\n\nSummary:\nAnsible tasks fails in case the host name is very large. This diff replaces the host name (%h) & remote user name (%r)\nwith %C (hash of the concatenation of %l%h%p%r, which includes the local hostname, remote hostname, port number, and remote username)\n\nTest Plan:\niTest pipeline\nManually tested universe creation.\n\nReviewers: muthu, amalyshev\n\nReviewed By: muthu\n\nSubscribers: yugaware\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35527","shortMessageHtmlLink":"[PLAT-13938] Use hash for ansible/ssh control path"}},{"before":"9baca081fe124516d394b7ad366bbf6399fcb0ac","after":"c3771b7184d8b6757b8afd7b500984494f9f11af","ref":"refs/heads/master","pushedAt":"2024-06-04T04:24:59.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"kv83821-yb","name":"Kumar Vivek","path":"/kv83821-yb","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/42805130?s=80&v=4"},"commit":{"message":"[PLAT-14203]Re-runnable tasks returns Edit Kubernetes Universe Task without validating with previous task params\n\nSummary:\nRe-runnable tasks was not checking the Edit Kubernetes Universe params before returning the\nlist of tasks allowed to be re-run. Thus, it was allowing scenarios in which we don't want to allow\ntask re-run, only retries. Modified code to correct this behavior.\n\nTest Plan: Manually tested the scenarios\n\nReviewers: anijhawan, #yba-api-review\n\nReviewed By: anijhawan, #yba-api-review\n\nSubscribers: yugaware\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35518","shortMessageHtmlLink":"[PLAT-14203]Re-runnable tasks returns Edit Kubernetes Universe Task w…"}},{"before":"272b1312056a817029d4f7861980a4f2c1257ee8","after":"9baca081fe124516d394b7ad366bbf6399fcb0ac","ref":"refs/heads/master","pushedAt":"2024-06-04T03:23:43.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"kv83821-yb","name":"Kumar Vivek","path":"/kv83821-yb","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/42805130?s=80&v=4"},"commit":{"message":"[PLAT-14208]Fix check_output got multiple values for keyword argument 'stderr'\n\nSummary:\nRecent fix https://phorge.dev.yugabyte.com/D35223 has a bug where keyword arg redirect_stderr\nis wrongly passed as `stderr`. Fixed it.\n\nTest Plan: Manually verified\n\nReviewers: anijhawan\n\nReviewed By: anijhawan\n\nSubscribers: yugaware\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35531","shortMessageHtmlLink":"[PLAT-14208]Fix check_output got multiple values for keyword argument…"}},{"before":"377de86b42d0ee4e26b0013d92a331f36b0b11ad","after":"272b1312056a817029d4f7861980a4f2c1257ee8","ref":"refs/heads/master","pushedAt":"2024-06-04T01:14:26.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"amannijhawan","name":"Aman Nijhawan","path":"/amannijhawan","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1563216?s=80&v=4"},"commit":{"message":"Fix Volume Resize to not remove overrides.\n\nSummary: Our current code removes overrides on volume resize, this change fixes that.\n\nTest Plan: phabricator-itest\n\nReviewers: sanketh, vkumar, sneelakantan\n\nReviewed By: vkumar\n\nSubscribers: yugaware\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35506","shortMessageHtmlLink":"Fix Volume Resize to not remove overrides."}},{"before":"936985b1c7baad1d4fc305788bc6fc82abdea169","after":"863ffadc92cd0d30a9f5d817864522c230068400","ref":"refs/heads/posriniv/update-dbs","pushedAt":"2024-06-04T00:57:17.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"posriniv","name":"Srinivas Pothuraju","path":"/posriniv","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/48309285?s=80&v=4"},"commit":{"message":"[PLAT-13870] [WIP] Support addition of database in DB scoped DR config\n\nSummary: The API to support addition of database in DB scoped DR config\n\nTest Plan:\nExecuted the API on local YBA DR clusters and checked whether replication\nhappens on the newly added database.\n\nReviewers: #yba-api-review!\n\nSubscribers: yugaware\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35498","shortMessageHtmlLink":"[PLAT-13870] [WIP] Support addition of database in DB scoped DR config"}},{"before":"ad513fec93d24b409667e31039ffb07d183e2c67","after":"936985b1c7baad1d4fc305788bc6fc82abdea169","ref":"refs/heads/posriniv/update-dbs","pushedAt":"2024-06-04T00:53:01.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"posriniv","name":"Srinivas Pothuraju","path":"/posriniv","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/48309285?s=80&v=4"},"commit":{"message":"[PLAT-13870] [WIP] Support addition of database in DB scoped DR config\n\nSummary: The API to support addition of database in DB scoped DR config\n\nTest Plan:\nExecuted the API on local YBA DR clusters and checked whether replication\nhappens on the newly added database.\n\nReviewers: #yba-api-review!\n\nSubscribers: yugaware\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35498","shortMessageHtmlLink":"[PLAT-13870] [WIP] Support addition of database in DB scoped DR config"}},{"before":"bf033bb75de978a13b2b23b9a0f4be40ebf53481","after":"d449d150d7b3c27a11d6e9c2e1a9e413591f0283","ref":"refs/heads/2024.1","pushedAt":"2024-06-04T00:41:28.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"myang2021","name":null,"path":"/myang2021","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/79478033?s=80&v=4"},"commit":{"message":"[BACKPORT 2024.1][#22649] YSQL: Removed unused function CatalogManager::WaitForDdlVerificationToFinish\n\nSummary:\nThe function `CatalogManager::WaitForDdlVerificationToFinish` is unused and its\nfunctionality is done in a different way via\n--ysql_ddl_transaction_wait_for_ddl_verification.\n\nI deleted the function to avoid confusion.\nJira: DB-11552\n\nOriginal commit: b6950bd5ae4c77dfe050ecd5518f210ccc923732 / D35532\n\nTest Plan: ./yb_build.sh\n\nReviewers: fizaa\n\nReviewed By: fizaa\n\nSubscribers: yql, ybase\n\nTags: #jenkins-ready\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35545","shortMessageHtmlLink":"[BACKPORT 2024.1][#22649] YSQL: Removed unused function CatalogManage…"}},{"before":"b6950bd5ae4c77dfe050ecd5518f210ccc923732","after":"377de86b42d0ee4e26b0013d92a331f36b0b11ad","ref":"refs/heads/master","pushedAt":"2024-06-04T00:32:15.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"amannijhawan","name":"Aman Nijhawan","path":"/amannijhawan","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1563216?s=80&v=4"},"commit":{"message":"PLAT-14185 Fix operator software upgrade issue\n\nSummary:\nPLAT-14185 Fix operator software upgrade issue\nWe were incorrectly passing the existing software version to the universe.\nThis change fixes that by removing that code.\n\nTest Plan: phabricator, operator-automation\n\nReviewers: vkumar\n\nReviewed By: vkumar\n\nSubscribers: yugaware\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35494","shortMessageHtmlLink":"PLAT-14185 Fix operator software upgrade issue"}},{"before":"a0c0bda6b3a7a06a81f9d496ba84e0d50b071485","after":"dd24929b2c26f282221fe4e31d2b93ce60f5dfba","ref":"refs/heads/pg15","pushedAt":"2024-06-03T23:00:13.000Z","pushType":"push","commitsCount":51,"pusher":{"login":"jasonyb","name":null,"path":"/jasonyb","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/93959687?s=80&v=4"},"commit":{"message":"[pg15] merge: pgaudit tag '1.7.0' into pg15\n\nSummary:\nMerge pgaudit/tags/1.7.0: into\nyugabyte-db/pg15:src/postgres/third-party-extensions/pgaudit using\nsubtree merge (with an indirection to make git blame good).\n\n- Makefile:\n - DATA: manually specify the existing scripts.\n- README.md:\n - Take pgaudit 1.7.0.\n- pgaudit.out:\n - Take pgaudit 1.7.0.\n- pgaudit--1.3.2.sql:\n - Delete. YB makes no modifications to the extension scripts\n- pgaudit--1.3.sql: (same)\n- pgaudit--1.3--1.3.1.sql: (same)\n- pgaudit--1.3.1--1.3.2.sql: (same)\n- pgaudit--1.7.sql:\n - Take pgaudit 1.7.0.\n- pgaudit--1.3.2--1.7.sql:\n - Add this brand new migration from YB master's 1.3.2 to YB pg15's\n 1.7. Make the content exactly like pgaudit\n 881c617084cac4985b9471426510af05c2a49dd7:pgaudit--1.6--1.6.1.sql.\n- pgaudit.c:\n - copyright: take pgaudit 1.7.0.\n - pgaudit_ProcessUtility_hook:\n - pgaudit_NextProcessUtility_hook: YB\n 329515ec6111daf5fe1f7d169de3a437273ad2c3 puts code into new helper\n pgaudit_NextProcessUtility_hook in two locations. pgaudit 1.7.0\n has\n - commit 70b30d43799edc106f9ebfaee178959043e33081 which adds extra\n parameter readOnlyTree to the function calls\n - commit 2f9661311b83dc481fc19f6e3bda015392010a40 which changes\n completionTag to qc\n Plumb those changes through the helper.\n - stack is not empty: false merge conflict. YB (based off pgaudit\n 1.3.2) knows of commit 671dd2a628e3ea7bc56f662c9b07d574546f5aeb\n while pgaudit 1.7.0 knows of commit\n 437a537345b38613fb28feaa0e343144c05309e1. They are both identical\n (one is a backport of the other), so stick with the previous\n resolution by YB pg15 329515ec6111daf5fe1f7d169de3a437273ad2c3.\n - If this is a create/alter extension command: add this section\n introduced by pgaudit 1.7.0\n 6460d9fec78c344245c70309551bb314415268a8.\n- pgaudit.control:\n - Take pgaudit 1.7.0.\n- pgaudit.sql:\n - Take pgaudit 1.7.0.\n- Vagrantfile:\n - Delete as in pgaudit 1.7.0. This file was originally fully owned by\n pgaudit, but the git merge 329515ec6111daf5fe1f7d169de3a437273ad2c3\n incorrectly thinks it is a new file on both sides.\n- build_postgres.py:\n - make_postgres: enable building pgaudit.\n- pg_wrapper.cc:\n - WritePostgresConfig: enable pgaudit as a shared preload library.\n- yb_pg_pgaudit.sql:\n - Align to new pgaudit.sql. Of particular note is that YB pg15\n initial merge 55782d561e55ef972f2470a4ae887dd791bb4a97 added bad\n SELECT 'yb'. Get rid of it now.\n- yb_pg_pgaudit.out:\n - Align to new pgaudit.out.\n- pg15_tests/passing_tests.tsv:\n - Add TestPgRegressThirdPartyExtensionsPgaudit to the list.\n\nTest Plan:\nOn Almalinux 8:\n\n #!/usr/bin/env bash\n set -eu\n ./yb_build.sh fastdebug --gcc11\n yes \"$(grep Pgaudit pg15_tests/passing_tests.tsv)\" | head -50 | pg15_tests/run_tests.sh\n\nJenkins: rebase: pg15\n\nReviewers: tfoucher\n\nReviewed By: tfoucher\n\nSubscribers: yql\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35505","shortMessageHtmlLink":"[pg15] merge: pgaudit tag '1.7.0' into pg15"}},{"before":"d46ff32eeadd4a5dd7dd5e04c2a1f8890bd3daee","after":"b6950bd5ae4c77dfe050ecd5518f210ccc923732","ref":"refs/heads/master","pushedAt":"2024-06-03T21:30:25.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"myang2021","name":null,"path":"/myang2021","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/79478033?s=80&v=4"},"commit":{"message":"[#22649] YSQL: Removed unused function CatalogManager::WaitForDdlVerificationToFinish\n\nSummary:\nThe function `CatalogManager::WaitForDdlVerificationToFinish` is unused and its\nfunctionality is done in a different way via\n--ysql_ddl_transaction_wait_for_ddl_verification.\n\nI deleted the function to avoid confusion.\nJira: DB-11552\n\nTest Plan: ./yb_build.sh\n\nReviewers: fizaa\n\nReviewed By: fizaa\n\nSubscribers: ybase, yql\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35532","shortMessageHtmlLink":"[#22649] YSQL: Removed unused function CatalogManager::WaitForDdlVeri…"}},{"before":"3138cd04b8af712f7b63c66d820126630944aa32","after":"bf033bb75de978a13b2b23b9a0f4be40ebf53481","ref":"refs/heads/2024.1","pushedAt":"2024-06-03T19:23:48.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"Sahith02","name":"Sahith Kurapati","path":"/Sahith02","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/52966918?s=80&v=4"},"commit":{"message":"[BACKPORT 2024.1][PLAT-13550] Add multiline support for ysql DB audit logs\n\nSummary:\nOriginal commit: 1cf02705281e33a90bbf333b6ba66718b08d9e6f / D34333\nThis PR fixes the DB audit log capture for multi-line queries. We now use the multiline config in the Filelog Receiver to split based on the audit log regex pattern prefix.\n\nWhen we run the following query before this PR:\n```\nCREATE TABLE emp (\n id int\n);\n```\nWe get output log in datadog like:\n```\n2024-04-19 13:59:52.021 UTC [10159] LOG: AUDIT: SESSION,2,1,DDL,CREATE TABLE,,,\"CREATE TABLE emp (\n```\nNotice the truncated query.\n\nTest Plan:\nManually tested the following scenario for ysql before and after this change:\nFlow:\n1. Create universe\n2. Enable DB audit logging\n3. Run single line query:\n```\nCREATE TABLE emp1 (id int);\n```\n4. Run multi line query:\n```\nCREATE TABLE emp2 (\n id int\n);\n```\n\nStep 3 datadog output:\n```\n2024-04-19 15:26:08.973 UTC [10011] LOG: AUDIT: SESSION,2,1,DDL,CREATE TABLE,,,create table emp1(id int);,\n```\n\nStep 4 datadog output:\n```\n2024-04-19 15:26:29.206 UTC [10011] LOG: AUDIT: SESSION,3,1,DDL,CREATE TABLE,,,\"create table emp2(\n\t id int\n\t);\",\n```\n\nReviewers: amalyshev\n\nReviewed By: amalyshev\n\nSubscribers: yugaware\n\nTags: #jenkins-ready\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35522","shortMessageHtmlLink":"[BACKPORT 2024.1][PLAT-13550] Add multiline support for ysql DB audit…"}},{"before":"e9f60b873fcd6574344969a348182c923f9b653c","after":"3138cd04b8af712f7b63c66d820126630944aa32","ref":"refs/heads/2024.1","pushedAt":"2024-06-03T19:22:02.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"Sahith02","name":"Sahith Kurapati","path":"/Sahith02","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/52966918?s=80&v=4"},"commit":{"message":"[BACKPORT 2024.1][PLAT-13297] Audit logs not exported from newly added read replica\n\nSummary:\nOriginal commit: 22c5ea439d8b0cbeff3226fda7838d99e3cf70fe / D33995\nAudit logs were not exported from newly added read replica cluster. We were sending `AuditLogConfig` as null when calling the read replica API. Since the primary cluster already has the `AuditLogConfig`, we should be using that instead when provisioning the new nodes.\n\nThe following flow didn't work before, which does now:\n```\nCreate a universe without RR.\nEnable DB audit logging.\nAdd RR to this universe.\nVerify --> Audit logs are not visible on DD\n```\n\nTest Plan:\nManually tested the following flow:\nTried the following scenarios:\nCase 1:\n```\nCreate a universe with primary cluster and RR.\nEnable DB audit logs.\nVerify. ---> Works as expected. Audit logs from primary cluster nodes and RR nodes are visible on DD.\n```\n\nCase 2:\n```\nCreate a universe without RR.\nEnable DB audit logging.\nAdd RR to this universe.\nVerify --> Works as expected. Audit logs from primary cluster nodes and RR nodes are visible on DD.\n```\n\nCase 3 (Patched this diff on my diff: https://phorge.dev.yugabyte.com/D33949):\n```\nCreate a universe without RR.\nEnable DB audit logging.\nAdd RR to this universe.\nVerify --> Works as expected. Audit logs from primary cluster nodes and RR nodes are visible on DD.\nAdd new node to universe.\nVerify --> Works as expected. Audit logs from both primary cluster nodes and RR nodes are visible on DD.\n```\n\nReviewers: amalyshev\n\nReviewed By: amalyshev\n\nSubscribers: yugaware\n\nTags: #jenkins-ready\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35526","shortMessageHtmlLink":"[BACKPORT 2024.1][PLAT-13297] Audit logs not exported from newly adde…"}},{"before":"4e79de3afca39a73b6521be2e2f296fc0c9c4028","after":"e9f60b873fcd6574344969a348182c923f9b653c","ref":"refs/heads/2024.1","pushedAt":"2024-06-03T19:19:52.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"Sahith02","name":"Sahith Kurapati","path":"/Sahith02","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/52966918?s=80&v=4"},"commit":{"message":"[BACKPORT 2024.1][PLAT-13368] [PLAT-13431] User level systemd steps fail when configuring DB audit logs\n\nSummary:\nOriginal commit: c75f6a76b11806efd5c152c143dea2c4c8d16622 / D33814\nUser level systemd steps fail when configuring DB audit logs.\nLooks like previously the otel collector user level systemd steps were run as `ec2-user`. We now run it as `yugabyte` user.\n\nEarlier when running the /audit_log_config API, it failed with following error:\n```\nTASK [manage_otel_collector : Control OpenTelemetry collector | User mode systemd | Stop existing running OpenTelemetry Collector] ***\nWednesday 27 March 2024 11:24:05 +0000 (0:00:00.612) 0:00:14.616 *******\nfatal: [10.21.152.3]: FAILED! => changed=false\n cmd: /bin/systemctl --user\n msg: 'Failed to get D-Bus connection: No such file or directory'\n rc: 1\n stderr: |-\n Failed to get D-Bus connection: No such file or directory\n stderr_lines: \n stdout: ''\n stdout_lines: \n```\n\nAlso fixed another issue which happens during edit universe ops:\n```\nTASK [manage_otel_collector : Control OpenTelemetry collector | User mode systemd | Enable and start OpenTelemetry collector] ***\nThursday 04 April 2024 07:16:27 +0000 (0:00:00.225) 0:01:38.157 ********\nfatal: [10.9.65.47]: FAILED! => changed=false\n msg: |-\n Unable to start service otel-collector: Failed to start otel-collector.service: Unit network-online.target not found.\n```\n\nTest Plan:\nManually tested following flows work now:\n\nFlow 1:\n1. Create a universe without RR.\n2. Enable DB audit logging.\n3. Add RR to this universe.\n4. Call the DB audit logging endpoint again.\nVerified all the above operations pass as expected.\nAlso tested that the RR node has the otel process running after step 4.\n\nFlow 2:\n1. Create universe with single node\n2. Enable audit log\n3. Expand universe to 2 nodes\nVerified all the above operations pass as expected.\nAlso tested that the new node has the otel process running after step 3.\n\nRun UTs.\nRun itests.\n\nReviewers: amalyshev\n\nReviewed By: amalyshev\n\nSubscribers: yugaware\n\nTags: #jenkins-ready\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35525","shortMessageHtmlLink":"[BACKPORT 2024.1][PLAT-13368] [PLAT-13431] User level systemd steps f…"}},{"before":"2349d7c2df5a519677b80a8eae902a816f34f95b","after":"a0c0bda6b3a7a06a81f9d496ba84e0d50b071485","ref":"refs/heads/pg15","pushedAt":"2024-06-03T18:36:30.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"jasonyb","name":null,"path":"/jasonyb","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/93959687?s=80&v=4"},"commit":{"message":"[pg15] test: fix shell tests failing on query id\n\nSummary:\nShell tests were not covered by YB pg15\nc55d458af8fd9a45839f3ea9e76d1c4d016cd4b5 when setting compute_query_id\nto hide query ids. Do that now.\n\nTest Plan:\nOn Almalinux 8:\n\n #!/usr/bin/env bash\n set -eu\n ./yb_build.sh fastdebug --gcc11\n pg15_tests/get_shell_test_specs.sh | grep -E '(test_D29662|test_D31292|test_D31365)' | pg15_tests/run_tests.sh\n\nJenkins: rebase: pg15, test regex: TestPgRegress\n\nReviewers: devansh.saxena, tfoucher\n\nReviewed By: devansh.saxena, tfoucher\n\nSubscribers: yql\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35507","shortMessageHtmlLink":"[pg15] test: fix shell tests failing on query id"}},{"before":"bb374e70c0123ec97051024b2d8b29b0a6757725","after":"d46ff32eeadd4a5dd7dd5e04c2a1f8890bd3daee","ref":"refs/heads/master","pushedAt":"2024-06-03T17:47:16.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"shubin-yb","name":"Daniel Shubin","path":"/shubin-yb","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/117757925?s=80&v=4"},"commit":{"message":"[PLAT-12731] upgrade software start as a v2 api\n\nSummary:\nThis combines both the upgrade software and upgrade db version\napis into a single api, with the ability to have (or skip) rollback\n\nTest Plan:\nunit tests for mapping\nmanual testing of the api to start an upgrade\n\nReviewers: sneelakantan, #yba-api-review\n\nReviewed By: sneelakantan, #yba-api-review\n\nSubscribers: yugaware\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35213","shortMessageHtmlLink":"[PLAT-12731] upgrade software start as a v2 api"}},{"before":"7358cd113ecbcb710e365c6ef8299fc7b72b9193","after":"bb374e70c0123ec97051024b2d8b29b0a6757725","ref":"refs/heads/master","pushedAt":"2024-06-03T17:41:23.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"shubin-yb","name":"Daniel Shubin","path":"/shubin-yb","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/117757925?s=80&v=4"},"commit":{"message":"[PLAT-13824] Update postgresql version\n\nSummary: updating postgresql version to latest 42.3.x\n\nTest Plan: itest\n\nReviewers: muthu\n\nReviewed By: muthu\n\nSubscribers: yugaware\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35454","shortMessageHtmlLink":"[PLAT-13824] Update postgresql version"}},{"before":"2080f687ae0a8eeadc03f8124098b44342f827d3","after":"7358cd113ecbcb710e365c6ef8299fc7b72b9193","ref":"refs/heads/master","pushedAt":"2024-06-03T17:34:45.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"timothy-e","name":"Timothy Elgersma","path":"/timothy-e","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26841748?s=80&v=4"},"commit":{"message":"[#22515] YSQL: Rearrange YB Bitmap Tests\n\nSummary:\nChanges:\n* Rename `yb_bitmap_scan_joins` to `yb_bitmap_scans_joins` to have a more uniform naming scheme.\n* Move testing the flags of bitmap scans to its own file, `yb_bitmap_scans_flags`.\n* remove the summary from every explain output (`EXPLAIN (SUMMARY OFF)`) - it didn't add any extra information.\n* move system queries to their own file `yb_bitmap_scans_system`.\n* duplicate `yb_bitmap_scans` to `yb_bitmap_scans_colo`. a few changes are made for the colocated version:\n * create and connect to a colocated database\n * remove the HASH index on multi\n\nBreaking these into seperate files allows for:\n* knowing more immediately what is broken (colocated / distributed / system)\n* more confidence that colocated tables are fully tested (each test case from distributed tables is running on colocated now too)\nJira: DB-11441\n\nTest Plan:\n```\n./yb_build.sh --java-test 'org.yb.pgsql.TestPgRegressYbBitmapScans'\n```\n\nJenkins: test regex: .*Bitmap.*\n\nReviewers: tnayak, amartsinchyk\n\nReviewed By: amartsinchyk\n\nSubscribers: yql\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35385","shortMessageHtmlLink":"[#22515] YSQL: Rearrange YB Bitmap Tests"}},{"before":"88d4289464b9656d3aa11a947a8ad3a09c03e21a","after":"fdd1f814f65cc1d9369f0a5aec659dee9bd529c7","ref":"refs/heads/2.23.0.1141","pushedAt":"2024-06-03T17:27:03.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"vitroth","name":"David Nolan","path":"/vitroth","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1354939?s=80&v=4"},"commit":{"message":"[BACKPORT 2.23.0.1141][PLAT-14209] Fix an issue with dual-nic script usage on Centos+GCP\n\nSummary:\nOriginal commit: 2080f687ae0a8eeadc03f8124098b44342f827d3 / D35535\nPrevious change to the configurenic script broke older OS usage\non GCP.\n\nTest Plan:\nError verified by Dmitry as related to change in the NIC script\nfor Alma, which broke older Centos deploys.\n\nReviewers: daniel, rguliyev, dskorobogaty\n\nReviewed By: daniel, rguliyev\n\nSubscribers: yugaware\n\nTags: #jenkins-ready\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35537","shortMessageHtmlLink":"[BACKPORT 2.23.0.1141][PLAT-14209] Fix an issue with dual-nic script …"}},{"before":"a2b9a1a34bacdd8da75591fd54311dc0787eebac","after":"2080f687ae0a8eeadc03f8124098b44342f827d3","ref":"refs/heads/master","pushedAt":"2024-06-03T17:19:19.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"vitroth","name":"David Nolan","path":"/vitroth","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1354939?s=80&v=4"},"commit":{"message":"[PLAT-14209] Fix an issue with dual-nic script usage on Centos+GCP\n\nSummary:\nPrevious change to the configurenic script broke older OS usage\non GCP.\n\nTest Plan:\nError verified by Dmitry as related to change in the NIC script\nfor Alma, which broke older Centos deploys.\n\nReviewers: daniel, rguliyev, dskorobogaty\n\nReviewed By: daniel\n\nSubscribers: yugaware\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35535","shortMessageHtmlLink":"[PLAT-14209] Fix an issue with dual-nic script usage on Centos+GCP"}},{"before":"7ac1b8b686b60858bb1bc82a211334ebea0072ac","after":"a2b9a1a34bacdd8da75591fd54311dc0787eebac","ref":"refs/heads/master","pushedAt":"2024-06-03T17:18:31.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"asharma-yb","name":"Ayush Sharma","path":"/asharma-yb","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/109268085?s=80&v=4"},"commit":{"message":"[#19848] Tests: Run YB Controller backups in java UTS\n\nSummary:\nPrimary goal is to run backups in the Java UTs via YB Controller.\nThis diff has the following changes\n1. Add helper functions to run backups via YB Controller.\n2. YB Controller's output differs from `yb_backup.py` so make changes where necessary to accommodate this change.\n3. Disable tests which are not supported via YB Controller.\nJira: DB-8791\n\nTest Plan:\nManually ran UTs locally to verify backup/restore works.\n```\nYB_TEST_YB_CONTROLLER=1 ./yb_build.sh --jo --java-test 'org.yb.pgsql.TestYbBackup#testMixedLegacyColocatedDatabase'\n```\nMonitor detective results.\n\nReviewers: loginov\n\nReviewed By: loginov\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D33881","shortMessageHtmlLink":"[#19848] Tests: Run YB Controller backups in java UTS"}},{"before":"9ce800a051d9a1e6676802871303653e1119a886","after":"7ac1b8b686b60858bb1bc82a211334ebea0072ac","ref":"refs/heads/master","pushedAt":"2024-06-03T17:18:13.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"myang2021","name":null,"path":"/myang2021","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/79478033?s=80&v=4"},"commit":{"message":"[#22598] YSQL: Fix database OID collision with system_postgres\n\nSummary:\nTo reproduce the bug\n(1) Create a local cluster with the gflag --TEST_ysql_oid_prefetch_adjustment=49148\nThis gflag simulates the scenario that those OIDs < 65532 are already used so\nthe next OID to allocate starts from 65532 (16384 + 49148).\n\n```\n./bin/yb-ctl create --rf 1 --tserver_flags TEST_ysql_oid_prefetch_adjustment=49148\n```\n\n(2) Prepare the following test script:\n```\ncat /tmp/test.sql\nCREATE DATABASE test0;\nCREATE DATABASE test1;\nCREATE DATABASE test2;\nCREATE DATABASE test3;\n\\c test3\nCREATE table t0 (id int);\nCREATE table t1 (id int);\nCREATE TABLE t(k SERIAL, v INT);\n```\n\n(3) Run the above test script\n```\n./bin/ysqlsh -a -f /tmp/test.sql\nCREATE DATABASE test0;\nCREATE DATABASE\nCREATE DATABASE test1;\nCREATE DATABASE\nCREATE DATABASE test2;\nCREATE DATABASE\nCREATE DATABASE test3;\nCREATE DATABASE\n\\c test3\nYou are now connected to database \"test3\" as user \"yugabyte\".\nCREATE table t0 (id int);\nCREATE TABLE\nCREATE table t1 (id int);\nCREATE TABLE\nCREATE TABLE t(k SERIAL, v INT);\nysqlsh:/tmp/test.sql:8: ERROR: Errors occurred while reaching out to the tablet servers\n```\n\n(4) Look into tserver log:\n```\nE0529 16:09:01.055245 8673 ql_expr_util.cc:59] Corruption (yb/qlexpr/ql_expr_util.cc:58): Column at 1 is not key column in Schema [\n 0:ybrowid[binary NOT NULL HASH],\n 1:id[int32 NULLABLE VALUE]\n]\nproperties: contain_counters: false is_transactional: true consistency_level: STRONG use_mangled_column_name: false is_ysql_catalog_table: false retain_delete_markers: false partitioning_version: 1 ysql_replica_identity: CHANGE\n```\n\n(5) Root cause\nThe root cause of this error is that the database oid 65535 (0xffff) is reserved\nby Yugabyte to store sequence table `system_postgres.sequences_data`, where\n`system_postgres` called `namespace` in docdb, and `sequences_data` is the table name.\n\n```\nstatic const uint32_t kPgSequencesDataTableOid = 0xFFFF;\nstatic const uint32_t kPgSequencesDataDatabaseOid = 0xFFFF;\n```\n\n`kPgSequencesDataDatabaseOid` is for `system_postgres`.\n`kPgSequencesDataTableOid` is for `sequences_data`.\n\nBut 65535 is also a normal OID that can be allocated by the PG oid allocator. In\nthe test a user created database `test3` has already occupied the same OID 65535.\nFurther more, inside the user database OID 65535 was also allocated for table\n`t1`. Later when the system tries to create the internal table\nsystem_postgres.sequences_data, it collided with `test3.t1` which has a\ndifferent schema.\n\nThis diff fixes the bug by skipping 65535 for database oid allocation.\nJira: DB-11504\n\nTest Plan: ./yb_build.sh release --cxx-test pg_libpq-test --gtest_filter=PgOidCollisionReservedNormalOid.PgOidCollisionSystemPostgresTest -n 20\n\nReviewers: yguan\n\nReviewed By: yguan\n\nSubscribers: yql\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35414","shortMessageHtmlLink":"[#22598] YSQL: Fix database OID collision with system_postgres"}},{"before":"aead171e524043624a7045fe6212ddf0eb8a06c5","after":"8e2ea6b0e86c0d818415fc375c70c7a21494040d","ref":"refs/heads/2.20","pushedAt":"2024-06-03T14:53:30.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"vipul-yb","name":"Vipul Bansal","path":"/vipul-yb","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/86227026?s=80&v=4"},"commit":{"message":"[BACKPORT 2.20][PLAT-14160]: Fix extraction of DB file for multi threads\n\nSummary:\nwhen two threds tries to read tar file at a time, their buffer could collide which corrupts the reads,\nso made this block synchronized.\nIt is not expected to increase runtime as customer will probably add only one release and these files are supposed to be added only once.\n\nOriginal diff/commit: 84cb950e6e/D35455\n\nTest Plan: Manually tested as this issue was easily reproducible through UI.\n\nReviewers: svarshney\n\nReviewed By: svarshney\n\nSubscribers: yugaware\n\nTags: #jenkins-ready\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35487","shortMessageHtmlLink":"[BACKPORT 2.20][PLAT-14160]: Fix extraction of DB file for multi threads"}},{"before":"b914731f8ca74d491a03b72318e8a58addfe5b31","after":"4e79de3afca39a73b6521be2e2f296fc0c9c4028","ref":"refs/heads/2024.1","pushedAt":"2024-06-03T14:51:48.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"vipul-yb","name":"Vipul Bansal","path":"/vipul-yb","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/86227026?s=80&v=4"},"commit":{"message":"[BACKPORT 2024.1][PLAT-14160]: Fix extraction of DB file for multi threads\n\nSummary:\nwhen two threds tries to read tar file at a time, their buffer could collide which corrupts the reads,\nso made this block synchronized.\nIt is not expected to increase runtime as customer will probably add only one release and these files are supposed to be added only once.\n\nOriginal diff/commit: 84cb950e6e/D35455\n\nTest Plan: Manually tested as this issue was easily reproducible through UI.\n\nReviewers: svarshney\n\nReviewed By: svarshney\n\nSubscribers: yugaware\n\nTags: #jenkins-ready\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35488","shortMessageHtmlLink":"[BACKPORT 2024.1][PLAT-14160]: Fix extraction of DB file for multi th…"}},{"before":"647c24bdcd54e331146492b135c34616a1bd0e67","after":"9ce800a051d9a1e6676802871303653e1119a886","ref":"refs/heads/master","pushedAt":"2024-06-03T13:01:10.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"Sahith02","name":"Sahith Kurapati","path":"/Sahith02","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/52966918?s=80&v=4"},"commit":{"message":"[PLAT-13939][PLAT-13925][PLAT-13943][PLAT-12172] Validate the task payload before triggering the audit_log_config\n\nSummary:\nThis diff covers 4 small tickets:\n1. [PLAT-13939] Validate the task payload before triggering the audit_log_config\n```\nIf the existing audit log config on the universe is same as the requested audit log config in the modifyAuditLogConfig API, we now throw an error.\n```\n2. [PLAT-13925] Regex parsing error in the otel logs\n```\nWhen YCQL logs are under the category ERROR, the keyspace and scope are not present in the audit log. We now make these 2 optional.\n```\n3. [PLAT-13943] Audit logs are skipped for the logLevel type (INFO, NOTICE)\n```\nDefault `log_min_messages` is \"WARNING\". Anything below that is not logged by default for YSQL. For API users, we leave the log level options as is and note down in API docs to set `log_min_messages` if log level below `WARNING` is required.\n```\n4. [PLAT-12172] Mark datadog logs with collector specific tags\n```\nWe now add a couple of datadog specific labels `ddsource:yugabyte`, `service:yb-otel-collector`, `host:` to each audit log that gets exported to datadog. All these labels can always be overridden at the telemetry provider layer or the `modifyAuditLogging` API layer.\n```\n\nTest Plan:\nManually tested all 4 of the above scenarios.\nRun UTs.\nRun itests.\n\nReviewers: amalyshev\n\nReviewed By: amalyshev\n\nSubscribers: yugaware\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35387","shortMessageHtmlLink":"[PLAT-13939][PLAT-13925][PLAT-13943][PLAT-12172] Validate the task pa…"}},{"before":"beeff4dabec3a148a241a470121a73d9a64d95ec","after":"b914731f8ca74d491a03b72318e8a58addfe5b31","ref":"refs/heads/2024.1","pushedAt":"2024-06-03T12:24:23.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"asharma-yb","name":"Ayush Sharma","path":"/asharma-yb","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/109268085?s=80&v=4"},"commit":{"message":"[BACKPORT 2024.1][PLAT-14149] fix tenant id error message in azure provider validation\n\nSummary:\nOriginal commit 4ac953971f245bf577f2efdf33606dab8d86458e/D35479\n\nupdate the tenant id error message so that correct field is highlighted on ui.\n\nTest Plan: manually verified tenant id error is thrown and correct field is highlighted on ui.\n\nReviewers: #yba-api-review!, svarshney\n\nReviewed By: svarshney\n\nSubscribers: yugaware\n\nTags: #jenkins-ready\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35514","shortMessageHtmlLink":"[BACKPORT 2024.1][PLAT-14149] fix tenant id error message in azure pr…"}},{"before":"0f3bd51dadb2f9572938648e2f01bc606ad3132e","after":"647c24bdcd54e331146492b135c34616a1bd0e67","ref":"refs/heads/master","pushedAt":"2024-06-03T11:58:31.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"yorq","name":"Yuriy Shchetinin","path":"/yorq","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/90446433?s=80&v=4"},"commit":{"message":"[PLAT-13603] Edit volume info showing inconsistent behavior between UI refresh\n\nSummary:\nWhen there is a stopped node in universe, /universe_configure method doesn't take this node into account\nthus adding new node. We should skip only ToBeRemoved nodes while calculations.\n\nTest Plan:\nsbt test\nEdit universe with stopped node -> check that updateOptions is empty in /universe_configure response\n\nReviewers: sanketh, cwang, nsingh\n\nReviewed By: cwang\n\nSubscribers: yugaware\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D34572","shortMessageHtmlLink":"[PLAT-13603] Edit volume info showing inconsistent behavior between U…"}},{"before":"4ac953971f245bf577f2efdf33606dab8d86458e","after":"0f3bd51dadb2f9572938648e2f01bc606ad3132e","ref":"refs/heads/master","pushedAt":"2024-06-03T11:44:06.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"asharma-yb","name":"Ayush Sharma","path":"/asharma-yb","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/109268085?s=80&v=4"},"commit":{"message":"[#22621] Tests: Redirect ybc logs to test output\n\nSummary:\nMoved ExternalDaemon to separate header.\nLogTailerThread was a private nested class of ExternalDaemon class.\nMade it public to be used in ExternalYBController.\n\nYB Controller logs are available in the test output with the prefix [ybc-$idx].\nJira: DB-11529\n\nTest Plan: Run backup tests manually to confirm functionality.\n\nReviewers: loginov\n\nReviewed By: loginov\n\nSubscribers: yql, ybase, loginov\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35161","shortMessageHtmlLink":"[#22621] Tests: Redirect ybc logs to test output"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAEW66I2QA","startCursor":null,"endCursor":null}},"title":"Activity · yugabyte/yugabyte-db"}