{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":105944401,"defaultBranch":"master","name":"yugabyte-db","ownerLogin":"yugabyte","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2017-10-05T21:56:00.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/17074854?v=4","public":true,"private":false,"isOrgOwned":true},"refInfo":{"name":"","listCacheKey":"v0:1716794724.0","currentOid":""},"activityList":{"items":[{"before":"8b56d8a1f4d513cf15beb79352da93532a5768bb","after":"36130a44c54cd595eb017bdd3ccb96352deb80dd","ref":"refs/heads/2024.1","pushedAt":"2024-05-27T08:20:38.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"d-uspenskiy","name":"Dmitry Uspenskiy","path":"/d-uspenskiy","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/47734295?s=80&v=4"},"commit":{"message":"[BACKPORT 2024.1][#21944] YSQL: Remove keep_order field from YbctidGenerator\n\nSummary:\n`keep_order` field should not be part of `YbctidGenerator`. This field is not used for\ngeneration, it is used for processing of generated result.\n\nOriginal commit: e224057d666aefa49f14deea5900057cc9157273 / D34064\n\nJira: DB-10861\n\nTest Plan: Jenkins\n\nReviewers: amartsinchyk, tnayak, jason, myang, pjain\n\nReviewed By: pjain\n\nSubscribers: yql\n\nTags: #jenkins-ready\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35338","shortMessageHtmlLink":"[BACKPORT 2024.1][<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2240026185\" data-permission-text=\"Title is private\" data-url=\"https://github.com/yugabyte/yugabyte-db/issues/21944\" data-hovercard-type=\"issue\" data-hovercard-url=\"/yugabyte/yugabyte-db/issues/21944/hovercard\" href=\"https://github.com/yugabyte/yugabyte-db/issues/21944\">#21944</a>] YSQL: Remove keep_order field from YbctidGe…"}},{"before":"216c60c918bf3bcc7b070f65aada792b1044f13e","after":"bc5b10edc2e27c71edb01d75d4c3d5902ef5d91d","ref":"refs/heads/pg15","pushedAt":"2024-05-27T04:12:21.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"devansh-ism","name":null,"path":"/devansh-ism","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/52438272?s=80&v=4"},"commit":{"message":"[pg15] fix: pg_stat_statements extension PG15.2 support\n\nSummary:\nThis diff adds the support for `pg_stat_statements` with an extra column of `yb_latency_histogram` on top of version 1.10.  Add a new version `1.10-yb-1.0` with upgrade path from `1.6-yb-1.0` to `1.10-yb-1.0`.\n\n`pg_stat_statements` retains the statistics data by saving the data in a file  on the disk (at `pg_stat/pg_stat_statements.stat`) during PostgreSQL shutdown and reads the file back at the startup to repopulate the entries. Using this mechanism allows PostgreSQL server to retain the data across graceful restart of the server. This diff supports this behaviour for PG 15.2. Retention of data and repopulating of shared memory during major version upgrade (from PG 11.2 to PG 15.2) will be addressed in separate diff.\n\npgssEntry struct:\n\n```\ntypedef struct pgssEntry\n{\n\tpgssHashKey\t\tkey;\t\t\t/* hash key of entry - MUST BE FIRST */\n\tCounters\t\tcounters;\t\t/* the statistics for this query */\n\tSize\t\t\tquery_offset;\t/* query text offset in external file */\n\tint\t\t\t\tquery_len;\t\t/* # of valid bytes in query string, or -1 */\n\tint\t\t\t\tencoding;\t\t/* query text encoding */\n\tslock_t\t\t\tmutex;\t\t\t/* protects the counters only */\n\tsize_t\t\t\tyb_slow_executions; /* # of executions >= yb_hdr_max_value * yb_hdr_latency_res_ms */\n\thdr_histogram\tyb_hdr_histogram; /* flexible array member at end - MUST BE LAST */\n} pgssEntry;\n```\n\nNoticeable changes from PG upstream\n\n  - Addition of planning statistics: 17e03282241c6ac58a714eb0c3b6a8018cf6167a\nThis commit makes pg_stat_statements support new GUC `pg_stat_statements.track_planning`. If this option is enabled, pg_stat_statements tracks the planning statistics of the statements, e.g., the number of times the statement was planned, the total time spent planning the statement, etc.\n\nDue to this all the in memory entry counters will have aggregated values in two valued array naming planning and execution.\n\n```\ntypedef enum pgssStoreKind {\n  PGSS_INVALID = -1,\n\n  /*\n   * PGSS_PLAN and PGSS_EXEC must be respectively 0 and 1 as they're used to\n   * reference the underlying values in the arrays in the Counters struct,\n   * and this order is required in pg_stat_statements_internal().\n   */\n  PGSS_PLAN = 0,\n  PGSS_EXEC,\n\n  PGSS_NUMKIND /* Must be last value of this enum */\n} pgssStoreKind;\n```\n\n```\ntypedef struct Counters\n{\n\tint64\t\tcalls[PGSS_NUMKIND];\t/* # of times planned/executed */\n\tdouble\t\ttotal_time[PGSS_NUMKIND];\t/* total planning/execution time,\n\t\t\t\t\t\t\t\t\t\t\t * in msec */\n\tdouble\t\tmin_time[PGSS_NUMKIND]; /* minimum planning/execution time in\n\t\t\t\t\t\t\t\t\t\t * msec */\n\tdouble\t\tmax_time[PGSS_NUMKIND]; /* maximum planning/execution time in\n\t\t\t\t\t\t\t\t\t\t * msec */\n\tdouble\t\tmean_time[PGSS_NUMKIND];\t/* mean planning/execution time in\n\t\t\t\t\t\t\t\t\t\t\t * msec */\n\tdouble\t\tsum_var_time[PGSS_NUMKIND]; /* sum of variances in\n```\n  - Track identical top vs nested queries independently: 6b4d23feef6e334fb85af077f2857f62ab781848\nThis change basically added a boolean value named `toplevel` to `pgsshashkey` to track statistics of same statements at top level and nested within a function. This changes structure of `pgsshashkey` which acts as key to `pgss_entry` in key-value hash map.\n\n```\ntypedef struct pgssHashKey\n{\n\tOid\t\t\tuserid;\t\t\t/* user OID */\n\tOid\t\t\tdbid;\t\t\t/* database OID */\n\tuint64\t\tqueryid;\t\t/* query identifier */\n\tbool\t\ttoplevel;\t\t/* query executed at top level */\n} pgssHashKey;\n```\n\n  - Display WAL usage statistics: 69bfaf2e1de49de76d7dec1c45511932a5ef502b\n  - Add JIT counters to pg_stat_statements: 57d6aea00fcefec3825a5948ce05cf2b4941097b\n  - Use has_privs_for_roles for predefined role checks: 6198420ad8a72e37f4fe4964616b17e0fd33b808\n  - Add some tests for older versions still usable: 2b0da0365bec6c62cc9c5c317bab6cbee3d52ef4\n\nIn total there are 82 commits between PG 11.2 and PG 15.2\n\n```\ngit log --format='%h %s' REL_11_2..REL_15_2 -- contrib/pg_stat_statements | wc -l\n      82\n```\n\nYB changes:\n\n  - `yb_latency_histogram` support was added as part of this diff: https://phorge.dev.yugabyte.com/D23163 . Changes as part of this are not mentioned here.\n  - Create new files `yb_pg_pg_stat_statements.sql `and `yb_pg_pg_stat_statements.out`. pg_stat_statements.sql and pg_stat_statements.out files are copy of PG upstream while **yb_pg_** files have changes for YB compatibility. Changes include\n   - Remove MERGE queries from test as currently not supported. (PG commit that added MERGE queries test: 72abf03b6491a8df880e1fea45798797bcc86c47)\n   - Remove Inherits queries from test as it is not supported yet.\n   - `DECLARE pgss_cursor CURSOR FOR SELECT * FROM pgss_matv ORDER BY a`; Add ORDER BY to get correct cursor output.\n   - Remove queries including selecting PG WAL related fields from pg_stat_statements. (PG commit that added WAL related queries 57d6aea00fcefec3825a5948ce05cf2b4941097b)\n  - As part of PG 15.2 merge `oldextversions.sql` and `oldextversions.out` files were added. But we do not add them in `yb_schedule` of TestPgRegressPgStatStatements#schedule test because these add tests for older versions which is not applicable for yb.\n  - Also create files `yb_pg_stat_statement.sql` and `yg_pg_stat_statements.out` to separate out YB specific tests including redacted password support added as part of https://phorge.dev.yugabyte.com/D14961. Added in `yb_schedule` also.\n  - Add YB_PG_STAT_STATEMENTS_COLS_V1_10 constant with value 44.  This is one more than PG_STAT_STATEMENTS_COLS_V1_10 (43) to add `yb_latency_histogram` column.\n  - Add api version `YB_PGSS_V1_10` to pgssVersion enum corresponding to `1.10-yb-1.0`.\n  - Enable pg_stat_statements extension creation in `initdb.c` via uncommenting the `enable_pg_stat_statements()`call. This was commented during PG15.2 merge.\n  - `Metrics.java`\n   - Modify `YSQLStat` class in such that it can be initialised correctly from the Json object returned from <tserver_ip>:13000/statements endpoint.\n   - Reason : With PG commit 17e03282241c6ac58a714eb0c3b6a8018cf6167a (titled, Allow pg_stat_statements to track planning statistics.) execution and planning statistics are tracked separately. Add distinction between planning and execution statistics while writing statement stats Json object by `getYsqlStatementStats` function.\n\n```\n\t\t\tWriteDoubleToJson(cb_arg, \"total_plan_time\", entry->counters.total_time[PGSS_PLAN]);\n\t\t\tWriteDoubleToJson(cb_arg, \"total_exec_time\", entry->counters.total_time[PGSS_EXEC]);\n```\n   - Use only execution statistics in test assertions as before PG commit 17e03282241c6ac58a714eb0c3b6a8018cf6167a (titled, Allow pg_stat_statements to track planning statistics.), only execution statistics were tracked.\n  - passing_tests.tsv: add these tests which are now passing\n   - `TestHdr`\n   - `TestPgRegressPgStatStatements`\n   - `TestYsqlMetrics`\n  - Disable password redaction of query in `pg_stat_statements.c`. Password redaction logic is causing some PL/pgSQL  procedures to fail with parsing error which is mentioned in https://yugabyte.atlassian.net/browse/DB-11332\n\nMakefile and extension upgrade sql file changes:\n\n - Only export `pg_stat_statements--1.10-yb-1.0.sql` , `pg_stat_statements--1.6-yb-1.0--1.10-yb-1.0.sql` and  `pg_stat_statements--1.6--1.6-yb-1.0.sql` via Makefile DATA. This will add support only for YB version whether in case of new installation (which would be to 1.10-yb-1.0) and upgrades.\n - pg_stat_statements--1.6-yb-1.0--1.10-yb-1.0.sql  have contents of:\n   - pg_stat_statements--1.6- - 1.7.sql\n   -  pg_stat_statements--1.7 - - 1.8.sql\n   -  pg_stat_statements--1.8 - - 1.9.sql\n   -  pg_stat_statements--1.9 - - 1.10.sql\n   -  pg_stat_statements--1.10 - - 1.10-yb.sql\n - pg_stat_statements--1.10-yb-1.0.sql  have contents of:\n   - pg_stat_statements--1.4.sql\n   -  pg_stat_statements--1.4-1.5.sql\n   -  pg_stat_statements--1.5-1.6.sql\n   -  pg_stat_statements--1.6 - 1.6-yb-1.0.sql\n   -  pg_stat_statements--1.6-yb-1.0--1.10-yb-1.0.sql\nThese files are added by manually copying them in single file with in between comments to distinguish from which file the statements are copied. Also in between echo statements are also removed having only single echo statement at top of these two files.\n\npg_stat_statements--1.6-yb-1.0--1.10-yb-1.0.sql:\n```\n-- complain if script is sourced in psql, rather than via ALTER EXTENSION\n\\echo Use \"ALTER EXTENSION pg_stat_statements UPDATE TO '1.10-yb-1.0'\" to load this file. \\quit\n\n/* Sourced from contrib/pg_stat_statements/pg_stat_statements--1.6--1.7.sql */\n\n/* First we have to remove them from the extension */\nALTER EXTENSION pg_stat_statements DROP FUNCTION pg_stat_statements_reset();\n\n/* Then we can drop them */\nDROP FUNCTION pg_stat_statements_reset();\n\n/* Now redefine */\nCREATE FUNCTION pg_stat_statements_reset(IN userid Oid DEFAULT 0,\n\tIN dbid Oid DEFAULT 0,\n\tIN queryid bigint DEFAULT 0\n)\nRETURNS void\nAS 'MODULE_PATHNAME', 'pg_stat_statements_reset_1_7'\nLANGUAGE C STRICT PARALLEL SAFE;\n\n-- Don't want this to be available to non-superusers.\nREVOKE ALL ON FUNCTION pg_stat_statements_reset(Oid, Oid, bigint) FROM PUBLIC;\n\n/* Sourced from contrib/pg_stat_statements/pg_stat_statements--1.7--1.8.sql */\n\"\"\"\ncode\n\"\"\"\n```\n\nTest Plan:\nJenkins: rebase: pg15\n\n  for _ in {1..50}; do grep TestHdr pg15_tests/passing_tests.tsv; done | pg15_tests/run_tests.sh\n  for _ in {1..50}; do grep TestPgRegressPgStatStatements pg15_tests/passing_tests.tsv; done | pg15_tests/run_tests.sh\n  for _ in {1..50}; do grep TestYsqlMetrics pg15_tests/passing_tests.tsv; done | pg15_tests/run_tests.sh\n\nReviewers: jason, aagrawal, smishra, skumar\n\nReviewed By: jason\n\nSubscribers: yql\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D34937","shortMessageHtmlLink":"[pg15] fix: pg_stat_statements extension PG15.2 support"}},{"before":"aad59ac64f632753b5d437f4991d80300f718dfa","after":"763a0d15f4ef7731c44cc65b7acf1f438a0bd047","ref":"refs/heads/master","pushedAt":"2024-05-27T00:53:44.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"ddhodge","name":"Dwight Hodge","path":"/ddhodge","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/79169168?s=80&v=4"},"commit":{"message":"[doc][yba] Backport node agent remove port ref #22466 (#22544)\n\n* node agen remove port ref\r\n\r\n* typos\r\n\r\n* Apply suggestions from code review\r\n\r\n* minor edits\r\n\r\n* minor edits\r\n\r\n---------\r\n\r\nCo-authored-by: Dwight Hodge <ghodge@yugabyte.com>\r\nCo-authored-by: Dwight Hodge <79169168+ddhodge@users.noreply.github.com>","shortMessageHtmlLink":"[doc][yba] Backport node agent remove port ref <a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2307869157\" data-permission-text=\"Title is private\" data-url=\"https://github.com/yugabyte/yugabyte-db/issues/22466\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/yugabyte/yugabyte-db/pull/22466/hovercard\" href=\"https://github.com/yugabyte/yugabyte-db/pull/22466\">#22466</a> (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2316639013\" data-permission-text=\"Title is private\" data-url=\"https://github.com/yugabyte/yugabyte-db/issues/22544\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/yugabyte/yugabyte-db/pull/22544/hovercard\" href=\"https://github.com/yugabyte/yugabyte-db/pull/22544\">#22544</a>)"}},{"before":"dccfb0f35cf5612fc8efb15857b7951da337604d","after":"8b56d8a1f4d513cf15beb79352da93532a5768bb","ref":"refs/heads/2024.1","pushedAt":"2024-05-26T13:33:33.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"d-uspenskiy","name":"Dmitry Uspenskiy","path":"/d-uspenskiy","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/47734295?s=80&v=4"},"commit":{"message":"[BACKPORT 2024.1][#20549] YSQL: Perform portal restart after instead of before transaction restart\n\nSummary:\nThis diff is the preparation for the further diff with actual fix for #20549. The fix will store YB related info `read_time_serial_no` in the postgres' native structure `SnapshotData`. In case of building `SnapshotData` it is required to store actual (i.e. non obsolete) value of `read_time_serial_no`. And this `read_time_serial_no` value should be valid after restarting YB's transaction.\nIn case of query failure YB may perform transparent retry. In this case portal is restarted and YB transaction may be restarted as well. Portal restart creates fresh snapshot (`SnapshotData` structure) and transaction restart performs `read_time_serial_no` update. To meet the requirement that new snapshot should have actual `read_time_serial_no` value it is necessary to restart YB's transaction before creation of the snapshot by portal restart.\nThis diff splits portal restart in 2 separate functions: `yb_clear_portal_before_restart` and `yb_attempt_to_retry_on_error`. Between these 2 function calls transaction is restarted (by the `yb_prepare_transaction_for_retry` function):\n\n```\nyb_clear_portal_before_restart\nyb_prepare_transaction_for_retry\nyb_attempt_to_retry_on_error\n```\n\n**Note:** it is necessary to clear the portal before transaction restart because current transaction owns portal and it will be freed by current transaction abort. The `yb_clear_portal_before_restart` function unattach the portal from current transaction to keep it alive even in case of transaction restart.\n\n**Note:** To simplify the code and increase readability some additional cleanup changes was implemented (including `restart` with `retry` substitution in names of some functions and structures)\nJira: DB-9557\n\nOriginal commit: b4f32139d5f0b290ee640005ba7be6f157a75646 / D34797\n\nTest Plan: Jenkins\n\nReviewers: pjain\n\nReviewed By: pjain\n\nSubscribers: yql\n\nTags: #jenkins-ready\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35316","shortMessageHtmlLink":"[BACKPORT 2024.1][<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2075363701\" data-permission-text=\"Title is private\" data-url=\"https://github.com/yugabyte/yugabyte-db/issues/20549\" data-hovercard-type=\"issue\" data-hovercard-url=\"/yugabyte/yugabyte-db/issues/20549/hovercard\" href=\"https://github.com/yugabyte/yugabyte-db/issues/20549\">#20549</a>] YSQL: Perform portal restart after instead …"}},{"before":"ca8d78d63610dd3e2f714c01c6401c6856f194b2","after":"dccfb0f35cf5612fc8efb15857b7951da337604d","ref":"refs/heads/2024.1","pushedAt":"2024-05-25T15:40:12.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"myang2021","name":null,"path":"/myang2021","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/79478033?s=80&v=4"},"commit":{"message":"[BACKPORT 2024.1][#22512] YSQL: Fix race condition in YBClient::CreateNamespaceIfNotExists\n\nSummary:\nIn YBClient::CreateNamespaceIfNotExists there is a race condition. If two\nconnections on two nodes call this function at the same time, then they will\nboth go ahead to create the namespace. Only one of them can go through on\nyb-master and the other will fail with an error like:\n\n```\n2024-05-22 08:26:48.078 UTC [43049] ERROR:  Keyspace with id '0000ffff000030008000000000000000' already exists\n```\n\nEven though the function `CreateNamespaceIfNotExists` appears to tolerate this\nerror, it does not retry for `YQLDatabase::YQL_DATABASE_PGSQL`.\n\nI changed the function `CreateNamespaceIfNotExists` to do one retry on \"already\nexists\" error for `YQLDatabase::YQL_DATABASE_PGSQL` type. Otherwise the old\nbehavior is retained.\n\nAdded a new unit test which would fail without the fix with an error like:\n```\n[ts-2] 2024-05-23 23:19:44.617 UTC [23883] ERROR:  Keyspace with id '0000ffff000030008000000000000000' already exists\n```\nJira: DB-11438\n\nOriginal commit: aad59ac64f632753b5d437f4991d80300f718dfa / D35306\n\nTest Plan: ./yb_build.sh --cxx-test pg_libpq-test --gtest_filter PgLibPqCreateSequenceNamespaceRaceTest.CreateSequenceNamespaceRaceTest -n 20\n\nReviewers: yguan\n\nReviewed By: yguan\n\nSubscribers: yql, ybase\n\nTags: #jenkins-ready\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35332","shortMessageHtmlLink":"[BACKPORT 2024.1][<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2313440103\" data-permission-text=\"Title is private\" data-url=\"https://github.com/yugabyte/yugabyte-db/issues/22512\" data-hovercard-type=\"issue\" data-hovercard-url=\"/yugabyte/yugabyte-db/issues/22512/hovercard\" href=\"https://github.com/yugabyte/yugabyte-db/issues/22512\">#22512</a>] YSQL: Fix race condition in YBClient::Creat…"}},{"before":"d212e435f51532b7193208f1c3c3ed7247b54143","after":"aad59ac64f632753b5d437f4991d80300f718dfa","ref":"refs/heads/master","pushedAt":"2024-05-25T04:18:25.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"myang2021","name":null,"path":"/myang2021","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/79478033?s=80&v=4"},"commit":{"message":"[#22512] YSQL: Fix race condition in YBClient::CreateNamespaceIfNotExists\n\nSummary:\nIn YBClient::CreateNamespaceIfNotExists there is a race condition. If two\nconnections on two nodes call this function at the same time, then they will\nboth go ahead to create the namespace. Only one of them can go through on\nyb-master and the other will fail with an error like:\n\n```\n2024-05-22 08:26:48.078 UTC [43049] ERROR:  Keyspace with id '0000ffff000030008000000000000000' already exists\n```\n\nEven though the function `CreateNamespaceIfNotExists` appears to tolerate this\nerror, it does not retry for `YQLDatabase::YQL_DATABASE_PGSQL`.\n\nI changed the function `CreateNamespaceIfNotExists` to do one retry on \"already\nexists\" error for `YQLDatabase::YQL_DATABASE_PGSQL` type. Otherwise the old\nbehavior is retained.\n\nAdded a new unit test which would fail without the fix with an error like:\n```\n[ts-2] 2024-05-23 23:19:44.617 UTC [23883] ERROR:  Keyspace with id '0000ffff000030008000000000000000' already exists\n```\nJira: DB-11438\n\nTest Plan: ./yb_build.sh --cxx-test pg_libpq-test --gtest_filter PgLibPqCreateSequenceNamespaceRaceTest.CreateSequenceNamespaceRaceTest -n 20\n\nReviewers: yguan\n\nReviewed By: yguan\n\nSubscribers: ybase, yql\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35306","shortMessageHtmlLink":"[<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2313440103\" data-permission-text=\"Title is private\" data-url=\"https://github.com/yugabyte/yugabyte-db/issues/22512\" data-hovercard-type=\"issue\" data-hovercard-url=\"/yugabyte/yugabyte-db/issues/22512/hovercard\" href=\"https://github.com/yugabyte/yugabyte-db/issues/22512\">#22512</a>] YSQL: Fix race condition in YBClient::CreateNamespaceIfNotEx…"}},{"before":"2929099465f79be85ab442313c8193fae16a451a","after":"250c6f1ebb0953056600a6327537142862c7961e","ref":"refs/heads/pg15-upgrade","pushedAt":"2024-05-25T02:21:00.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"svarnau","name":"Steve Varnau","path":"/svarnau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/5273190?s=80&v=4"},"commit":{"message":"[pg15-upgrade] merge: pg15 branch commit e2bd69b237811df5f4e5c599a694b08e73e0ac0c into pg15-upgrade\n\nSummary:\nMerge pg15 commit e2bd69b237811df5f4e5c599a694b08e73e0ac0c titled,\n  Revert \"[pg15] fix: pg_stat_statements extension PG15.2 support\"\nand committed 2024-05-07T14:37:59-07:00 into pg15-upgrade.\n\n - pg_yb_common.{c,h}: yb master commit bda4da792dd37fdb68372b0378e0a37f313081a1 removed YBEnableAsh() function. pg15-upgrade commit d25d78c3026c2c0481038e8065f0d2329e1ab296 added nearby lines and pg15-upgrade merge commit 61a436fe9b2345c08de7d3a2c66be3dbe3d3c648 put the two changes in sequence. Remove YBEnableAsh().\n - catalog_entity_info.proto: SysNamespaceEntryPB: yb master commit 51fbb38416de113174998f8554ca1e8ace83c270 added field clone_request_seq_no with field ID 7. pg15-upgrade commit 0be603365d8220bfc3fdedcc1d660575777134a0 added field ysql_next_major_version_state also with field ID 7. Adjust pg15-upgrade’s field number to 8 to avoid the conflict.\n - catalog_manager.cc: Adjacent lines between pg15-upgrade commit 0be603365d8220bfc3fdedcc1d660575777134a0 and yb master commit e9f71ee7e2714fc85bbef0a74aa7c71ded6bba0a. Trivial resolution.\n - pg_session.cc: pg15-upgrade commit 0be603365d8220bfc3fdedcc1d660575777134a0 added some additional logic to PgSession::DoRunAsync to compute whether non_ddl_txn_for_sys_tables_allowed is true. yb master commit 01a4cd802e146758af315cb3cf0e713cd7fe41fd restructured DoRunAsync. Carefully merge the new master branch logic with pg15-upgrade changes.\n - pg_dump.c: yb master commit 5659b73a36611898ff508ab36897a3b0aad655e7 added a psql meta-command which pg_restore cannot execute. Condition the logic on binary_upgrade and add a YB_TODO to investigate further.\n - relcache.c: YbRunWithPrefetcher: yb master commit 103ef93bc14547291ea90a5f98ecb22fbe8e3c82 enabled read request caching by default, which breaks test_upgrade_index.sh and test_upgrade_dropped_col.sh. I investigated the index test, and the dropped_col test is probably similar. The cause of the breakage is that enabling the flag more aggressively updates the catalog. Upstream PG drops and recreates each database as part of the upgrade, and YB has some code (mostly in catalog_manager.cc) to accommodate this. In YB, databases are represented by namespaces in the catalog manager. Since namespaces in YB are versionless and we semi-ignore the drop namespace, if a catalog update occurs in the middle, the PG code sees the database name as being in use, and blocks “creating” the database again. Catalog updates should not occur at all during the upgrade, but a full rethinking is outside the scope of a merge. As a workaround, treat binary upgrade the same way as initdb and skip the prefetching initialization. Add a YB_TODO to decide if this is the right approach.\n\nTest Plan:\nJenkins\n\nOn MacOS 14 arm64:\n  ./yb_build.sh release --sj\n  pg15_tests/get_shell_test_specs.sh | grep upgrade | pg15_tests/run_tests.sh\n\nOn AlmaLinux 8:\n  ./yb_build.sh fastdebug --sj\n  pg15_tests/run_tests.sh\n\nOf the AlmaLinux tests, the following tests fail because they require release mode, a known limitation:\n * `SHELL\ttest_upgrade_comment`\n * `SHELL\ttest_upgrade`\n * `SHELL\ttest_upgrade_dropped_col`\n * `SHELL\ttest_upgrade_index`\n * `SHELL\ttest_upgrade_preflight`\n\nOf the AlmaLinux tests, the following test fails, but it also fails on the pg15 commit being merged:\n * `JAVA    org.yb.pgsql.TestPgSelect#testNullPushdown`\n\nOf the AlmaLinux tests, the following test failed but succeeded when tried again, so possibly flaky:\n * `integration-tests_cdcsdk_ysql-test      CDCSDKYsqlTest.TestIntentCountPersistencyBootstrapImplicit`\n\nI ran all flaming Detective tests on my AlmaLinux 8 dev server. All passed except for:\n```\n./yb_build.sh release --cxx-test pgwrapper_pg_libpq-test --gtest_filter PgLibPqTest.TestParallelCounterSerializable --clang17\n```\nThis test also fails on the pg15 commit being merged.\n\nReviewers: jason\n\nReviewed By: jason\n\nSubscribers: ybase, yql\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D34810","shortMessageHtmlLink":"[pg15-upgrade] merge: pg15 branch commit <a class=\"commit-link\" data-hovercard-type=\"commit\" data-hovercard-url=\"https://github.com/yugabyte/yugabyte-db/commit/e2bd69b237811df5f4e5c599a694b08e73e0ac0c/hovercard\" href=\"https://github.com/yugabyte/yugabyte-db/commit/e2bd69b237811df5f4e5c599a694b08e73e0ac0c\"><tt>e2bd69b</tt></a> into pg15-upgrade"}},{"before":"fe7d36e858412a209a7e4c29a45e198cc3af5db2","after":"d212e435f51532b7193208f1c3c3ed7247b54143","ref":"refs/heads/master","pushedAt":"2024-05-25T02:18:39.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"svarnau","name":"Steve Varnau","path":"/svarnau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/5273190?s=80&v=4"},"commit":{"message":"[PLAT-13829] Upgrade commons-compress to 1.26.0\n\nSummary:\nFix a CVE in org.apache.commons:commons-compress by bumping it\nup from 1.25.0 to 1.26.0\n\nTest Plan: Ran dev Itests\n\nReviewers: muthu, anijhawan\n\nReviewed By: muthu\n\nSubscribers: yugaware\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35176","shortMessageHtmlLink":"[PLAT-13829] Upgrade commons-compress to 1.26.0"}},{"before":"23ce4a39010475d7376514ba4e063325e12abf93","after":"fe7d36e858412a209a7e4c29a45e198cc3af5db2","ref":"refs/heads/master","pushedAt":"2024-05-25T01:33:06.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"svarnau","name":"Steve Varnau","path":"/svarnau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/5273190?s=80&v=4"},"commit":{"message":"[PLAT-13829] Upgrade commons-compress to 1.26.0\n\nSummary:\nFix a CVE in org.apache.commons:commons-compress by bumping it\nup from 1.25.0 to 1.26.0\n\nTest Plan: Ran dev Itests\n\nReviewers: muthu, anijhawan\n\nReviewed By: muthu\n\nSubscribers: yugaware\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35176","shortMessageHtmlLink":"[PLAT-13829] Upgrade commons-compress to 1.26.0"}},{"before":"506e9f43a79a515be6a5746180927ff768d873c6","after":"216c60c918bf3bcc7b070f65aada792b1044f13e","ref":"refs/heads/pg15","pushedAt":"2024-05-25T00:58:02.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"svarnau","name":"Steve Varnau","path":"/svarnau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/5273190?s=80&v=4"},"commit":{"message":"[pg15] fix: port tablegroups in ysql_dump\n\nSummary:\nA number of changes have taken place in `pg_dump` between PG11 and PG15, that have an impact on tablegroup dumps in YugabyteDB:\n * The way storage and lookup of objects for the dump is done was changed in commit https://github.com/postgres/postgres/commit/92316a4582a5714d4e494aaf90360860e7fec37a\n * “Rethink pg_dump’s handling of object ACLs” https://github.com/postgres/postgres/commit/0c9d84427f441602425b0e18be5cfe751d1d8ebe#diff-36e3806266f5e832b422f878e1d7fef4ebeea2143b1733ea3fa2f90a99bad5c5\n * Role names now retrieved through `getRoleName()` https://github.com/postgres/postgres/commit/d5e8930f50e31d836d84b353b9dadedd5007bb70#diff-470e1ed6b3775d53b5e971a7bc2640c55b20d405b66c114f28b49b392a7850ed\n\nI applied these changes to enable tablegroup dumps in PG15, and also cleaned up the SQL which had an unnecessary table join, and adjusted the ordering of tablegroups to be sorted at the same point they were in PG11.\n\nTest Plan:\nJenkins: compile only\n\nIn `ysqlsh`:\n```\nCREATE TABLEGROUP grp1;\nCREATE TABLE tgrp1 (a int) TABLEGROUP grp1;\n\nCREATE TABLESPACE tsp1 LOCATION '/data';\nCREATE TABLEGROUP grp_with_spc TABLESPACE tsp1;\nCREATE TABLE tgroup_with_spc (a INT) TABLEGROUP grp_with_spc;\n\nCREATE USER tablegroup_test_user SUPERUSER;\nSET SESSION AUTHORIZATION tablegroup_test_user;\nCREATE TABLEGROUP grp2;\nCREATE TABLE tgrp2 (a int) TABLEGROUP grp2;\n```\nThen:\n```\nbuild/latest/postgres/bin/ysql_dump --include-yb-metadata\n```\n\nReviewers: yguan\n\nReviewed By: yguan\n\nSubscribers: aagrawal, smishra, yql\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D29541","shortMessageHtmlLink":"[pg15] fix: port tablegroups in ysql_dump"}},{"before":"84b41cd99d76e982c083e3614d3689f166bae03b","after":"1a0c0fbf64e057fdb54db28c3d4a3fbce3805849","ref":"refs/heads/pg15_backup","pushedAt":"2024-05-24T23:37:42.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"jharveysmith","name":"Jeff Harvey-Smith","path":"/jharveysmith","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6226895?s=80&v=4"},"commit":{"message":"[MERGE PG15] Adds missing oid columns and indexes to YB system tables\n\nSummary: Adds missing oid columns and indexes to YB system tables\n\nTest Plan: N/A\n\nReviewers: mihnea, neil\n\nReviewed By: neil\n\nSubscribers: yql\n\nDifferential Revision: https://phabricator.dev.yugabyte.com/D24959","shortMessageHtmlLink":"[MERGE PG15] Adds missing oid columns and indexes to YB system tables"}},{"before":"3fae996d14d95907c52d069719b147fe2270fb7b","after":"21c4e27ae9a86c80c96f32d84f0f775d3565c885","ref":"refs/heads/tracing","pushedAt":"2024-05-24T23:37:42.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"jharveysmith","name":"Jeff Harvey-Smith","path":"/jharveysmith","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6226895?s=80&v=4"},"commit":{"message":"[PLAT-11862] Turning off background tasks for YBA follower\n\nSummary: Today follower YBA operates almost exactly the same as HA leader, except for API being disabled. We also want any background tasks/threads that are started during AppInit to be skipped for follower YBA, as there isn't a reason it should be operational and it could lead to more corner cases where universes were operated on with incorrect metadata.\n\nTest Plan: Start up follower with diff, ensure that standby promotion and all still works but that AppInit doesn't fire off taskGC and others.\n\nReviewers: dshubin, sanketh, nsingh, amalyshev\n\nReviewed By: nsingh, amalyshev\n\nSubscribers: yugaware\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D32719","shortMessageHtmlLink":"[PLAT-11862] Turning off background tasks for YBA follower"}},{"before":"a6efc579da7c4cb3762aba6f97119f002b5716f4","after":"23ce4a39010475d7376514ba4e063325e12abf93","ref":"refs/heads/master","pushedAt":"2024-05-24T23:37:42.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"jharveysmith","name":"Jeff Harvey-Smith","path":"/jharveysmith","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6226895?s=80&v=4"},"commit":{"message":"[#22512] YSQL: Fix race condition in YBClient::CreateNamespaceIfNotExists\n\nSummary:\nIn YBClient::CreateNamespaceIfNotExists there is a race condition. If two\nconnections on two nodes call this function at the same time, then they will\nboth go ahead to create the namespace. Only one of them can go through on\nyb-master and the other will fail with an error like:\n\n```\n2024-05-22 08:26:48.078 UTC [43049] ERROR:  Keyspace with id '0000ffff000030008000000000000000' already exists\n```\n\nEven though the function `CreateNamespaceIfNotExists` appears to tolerate this\nerror, it does not retry for `YQLDatabase::YQL_DATABASE_PGSQL`.\n\nI changed the function `CreateNamespaceIfNotExists` to do one retry on \"already\nexists\" error for `YQLDatabase::YQL_DATABASE_PGSQL` type. Otherwise the old\nbehavior is retained.\n\nAdded a new unit test which would fail without the fix with an error like:\n```\n[ts-2] 2024-05-23 23:19:44.617 UTC [23883] ERROR:  Keyspace with id '0000ffff000030008000000000000000' already exists\n```\nJira: DB-11438\n\nTest Plan: ./yb_build.sh --cxx-test pg_libpq-test --gtest_filter PgLibPqCreateSequenceNamespaceRaceTest.CreateSequenceNamespaceRaceTest -n 20\n\nReviewers: yguan\n\nReviewed By: yguan\n\nSubscribers: ybase, yql\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35306","shortMessageHtmlLink":"[<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2313440103\" data-permission-text=\"Title is private\" data-url=\"https://github.com/yugabyte/yugabyte-db/issues/22512\" data-hovercard-type=\"issue\" data-hovercard-url=\"/yugabyte/yugabyte-db/issues/22512/hovercard\" href=\"https://github.com/yugabyte/yugabyte-db/issues/22512\">#22512</a>] YSQL: Fix race condition in YBClient::CreateNamespaceIfNotEx…"}},{"before":"250c6f1ebb0953056600a6327537142862c7961e","after":"2929099465f79be85ab442313c8193fae16a451a","ref":"refs/heads/pg15-upgrade","pushedAt":"2024-05-24T23:37:42.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"jharveysmith","name":"Jeff Harvey-Smith","path":"/jharveysmith","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6226895?s=80&v=4"},"commit":{"message":"[pg15-upgrade] merge: pg15 branch commit 5cf7921073176a21df8c7b657f73c595a4ac5b9c into pg15-upgrade\n\nSummary:\nMerge pg15 commit 5cf7921073176a21df8c7b657f73c595a4ac5b9c titled,\n  Revert \"[pg15] fix: pg_stat_statements extension PG15.2 support\"\nand committed 2024-05-07T14:37:59-07:00 into pg15-upgrade.\n\n - pg_yb_common.{c,h}: yb master commit cb19775eff2c18f33fcb0ea2bb96f28281d2fd6c removed YBEnableAsh() function. pg15-upgrade commit 9163ebcc195c8ae267f25034e3e4ade312ec76bf added nearby lines and pg15-upgrade merge commit b607f33415f419aed74188c975b42f10cd81a678 put the two changes in sequence. Remove YBEnableAsh().\n - catalog_entity_info.proto: SysNamespaceEntryPB: yb master commit 1bc8d6c19bec5842f13e8104fd09d8e4787c3350 added field clone_request_seq_no with field ID 7. pg15-upgrade commit 3703818c1e772ed41224075c06674e43b9884e30 added field ysql_next_major_version_state also with field ID 7. Adjust pg15-upgrade’s field number to 8 to avoid the conflict.\n - catalog_manager.cc: Adjacent lines between pg15-upgrade commit 3703818c1e772ed41224075c06674e43b9884e30 and yb master commit 542f80fd511a7d9c2849ea7a59b3e081339013e5. Trivial resolution.\n - pg_session.cc: pg15-upgrade commit 3703818c1e772ed41224075c06674e43b9884e30 added some additional logic to PgSession::DoRunAsync to compute whether non_ddl_txn_for_sys_tables_allowed is true. yb master commit 4e8e5cafb0711f85a7764cf7d18c716ad297f02b restructured DoRunAsync. Carefully merge the new master branch logic with pg15-upgrade changes.\n - pg_dump.c: yb master commit f6f67b25600833352abcd3e942ba5b63d5b04dce added a psql meta-command which pg_restore cannot execute. Condition the logic on binary_upgrade and add a YB_TODO to investigate further.\n - relcache.c: YbRunWithPrefetcher: yb master commit 19d60dce3c0e3d39fdef85b47c81f7523840961b enabled read request caching by default, which breaks test_upgrade_index.sh and test_upgrade_dropped_col.sh. I investigated the index test, and the dropped_col test is probably similar. The cause of the breakage is that enabling the flag more aggressively updates the catalog. Upstream PG drops and recreates each database as part of the upgrade, and YB has some code (mostly in catalog_manager.cc) to accommodate this. In YB, databases are represented by namespaces in the catalog manager. Since namespaces in YB are versionless and we semi-ignore the drop namespace, if a catalog update occurs in the middle, the PG code sees the database name as being in use, and blocks “creating” the database again. Catalog updates should not occur at all during the upgrade, but a full rethinking is outside the scope of a merge. As a workaround, treat binary upgrade the same way as initdb and skip the prefetching initialization. Add a YB_TODO to decide if this is the right approach.\n\nTest Plan:\nJenkins\n\nOn MacOS 14 arm64:\n  ./yb_build.sh release --sj\n  pg15_tests/get_shell_test_specs.sh | grep upgrade | pg15_tests/run_tests.sh\n\nOn AlmaLinux 8:\n  ./yb_build.sh fastdebug --sj\n  pg15_tests/run_tests.sh\n\nOf the AlmaLinux tests, the following tests fail because they require release mode, a known limitation:\n * `SHELL\ttest_upgrade_comment`\n * `SHELL\ttest_upgrade`\n * `SHELL\ttest_upgrade_dropped_col`\n * `SHELL\ttest_upgrade_index`\n * `SHELL\ttest_upgrade_preflight`\n\nOf the AlmaLinux tests, the following test fails, but it also fails on the pg15 commit being merged:\n * `JAVA    org.yb.pgsql.TestPgSelect#testNullPushdown`\n\nOf the AlmaLinux tests, the following test failed but succeeded when tried again, so possibly flaky:\n * `integration-tests_cdcsdk_ysql-test      CDCSDKYsqlTest.TestIntentCountPersistencyBootstrapImplicit`\n\nI ran all flaming Detective tests on my AlmaLinux 8 dev server. All passed except for:\n```\n./yb_build.sh release --cxx-test pgwrapper_pg_libpq-test --gtest_filter PgLibPqTest.TestParallelCounterSerializable --clang17\n```\nThis test also fails on the pg15 commit being merged.\n\nReviewers: jason\n\nReviewed By: jason\n\nSubscribers: ybase, yql\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D34810","shortMessageHtmlLink":"[pg15-upgrade] merge: pg15 branch commit <a class=\"commit-link\" data-hovercard-type=\"commit\" data-hovercard-url=\"https://github.com/yugabyte/yugabyte-db/commit/5cf7921073176a21df8c7b657f73c595a4ac5b9c/hovercard\" href=\"https://github.com/yugabyte/yugabyte-db/commit/5cf7921073176a21df8c7b657f73c595a4ac5b9c\"><tt>5cf7921</tt></a> into pg15-upgrade"}},{"before":"dce576f2974b5ea22da581f0951af537d1e2a662","after":"35ec1924bbe64afd60b1fc6c78f3fa02ed5d3464","ref":"refs/heads/yb_auh","pushedAt":"2024-05-24T23:37:42.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"jharveysmith","name":"Jeff Harvey-Smith","path":"/jharveysmith","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6226895?s=80&v=4"},"commit":{"message":"added table state; (#20324)\n\nCo-authored-by: Nandhini Shenoy <nandhini.shenoy@yugabyte.com>","shortMessageHtmlLink":"added table state; (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2043028655\" data-permission-text=\"Title is private\" data-url=\"https://github.com/yugabyte/yugabyte-db/issues/20324\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/yugabyte/yugabyte-db/pull/20324/hovercard\" href=\"https://github.com/yugabyte/yugabyte-db/pull/20324\">#20324</a>)"}},{"before":"5d2dc7425abe43432d6579d5f6c7a73c7e7c7603","after":"484629ddcee3734d2a6f28229090dcffb74ab4ea","ref":"refs/heads/build-and-learn-tutorial","pushedAt":"2024-05-24T23:37:42.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"jharveysmith","name":"Jeff Harvey-Smith","path":"/jharveysmith","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6226895?s=80&v=4"},"commit":{"message":"minor edits","shortMessageHtmlLink":"minor edits"}},{"before":"216c60c918bf3bcc7b070f65aada792b1044f13e","after":"506e9f43a79a515be6a5746180927ff768d873c6","ref":"refs/heads/pg15","pushedAt":"2024-05-24T23:37:42.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"jharveysmith","name":"Jeff Harvey-Smith","path":"/jharveysmith","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6226895?s=80&v=4"},"commit":{"message":"[pg15] fix: port tablegroups in ysql_dump\n\nSummary:\nA number of changes have taken place in `pg_dump` between PG11 and PG15, that have an impact on tablegroup dumps in YugabyteDB:\n * The way storage and lookup of objects for the dump is done was changed in commit https://github.com/postgres/postgres/commit/92316a4582a5714d4e494aaf90360860e7fec37a\n * “Rethink pg_dump’s handling of object ACLs” https://github.com/postgres/postgres/commit/0c9d84427f441602425b0e18be5cfe751d1d8ebe#diff-36e3806266f5e832b422f878e1d7fef4ebeea2143b1733ea3fa2f90a99bad5c5\n * Role names now retrieved through `getRoleName()` https://github.com/postgres/postgres/commit/d5e8930f50e31d836d84b353b9dadedd5007bb70#diff-470e1ed6b3775d53b5e971a7bc2640c55b20d405b66c114f28b49b392a7850ed\n\nI applied these changes to enable tablegroup dumps in PG15, and also cleaned up the SQL which had an unnecessary table join, and adjusted the ordering of tablegroups to be sorted at the same point they were in PG11.\n\nTest Plan:\nJenkins: compile only\n\nIn `ysqlsh`:\n```\nCREATE TABLEGROUP grp1;\nCREATE TABLE tgrp1 (a int) TABLEGROUP grp1;\n\nCREATE TABLESPACE tsp1 LOCATION '/data';\nCREATE TABLEGROUP grp_with_spc TABLESPACE tsp1;\nCREATE TABLE tgroup_with_spc (a INT) TABLEGROUP grp_with_spc;\n\nCREATE USER tablegroup_test_user SUPERUSER;\nSET SESSION AUTHORIZATION tablegroup_test_user;\nCREATE TABLEGROUP grp2;\nCREATE TABLE tgrp2 (a int) TABLEGROUP grp2;\n```\nThen:\n```\nbuild/latest/postgres/bin/ysql_dump --include-yb-metadata\n```\n\nReviewers: yguan\n\nReviewed By: yguan\n\nSubscribers: aagrawal, smishra, yql\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D29541","shortMessageHtmlLink":"[pg15] fix: port tablegroups in ysql_dump"}},{"before":"64b6eee6171d323d04dc4a26f7170eb64f91eee6","after":"524d0a6aff8651fa93a68c0aea5e09a34e2d3624","ref":"refs/heads/rename","pushedAt":"2024-05-24T23:37:42.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"jharveysmith","name":"Jeff Harvey-Smith","path":"/jharveysmith","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6226895?s=80&v=4"},"commit":{"message":"[WIP] rename Remote Filter to Storage Filter\n\nTest Plan: jenkins\n\nReviewers: asaha\n\nSubscribers: bogdan\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D32049","shortMessageHtmlLink":"[WIP] rename Remote Filter to Storage Filter"}},{"before":"644582f275ab8411e88df241d3036af6a41d2131","after":"60f60038b6b197bcdf7154a0e9c19a978dd36cf1","ref":"refs/heads/PLAT-13223","pushedAt":"2024-05-24T23:37:42.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"jharveysmith","name":"Jeff Harvey-Smith","path":"/jharveysmith","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6226895?s=80&v=4"},"commit":{"message":"swagger","shortMessageHtmlLink":"swagger"}},{"before":"dd725427a74c095f3d61f69e3ee458b8c9c3f1ed","after":"6f6a8d35e2a4117b81c176d30650a9b69f64a319","ref":"refs/heads/rthallamko3-patch-1","pushedAt":"2024-05-24T23:37:42.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"jharveysmith","name":"Jeff Harvey-Smith","path":"/jharveysmith","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6226895?s=80&v=4"},"commit":{"message":"Update performance.md\n\nRemove references to ysql_enable_packed_row as the gflag ison by default starting 2.20","shortMessageHtmlLink":"Update performance.md"}},{"before":"2415cdced7ab0b560095cce87ea242b0c0d49ac5","after":"6fc1507ed0a7c74510a3384ef7a1fd1cb00b6383","ref":"refs/heads/mock-ts-service","pushedAt":"2024-05-24T23:37:42.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"jharveysmith","name":"Jeff Harvey-Smith","path":"/jharveysmith","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6226895?s=80&v=4"},"commit":{"message":"(PLAT)-Add common component for Date time picker","shortMessageHtmlLink":"(PLAT)-Add common component for Date time picker"}},{"before":"cc96fcf45f8bb3dc5161657121e07f9bbf440c64","after":"a3c237e1cad72b95ca6eb78b4638a24cd6998507","ref":"refs/heads/DEVOPS-2849","pushedAt":"2024-05-24T23:37:42.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"jharveysmith","name":"Jeff Harvey-Smith","path":"/jharveysmith","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6226895?s=80&v=4"},"commit":{"message":"Increase test failure threshold","shortMessageHtmlLink":"Increase test failure threshold"}},{"before":"2f87e0eb2fb6495904f90e53bd529b8b3256f6e7","after":"a9c399a5874505e36e8172a4c5c92815f4ed907f","ref":"refs/heads/origin","pushedAt":"2024-05-24T23:37:42.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"jharveysmith","name":"Jeff Harvey-Smith","path":"/jharveysmith","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6226895?s=80&v=4"},"commit":{"message":"[BACKPORT 2.20.1][PLAT-9799] Create an API to sync DB users with LDAP\n\nSummary:\nOriginal commit: cf2f141a14d6d036a8727111c93fbe57867278f6 / D28391\nThe API exposed will automatically sync the users and roles from the upstream LDAP server to a YBDB cluster running YSQL/YCQL. This is a YBA implementation of the manual script\nused by our customers to sync LDAP users and roles periodically with YBDB users and roles. Here is a link to the original script from Alan : https://yugabyte.slack.com/files/U02RTQ157BN/F05BH681YRZ/yb_sync_ldap.py .\n\nThe API that is added:\nPOST /customers/:cUUID/universes/:univUUID/ldap_roles_sync\n\nParameters - customerUUID, universeUUID\nRequest Body - A JSON body\n{\n  \"targetApi\": \"ycql\",\n  \"dbUser\": \"\",\n  \"dbuserPassword\": \"\",\n  \"ldapServer\": \"\",\n  \"ldapPort\": \"\",\n  \"ldapBindDn\": \"\",\n  \"ldapBindPassword\": \"\",\n  \"ldapSearchFilter\": \"\",\n  \"ldapBasedn\": \"\",\n  \"ldapGroupMemberOfAttribute\":\"\",\n  \"ldapUserfield\": \"cn\",\n  \"ldapGroupfield\": \"cn\",\n  \"useLdapTls\": \"\",\n  \"createGroups\": \"\",\n  \"allowDropSuperuser\": \"\",\n  \"excludeUsers\": [],\n  \"ldapTlsProtocol\": \"TLSv1_2\"\n}\n\nResponse Body :\n{\n    \"taskUUID\": \"<taskUUID>\",\n    \"resourceUUID\": \"<univUUID>\"\n}\nError cases:\n\n  - LDAP Authentication/Connection Failure: Exception is raised\n\n  - DB Authentication Failure: Exception is raised\n\n  - DB Action Failure: We will fail the YBA task, leaving the system in partial synced state & at the same time calling out the failures that occurred as part of the sync that users can choose to resolve manually -or- can retrigger the sync post fixing the issue which will skip the users that are added before the failure as part of computing the diff.\n\nExample: - curl --request POST 'http://localhost:9000/api/v1/customers/<customer-uuid>/universes/<universe-uuid>/ldap_sync' \\\n--header 'X-AUTH-YW-API-TOKEN: <token>' \\\n--header 'Content-Type: application/json' \\\n--data '{\n  \"targetApi\": \"ycql\",\n  \"dbUser\": \"cassandra\",\n  \"dbuserPassword\": \"test\",\n  \"ldapServer\": \"0.0.0.0\",\n  \"ldapPort\": \"389\",\n  \"ldapBindDn\": \"cn=test,dc=example,dc=org\",\n  \"ldapBindPassword\": \"test\",\n  \"ldapSearchFilter\": \"(objectclass=person)\",\n  \"ldapBasedn\": \"dc=example,dc=org\",\n  \"ldapGroupMemberOfAttribute\": \"memberOf\",\n  \"ldapUserfield\": \"CN\",\n  \"ldapGroupfield\": \"CN\",\n  \"useLdapTls\": \"false\",\n  \"createGroups\": \"true\",\n  \"allowDropSuperuser\": \"true\",\n  \"excludeUsers\": [\"test\"],\n  \"ldapTlsProtocol\": \"TLSv1_2\"\n}'\n\nTest Plan: https://docs.google.com/spreadsheets/d/1HVRT9qfnaFcInJbHNgRNjnObCzySp6m3_AVH7SOr3xc/edit#gid=875409392\n\nReviewers: svarshney, nbhatia, #yba-api-review\n\nReviewed By: svarshney, #yba-api-review\n\nSubscribers: yugaware, nbhatia, sneelakantan\n\nTags: #jenkins-ready\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D30718","shortMessageHtmlLink":"[BACKPORT 2.20.1][PLAT-9799] Create an API to sync DB users with LDAP"}},{"before":"ae4ff977762007a2e11db55597986036bfa15fe5","after":"c3c62701e6e0a2dbd4569c63a56bd31b81f90754","ref":"refs/heads/sgarg/test-ipv6","pushedAt":"2024-05-24T23:37:42.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"jharveysmith","name":"Jeff Harvey-Smith","path":"/jharveysmith","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6226895?s=80&v=4"},"commit":{"message":"[yugabyted] Making changes to support IPv^ addresses in format like 0:0:0:0:0:0:1","shortMessageHtmlLink":"[yugabyted] Making changes to support IPv^ addresses in format like 0…"}},{"before":"c930bdf50a57a476352236418f452bdc4abaa7e5","after":"d08f5f053c354d35dec5678f931dd968aa47d57b","ref":"refs/heads/DistributedTracing","pushedAt":"2024-05-24T23:37:42.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"jharveysmith","name":"Jeff Harvey-Smith","path":"/jharveysmith","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6226895?s=80&v=4"},"commit":{"message":"explain works","shortMessageHtmlLink":"explain works"}},{"before":"9aa768235e59a3c3415889cde88d8ffba7cb228c","after":"f160732b654bcfb08f3b15a3d55637a3d9f537d9","ref":"refs/heads/temp_2023.2","pushedAt":"2024-05-24T23:37:42.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"jharveysmith","name":"Jeff Harvey-Smith","path":"/jharveysmith","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6226895?s=80&v=4"},"commit":{"message":"Temorary test of version number change 2023.2.x.x\n\nBranched from 2.21.0.0-b312","shortMessageHtmlLink":"Temorary test of version number change 2023.2.x.x"}},{"before":"d2d922cb627b4f26cecf5324733b534db1510981","after":"f45b3125e8bce1f32ff03114259f183f8a0e56e3","ref":"refs/heads/test-tf-demo","pushedAt":"2024-05-24T23:37:42.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"jharveysmith","name":"Jeff Harvey-Smith","path":"/jharveysmith","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6226895?s=80&v=4"},"commit":{"message":"Merge branch 'master' into test-tf-demo","shortMessageHtmlLink":"Merge branch 'master' into test-tf-demo"}},{"before":"5ad2b580fc03c7c188e3ad57e582afb7fe26fb2e","after":"fbe5b0e8da0b738580ab8934207a92cabb4ae665","ref":"refs/heads/DOC-65-Transaction-prepare-bind-syntax-errors","pushedAt":"2024-05-24T23:37:42.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"jharveysmith","name":"Jeff Harvey-Smith","path":"/jharveysmith","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6226895?s=80&v=4"},"commit":{"message":"Update acid-transactions-ycql.md","shortMessageHtmlLink":"Update acid-transactions-ycql.md"}},{"before":"f37018322fd69136fc1afa80f319e835955b1176","after":"a6efc579da7c4cb3762aba6f97119f002b5716f4","ref":"refs/heads/master","pushedAt":"2024-05-24T23:22:02.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"shahrooz1997","name":"Hamidreza Zare","path":"/shahrooz1997","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/23063104?s=80&v=4"},"commit":{"message":"[PLAT-14110] Fix universeUtils.test.js utest failure\n\nSummary:\nThis diff removes '=' from the utest sample string to fix the UI utest which was added in D35321.\n\nTest Plan: Made sure the utest for universeUtils.test.js passes after this change.\n\nReviewers: rmadhavan, jmak, sanketh, cwang\n\nReviewed By: cwang\n\nSubscribers: yugaware\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35330","shortMessageHtmlLink":"[PLAT-14110] Fix universeUtils.test.js utest failure"}},{"before":"25556db2b094f3c1f5e6229499c73b1de3d266c6","after":"2415cdced7ab0b560095cce87ea242b0c0d49ac5","ref":"refs/heads/mock-ts-service","pushedAt":"2024-05-24T23:20:35.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"jharveysmith","name":"Jeff Harvey-Smith","path":"/jharveysmith","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/6226895?s=80&v=4"},"commit":{"message":"(PLAT)-Add common component for Date time picker","shortMessageHtmlLink":"(PLAT)-Add common component for Date time picker"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAEVKW45gA","startCursor":null,"endCursor":null}},"title":"Activity · yugabyte/yugabyte-db"}