[CDCSDK] Fix race condition between tablet creation and addition of table to the stream in the background thread #22408
Assignees
Labels
Comments
yugabyte-ci
added
area/cdcsdk
yugabyte-ci
changed the title
yugabyte-ci
added
priority/high
Sumukh-Phalgaonkar
added a commit
that referenced
this issue
May 22, 2024
Summary: There is a possible race condition when during dynamic table creation, the tablet creation is not completed i.e the retention barriers have not yet been set, but the background thread proceeds and adds the table to the stream. In such a case till the time the tablet creation succeeds, the retention barriers are not set on the tablets of this table, even though it has been added to the stream. This was observed as flakiness in the test `TestRetentionBarrierRaceWithUpdatePeersAndMetrics` . This diff introduces a fix for this race condition by calling GetTableLocations on the table before adding it to the stream. If the tablets are not yet initialized then this will fail and the addition of the table to the stream will take place in the next round of the background task. Jira: DB-11311 Test Plan: Jenkins: test regex: .*CDCSDKConsumptionConsistentChangesTest.* Reviewers: asrinivasan, stiwary, skumar Reviewed By: asrinivasan Subscribers: ybase, ycdcxcluster Tags: #jenkins-ready Differential Revision: https://phorge.dev.yugabyte.com/D35160
svarnau
pushed a commit
that referenced
this issue
May 25, 2024
Summary: There is a possible race condition when during dynamic table creation, the tablet creation is not completed i.e the retention barriers have not yet been set, but the background thread proceeds and adds the table to the stream. In such a case till the time the tablet creation succeeds, the retention barriers are not set on the tablets of this table, even though it has been added to the stream. This was observed as flakiness in the test `TestRetentionBarrierRaceWithUpdatePeersAndMetrics` . This diff introduces a fix for this race condition by calling GetTableLocations on the table before adding it to the stream. If the tablets are not yet initialized then this will fail and the addition of the table to the stream will take place in the next round of the background task. Jira: DB-11311 Test Plan: Jenkins: test regex: .*CDCSDKConsumptionConsistentChangesTest.* Reviewers: asrinivasan, stiwary, skumar Reviewed By: asrinivasan Subscribers: ybase, ycdcxcluster Tags: #jenkins-ready Differential Revision: https://phorge.dev.yugabyte.com/D35160
svarnau
pushed a commit
that referenced
this issue
May 29, 2024
…ble creation. Summary: Original commit: 6e6cb2a / D35160 There is a possible race condition when during dynamic table creation, the tablet creation is not completed i.e the retention barriers have not yet been set, but the background thread proceeds and adds the table to the stream. In such a case till the time the tablet creation succeeds, the retention barriers are not set on the tablets of this table, even though it has been added to the stream. This was observed as flakiness in the test `TestRetentionBarrierRaceWithUpdatePeersAndMetrics` . This diff introduces a fix for this race condition by calling GetTableLocations on the table before adding it to the stream. If the tablets are not yet initialized then this will fail and the addition of the table to the stream will take place in the next round of the background task. ######Backport Description No merge conflicts were encountered. Jira: DB-11311 Test Plan: Jenkins: test regex: .*CDCSDKConsumptionConsistentChangesTest.* Reviewers: asrinivasan, stiwary, skumar Reviewed By: asrinivasan Subscribers: ycdcxcluster, ybase Tags: #jenkins-ready Differential Revision: https://phorge.dev.yugabyte.com/D35263
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Jira Link: DB-11311
The text was updated successfully, but these errors were encountered: