-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BugFix] cleanup in-memory partition state if check fail when creating multiple-partitions #45772
[BugFix] cleanup in-memory partition state if check fail when creating multiple-partitions #45772
Conversation
Signed-off-by: Murphy <mofei@starrocks.com>
idToRange.remove(partitionId); | ||
} | ||
} | ||
|
||
public void moveRangeFromTempToFormal(long tempPartitionId) { | ||
Range<PartitionKey> range = idToTempRange.remove(tempPartitionId); | ||
if (range != null) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The most risky bug in this code is:
Inconsistent handling of exceptions for partially processed partition entries leading to potential data inconsistency or resource leak.
You can modify the code like this:
@@ -253,25 +253,38 @@ public void createAutomaticShadowPartition(long partitionId, String replicateNum
public void handleNewRangePartitionDescs(List<Pair<Partition, PartitionDesc>> partitionList,
Set<String> existPartitionNameSet,
boolean isTemp) throws DdlException {
+ List<Long> successfullyProcessedPartitions = new ArrayList<>();
try {
for (Pair<Partition, PartitionDesc> entry : partitionList) {
Partition partition = entry.first;
if (!existPartitionNameSet.contains(partition.getName())) {
long partitionId = partition.getId();
SingleRangePartitionDesc desc = (SingleRangePartitionDesc) entry.second;
Preconditions.checkArgument(desc.isAnalyzed());
Range<PartitionKey> range;
try {
range = checkAndCreateRange((SingleRangePartitionDesc) entry.second, isTemp);
setRangeInternal(partitionId, isTemp, range);
+ successfullyProcessedPartitions.add(partitionId); // Add ID to list after successful processing
} catch (IllegalArgumentException e) {
// Range.closedOpen may throw this if (lower > upper)
throw new DdlException("Invalid key range: " + e.getMessage());
}
idToDataProperty.put(partitionId, desc.getPartitionDataProperty());
idToReplicationNum.put(partitionId, desc.getReplicationNum());
idToInMemory.put(partitionId, desc.isInMemory());
idToStorageCacheInfo.put(partitionId, desc.getDataCacheInfo());
}
}
} catch (Exception e) {
// cleanup only for processed partitions
- partitionList.forEach(entry -> {
+ successfullyProcessedPartitions.forEach(partitionId -> {
removeRangeInternal(partitionId, isTemp);
idToDataProperty.remove(partitionId);
idToReplicationNum.remove(partitionId);
idToInMemory.remove(partitionId);
idToStorageCacheInfo.remove(partitionId);
});
throw e;
}
}
This modification ensures that the cleanup process applies only to those partitions that were successfully processed before an exception occurred, thus preventing removal or alteration of entries that weren't affected by the current operation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it doesn't matter, bc the remove
is safe even for non-existent partitions
Quality Gate passedIssues Measures |
[FE Incremental Coverage Report]❌ fail : 13 / 29 (44.83%) file detail
|
[BE Incremental Coverage Report]✅ pass : 0 / 0 (0%) |
@Mergifyio backport branch-3.3 |
@Mergifyio backport branch-3.2 |
✅ Backports have been created
|
✅ Backports have been created
|
…g multiple-partitions (StarRocks#45772)
Why I'm doing:
What I'm doing:
Fixes SR-26924
What type of PR is this:
Does this PR entail a change in behavior?
If yes, please specify the type of change:
Checklist:
Bugfix cherry-pick branch check: