Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix] Fix concurrent containers concurrency issue #22604

Closed
wants to merge 2 commits into from
Closed

[fix] Fix concurrent containers concurrency issue #22604

wants to merge 2 commits into from

Conversation

dao-jun
Copy link
Member

@dao-jun dao-jun commented Apr 27, 2024

Fixes #22603

Motivation

Fix concurrent containers concurrency issue

Modifications

Verifying this change

  • Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end deployment with large payloads (10MB)
  • Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository:

@dao-jun dao-jun added this to the 3.3.0 milestone Apr 27, 2024
@dao-jun dao-jun self-assigned this Apr 27, 2024
@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Apr 27, 2024
@codecov-commenter
Copy link

Codecov Report

Attention: Patch coverage is 93.33333% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 73.96%. Comparing base (bbc6224) to head (3a2c24a).
Report is 201 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #22604      +/-   ##
============================================
+ Coverage     73.57%   73.96%   +0.38%     
- Complexity    32624    33148     +524     
============================================
  Files          1877     1885       +8     
  Lines        139502   140543    +1041     
  Branches      15299    15431     +132     
============================================
+ Hits         102638   103946    +1308     
+ Misses        28908    28563     -345     
- Partials       7956     8034      +78
Flag Coverage Δ
inttests 27.27% <53.33%> (+2.68%) ⬆️
systests 24.61% <56.66%> (+0.28%) ⬆️
unittests 73.24% <93.33%> (+0.39%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
...common/util/collections/ConcurrentLongHashMap.java 90.40% <100.00%> (+1.51%) ⬆️
...til/collections/ConcurrentLongLongPairHashMap.java 89.23% <100.00%> (+0.71%) ⬆️
...common/util/collections/ConcurrentOpenHashMap.java 94.59% <100.00%> (-0.30%) ⬇️
...common/util/collections/ConcurrentOpenHashSet.java 85.60% <100.00%> (-0.77%) ⬇️
...common/util/collections/ConcurrentLongPairSet.java 87.32% <66.66%> (+1.00%) ⬆️

... and 271 files with indirect coverage changes

lhotari
lhotari requested changes Apr 27, 2024
View reviewed changes
Copy link
Member

@lhotari lhotari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for reporting the issue and starting the work to fix this issue.

Catching the exception would be the preferred solution since there's no point of using a StampedLock unless optimistic locking is used.

I think that we had this same bug discussed years ago, but for some reason we didn't fix this.

@dao-jun
Copy link
Member Author

dao-jun commented Apr 27, 2024

Thanks for reporting the issue and starting the work to fix this issue.

Catching the exception would be the preferred solution since there's no point of using a StampedLock unless optimistic locking is used.

I think that we had this same bug discussed years ago, but for some reason we didn't fix this.

@lhotari Is there any discuss thread links? I don't see any discuss linked to the origin PRs.

@lhotari
Copy link
Member

lhotari commented Apr 27, 2024

Thanks for reporting the issue and starting the work to fix this issue.
Catching the exception would be the preferred solution since there's no point of using a StampedLock unless optimistic locking is used.
I think that we had this same bug discussed years ago, but for some reason we didn't fix this.

@lhotari Is there any discuss thread links? I don't see any discuss linked to the origin PRs.

Perhaps in #18390

@lhotari
Copy link
Member

lhotari commented Apr 27, 2024 •
edited

Found it: #18390 (comment) . @thetumbled had originally made a PR to catch IndexOutOfBoundsException, which I think is the correct solution.

@dao-jun
Copy link
Member Author

dao-jun commented Apr 27, 2024 •
edited

Oh, I see, it looks like introduced local variables to avoid ArrayIndexOutOfBoundsExeception, I did't read it carefully, it makes sense, we can sync #18390 to bookkeeper.

@dao-jun dao-jun closed this Apr 27, 2024
@dao-jun dao-jun deleted the fix/fix_concurrency_issue branch April 27, 2024 08:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lhotari lhotari lhotari requested changes

@codelipenghui codelipenghui Awaiting requested review from codelipenghui

@BewareMyPower BewareMyPower Awaiting requested review from BewareMyPower

@poorbarcode poorbarcode Awaiting requested review from poorbarcode

@mattisonchao mattisonchao Awaiting requested review from mattisonchao

Assignees

@dao-jun dao-jun

Labels
area/broker area/client doc-not-needed Your PR changes do not impact docs ready-to-test release/3.0.5 release/3.2.3 type/bug The PR fixed a bug or issue reported a bug
Projects
None yet
Milestone
3.3.0
Development

Successfully merging this pull request may close these issues.

[Bug] Pulsar concurrent containers has concurrency issues
3 participants
@dao-jun @codecov-commenter @lhotari