Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mpmc xadd q poll can use consumerIndex parity bit to mark a rotation #265

Open
franz1981 opened this issue Oct 5, 2019 · 1 comment
Open

Comments

@franz1981
Copy link
Collaborator

franz1981 commented Oct 5, 2019

mpmc xadd's poll uses the consumer index to move forward the consumers, but consumers that need to recognize that a rotation is in progress have to check consumerBuffer::index value.

The rotation in progress case can be handled more transpanrently by stealing the parity bit of consumerIndex, saving to read the consumerBuffer (and its index) field if not necessary.

@franz1981 franz1981 changed the title Mpmc xadd q poll can just use consumerIndex Mpmc xadd q poll can use consumerIndex parity bit to mark a rotation Oct 28, 2019
@franz1981
Copy link
Collaborator Author

franz1981 commented Jan 3, 2020

The changes of #281 have shown the need to simplify the mpmc xadd poll.

To summarize the findings for the future:
before the changes of #281, with chunkSize = 1 and 1 recycled chunk, a poll can trigger a mid-element rotation if the consumer read consumerChunk and consumerIndex right after the rotating consumer has won a casConsumerIndex, but hasn't yet soConsumerChunk the next one.
The mid-element consumer can see the old consumerChunk, now recycled for the next offer, matching ciChunkIndex and then making it able to perform a consume on it.
eg

  1. C1: ci = 2 cc = 1
  2. C1: start rotation: ci = 2 -> 3 and pool cc (cc->next!=null && cc->next = 2)
  3. C2: read ci ( == 3), cc (==1)
  4. P1: read a chunk (== 1) from the pool, soIndex(3) and writing the element & sequence on it
  5. C2: check cc.index == ci ( == 3), sequence and consume it ie ci = 3->4
  6. C1: end rotation: set cc = 1 -> 2
  7. C3: cc = 2 and ci = 4: it BLOCKS

The issue happen because a rotation can change ci to point to a new chunk (!= next) and this chunk could be a pooled one right after winning the casConsumerIndex on rotation: with chunkSize > 1 (eg 2) the rotation can change ci, but it's forced to stay on the next chunk.
The only case that can make the mid-element to be consumed is with ciChunkIndex == ccChunkIndex and ci already rotated (ie pointing to the second element of next, yet not cc), but given that we need next to continue and cc could be either the current one (recycled too) or next, it will wait it to become next to proceed, avoid an early consume to happen.

The fix on the PR is valid for the chunkSize = 1 case, because it forces the distance of 1 between cc and ci when we handle the first element in the next chunk.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants