New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
sync: Add is_closed
method to mpsc senders
#2726
Conversation
Thanks for exploring this issue and giving it a try; I'm still very interested in seeing an approach to this feature. |
@zeroed This was likely not a |
@MikailBag looks like the loom ci test is failing, could you fix that up? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like a good and simple start, I want to make sure loom is happy though. Also would be good to add more tests.
tokio/src/sync/mpsc/chan.rs
Outdated
@@ -432,6 +438,27 @@ impl Semaphore for (crate::sync::semaphore_ll::Semaphore, usize) { | |||
self.0.add_permits(1) | |||
} | |||
|
|||
fn is_closed(&self) -> bool { | |||
// TODO find more efficient way |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would be good if we can open an issue for this TODO.
IIUC these are compile errors, because in loom mode Arc != std::sync::Arc. I'll be able to fix these problems (and doc-comment) around 17th of August. |
@MikailBag 👍 no rush, feel free to ping me here or discord when you get a new revision up. |
Please, don't be: It has been an interesting learning experience. Mikail anticipated me with the "permits" approach and it's more than fine. Thanks! |
tokio/src/sync/mpsc/chan.rs
Outdated
@@ -488,6 +515,10 @@ impl Semaphore for AtomicUsize { | |||
} | |||
} | |||
|
|||
fn is_closed(&self) -> bool { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need this implementation?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you mean unbounded channels should not have fn is_closed
?
tokio/src/sync/mpsc/chan.rs
Outdated
// TODO find more efficient way | ||
struct NoopWaker; | ||
impl crate::util::Wake for NoopWaker { | ||
fn wake(self: std::sync::Arc<Self>) {} | ||
fn wake_by_ref(_arc_self: &std::sync::Arc<Self>) {} | ||
} | ||
let waker = std::sync::Arc::new(NoopWaker); | ||
let waker = crate::util::waker_ref(&waker); | ||
let mut noop_cx = std::task::Context::from_waker(&*waker); | ||
let mut permit = Permit::new(); | ||
match permit.poll_acquire(&mut noop_cx, 1, &self.0) { | ||
Poll::Ready(Err(_)) => true, | ||
Poll::Ready(Ok(())) => { | ||
permit.release(1, &self.0); | ||
false | ||
} | ||
Poll::Pending => false, | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm pretty sure that the semaphore's state field has a bit that we can read to check if it's closed:
tokio/tokio/src/sync/semaphore_ll.rs
Lines 116 to 117 in be7462e
/// Semaphore has been closed, no more permits will be issued. | |
const CLOSED: usize = 0b10; |
For example, we use this here:
tokio/tokio/src/sync/semaphore_ll.rs
Lines 219 to 243 in be7462e
// Load the current state | |
let mut curr = SemState(self.state.load(Acquire)); | |
// Saves a ref to the waiter node | |
let mut maybe_waiter: Option<NonNull<Waiter>> = None; | |
/// Used in branches where we attempt to push the waiter into the wait | |
/// queue but fail due to permits becoming available or the wait queue | |
/// transitioning to "closed". In this case, the waiter must be | |
/// transitioned back to the "idle" state. | |
macro_rules! revert_to_idle { | |
() => { | |
if let Some(waiter) = maybe_waiter { | |
unsafe { waiter.as_ref() }.revert_to_idle(); | |
} | |
}; | |
} | |
loop { | |
let mut next = curr; | |
if curr.is_closed() { | |
revert_to_idle!(); | |
return Ready(Err(AcquireError::closed())); | |
} |
I think it should be possible to have an is_closed
method on semaphore_ll
that looks something like this:
impl Semaphore {
// ...
pub(crate) fn is_closed(&self) -> bool {
SemState(self.state.load(Acquire)).is_closed()
}
// ...
}
There's additional logic that occurs when closing a semaphore, to assign permits to any pending waiters prior to marking the semaphore as closed. However, if this process is currently taking place, I think it's correct to not return true
from is_closed
yet. So AFAICT, this implementation ought to be valid.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added is_closed
method to semaphore_ll and now it is used instead of hacks with null waker.
@hawkw can you take another look if you get the chance? |
623ab17
to
13e444b
Compare
I've rebased onto master. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have some documentation suggestions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is my last comment.
Co-authored-by: Alice Ryhl <alice@ryhl.io>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!
is_closed
method to mpsc senders
Motivation
Closes #2469
Solution
I should note that I looked at mpsc internals first time, so I can be wrong / selected wrong implementation path.
I discovered that both Sender kinds (bounded and Unbounded) implement on top of
tokio::sync::mpsc::Chan
, which in turn usesSemaphore
trait. I added new methodis_closed
to this trait. I was able to implement it properly for unbounded implementation.However, I didn't find beautiful way to check if
semaphore_ll
is closed, so I implemented hack (we try to acquire new permit and immediately release it back). While it is inefficient and probably ugly, from the user side it should work OK (for example, it does not mess Sender internal state and it does not take mutable reference to sender).