Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add task::LocalSet API for running !Send futures #1733

Merged
merged 76 commits into from Nov 27, 2019
Merged
Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
7eee2ec
first pass on local task set
hawkw Nov 4, 2019
366ecbd
it works!
hawkw Nov 4, 2019
82857dd
change API to work with multiple block ons
hawkw Nov 5, 2019
edd8ca4
shutdown on drop
hawkw Nov 5, 2019
2e6cf55
use marker types to indicate whether a task is Send
hawkw Nov 5, 2019
c1e4e4a
add docs
hawkw Nov 5, 2019
968eea5
re-export local mod from root, feature flag
hawkw Nov 5, 2019
983b6b7
placate clippy
hawkw Nov 5, 2019
fb99e68
fix doctest
hawkw Nov 5, 2019
d6ff66e
*actually* fix doctest
hawkw Nov 5, 2019
bc93168
fix wrong assertions
hawkw Nov 5, 2019
17c7f7e
fix missing pin-project without default features
hawkw Nov 5, 2019
78d41b8
fix unused code when `local` is disabled
hawkw Nov 5, 2019
057f415
add send markers to debug output
hawkw Nov 5, 2019
f00f099
fdsgkhsdfhjkgljkfdg;jklagdsf
hawkw Nov 5, 2019
157e2e2
fix more feature-flaggy sadness
hawkw Nov 5, 2019
2a88b5f
rename marker types
hawkw Nov 6, 2019
eef8727
make markers void structs
hawkw Nov 6, 2019
1c0cea1
remove duplicate file
hawkw Nov 6, 2019
b6243c6
fix wrong assertions
hawkw Nov 6, 2019
44fb6c8
add test that timer exists
hawkw Nov 6, 2019
ff05221
document and test interaction with blocking
hawkw Nov 6, 2019
45dc55a
fix wrong doctest
hawkw Nov 6, 2019
b92ff94
fix missing feature flag
hawkw Nov 6, 2019
e09fb8e
agh, doctest
hawkw Nov 6, 2019
fec3b91
don't run local tests when running loom tests
hawkw Nov 6, 2019
478e73f
rename TaskSet -> TaskGroup
hawkw Nov 7, 2019
da87e96
rename `spawn_local` to `local::spawn`
hawkw Nov 7, 2019
267f3f3
improve API docs
hawkw Nov 7, 2019
c06cba8
Merge branch 'master' into eliza/local-spawn
hawkw Nov 7, 2019
99e05b2
track `timer` being renamed to `time` (#1745)
hawkw Nov 7, 2019
44ad494
Merge branch 'master' into eliza/local-spawn
hawkw Nov 13, 2019
1b9b25c
move local out of runtime and into task
hawkw Nov 13, 2019
b8488c7
rename to `spawn_local`
hawkw Nov 13, 2019
07317af
rename `TaskGroup` -> `LocalSet`
hawkw Nov 13, 2019
6f7607e
remove root reexport
hawkw Nov 13, 2019
2246de2
remove from runtime
hawkw Nov 13, 2019
c4f6034
doctest fixup
hawkw Nov 13, 2019
f4d8b5a
simplify Send marker
hawkw Nov 13, 2019
8f21118
fix tests using the wrong blocking
hawkw Nov 13, 2019
cbc1a42
naming + internal docs improvement
hawkw Nov 13, 2019
72472ca
set thread local in block_on
hawkw Nov 13, 2019
927ae9b
fix local future spinning instead of parking
hawkw Nov 13, 2019
34e3428
local scheduler doesn't need to be Send + Sync
hawkw Nov 13, 2019
0fbe6ab
add tests for waking a local set from IO
hawkw Nov 13, 2019
1ad14c7
use the rt's waker in block_on
hawkw Nov 13, 2019
ab19812
rm phantomdata
hawkw Nov 13, 2019
1007cad
fix local queue being pushed to from remote
hawkw Nov 13, 2019
5265b41
remove "local" feature
hawkw Nov 13, 2019
c42992b
add additional assertions
hawkw Nov 13, 2019
0514a56
use AtomicWaker instead of mutex
hawkw Nov 13, 2019
c7b0cb7
avoid double panics
hawkw Nov 13, 2019
36efbb4
fix warnings
hawkw Nov 13, 2019
25d9bf4
rustfmt
hawkw Nov 13, 2019
26994c9
placate clippy
hawkw Nov 13, 2019
20ccb39
maybe they were supposed to be underscores???
hawkw Nov 13, 2019
f28ded7
undo unrelated naming change
hawkw Nov 13, 2019
ef29af8
review feedback
hawkw Nov 14, 2019
7a9c84e
put clippy attribute in the right place
hawkw Nov 14, 2019
f1211e3
fix unused code based on feature flags
hawkw Nov 14, 2019
6ff8c4f
review feedback: rename `ScheduleSend` -> `ScheduleSendOnly`
hawkw Nov 14, 2019
b21cca7
Merge master + update some things
hawkw Nov 18, 2019
17d520a
Avoid starving the remote queue
hawkw Nov 18, 2019
cfa2808
allow tasks to be cancelled remotely
hawkw Nov 18, 2019
6f43e7e
update tests to use schedulers that actually do stuff
hawkw Nov 18, 2019
f1c82c5
fix accidental hang in test
hawkw Nov 18, 2019
b919c95
add test for sending a local future's JoinHandle
hawkw Nov 18, 2019
fe93b45
fix yielding
hawkw Nov 21, 2019
66f4a45
Merge branch 'master' into eliza/local-spawn
hawkw Nov 26, 2019
40374f7
fix bad merge
hawkw Nov 26, 2019
7260998
fix bad merge 2: the fixening
hawkw Nov 26, 2019
e40faca
more cleanup
hawkw Nov 26, 2019
9c805b6
rm empty file committed accidentally
hawkw Nov 26, 2019
04b0189
move to rt-util feature
hawkw Nov 26, 2019
0528537
panic if nested
hawkw Nov 26, 2019
3418bfb
fix AtomicWaker being enabled when not used
hawkw Nov 27, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions tokio/Cargo.toml
Expand Up @@ -28,6 +28,7 @@ default = [
"blocking",
"fs",
"io",
"local",
hawkw marked this conversation as resolved.
Show resolved Hide resolved
"net-full",
"process",
"rt-full",
Expand All @@ -42,6 +43,7 @@ fs = ["blocking", "io-traits"]
io-traits = ["bytes", "iovec"]
io-util = ["io-traits", "pin-project", "memchr"]
io = ["io-traits", "io-util"]
local = ["rt-current-thread", "pin-project"]
macros = ["tokio-macros"]
net-full = ["tcp", "udp", "uds"]
net-driver = ["io-traits", "mio", "blocking", "lazy_static"]
Expand Down
68 changes: 67 additions & 1 deletion tokio/src/lib.rs
Expand Up @@ -120,8 +120,74 @@ pub mod time;
#[cfg(feature = "rt-full")]
mod util;

if_runtime! {
#[cfg(feature = "local")]
pub mod local {
//! Runs `!Send` futures on the current thread.
Copy link
Sponsor Contributor

@jonhoo jonhoo Nov 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should mention in-place blocking somewhere here.

//!
//! In some cases, it is necessary to run one or more futures that do not
//! implement [`Send`] and thus are unsafe to send between threads. In these
//! cases, a [local task group] may be used to schedule one or more `!Send`
//! futures to run together on the same thread.
//!
//! For example, the following code will not compile:
//!
//! ```rust,compile_fail
//! # use tokio::runtime::Runtime;
//! use std::rc::Rc;
//!
//! // `Rc` does not implement `Send`, and thus may not be sent between
//! // threads safely.
//! let unsend_data = Rc::new("my unsend data...");
//!
//! let mut rt = Runtime::new().unwrap();
//!
//! rt.block_on(async move {
//! let unsend_data = unsend_data.clone();
//! // Because the `async` block here moves `unsend_data`, the future is `!Send`.
//! // Since `tokio::spawn` requires the spawned future to implement `Send`, this
//! // will not compile.
//! tokio::spawn(async move {
//! println!("{}", unsend_data);
//! // ...
//! }).await.unwrap();
//! });
//! ```
//! In order to spawn `!Send` futures, we can use a local task group to
//! schedule them on the thread calling [`Runtime::block_on`]. When running
//! inside of the local task group, we can use [`local::spawn`], which can
//! spawn `!Send` futures. For example:
//!
//! ```rust
//! # use tokio::runtime::Runtime;
//! use std::rc::Rc;
//! use tokio::local;
//!
//! let unsend_data = Rc::new("my unsend data...");
//!
//! let mut rt = Runtime::new().unwrap();
//! // Construct a local task group that can run `!Send` futures.
//! let local_group = local::TaskGroup::new();
//!
//! // Run the local task group.
//! local_group.block_on(&mut rt, async move {
//! let unsend_data = unsend_data.clone();
//! // `local::spawn` ensures that the future is spawned on the local
//! // task group.
//! local::spawn(async move {
//! println!("{}", unsend_data);
//! // ...
//! }).await.unwrap();
//! });
//! ```
//!
//! [`Send`]: https://doc.rust-lang.org/std/marker/trait.Send.html
//! [local task group]: struct.TaskGroup.html
//! [`Runtime::block_on`]: ../struct.Runtime.html#method.block_on
//! [`local::spawn`]: fn.spawn.html
pub use crate::runtime::local::*;
}

if_runtime! {
#[doc(inline)]
pub use crate::runtime::spawn;

Expand Down
4 changes: 2 additions & 2 deletions tokio/src/runtime/current_thread/mod.rs
Expand Up @@ -35,7 +35,7 @@ pub(super) struct Scheduler {
/// # Safety
///
/// Must only be accessed from the primary thread
owned_tasks: UnsafeCell<task::OwnedList<Self>>,
owned_tasks: UnsafeCell<task::OwnedList<Self, task::Sendable>>,

/// Local run queue.
///
Expand Down Expand Up @@ -254,7 +254,7 @@ impl Scheduler {
}
}

impl Schedule for Scheduler {
impl Schedule<task::Sendable> for Scheduler {
fn bind(&self, task: &Task<Self>) {
unsafe {
(*self.owned_tasks.get()).insert(task);
Expand Down