Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

util: implement JoinMap #4640

Merged
merged 21 commits into from Apr 26, 2022
Merged

util: implement JoinMap #4640

merged 21 commits into from Apr 26, 2022

Conversation

hawkw
Copy link
Member

@hawkw hawkw commented Apr 25, 2022

Depends on #4630.

Motivation

In many cases, it is desirable to spawn a set of tasks associated with
keys, with the ability to cancel them by key. As an example use case for
this sort of thing, see Tower's ReadyCache type.

Now that PR #4530 adds a way of cancelling tasks in a
tokio::task::JoinSet, we can implement a map-like API based on the
same IdleNotifiedSet primitive.

Solution

This PR adds an implementation of a JoinMap type to
tokio_util::task, using the JoinSet type from tokio::task, the
AbortHandle type added in #4530, and the new task IDs added in #4630.

Individual tasks can be aborted by key using the JoinMap::abort
method, and a set of tasks whose key match a given predicate can be
aborted using JoinMap::abort_matching.

When tasks complete, JoinMap::join_one returns their associated key
alongside the output from the spawned future, or the key and the
JoinError if the task did not complete successfully.

Overall, I think the way this works is pretty straightforward; much of
this PR is just API boilerplate to implement the union of applicable
APIs from JoinSet and HashMap. Unlike previous iterations on the
JoinMap API (e.g. #4538), this version is implemented entirely in
tokio_util, using only public APIs from the tokio crate. Currently,
the required tokio APIs are unstable, but implementing JoinMap in
tokio-util means we will never have to make stability commitments for
the JoinMap API itself.

hawkw added 11 commits April 25, 2022 12:35
@hawkw
WIP task-id-based joinmap
ab60c69 
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
@hawkw
add tests
fed0b8c 
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
@hawkw
wip
a478672 
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
@hawkw
abort handles by ref so keys are still returned
9940c67 
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
@hawkw
fix ids getting hashed
d5014d5 
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
@hawkw
put back abort_matching
9397dc4 
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
@hawkw
fix detach_all
c8ed02c 
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
@hawkw
fixup doctests
944a2c3 
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
@hawkw
fixup docs
c400dbc 
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
@hawkw
nicer debug impl
04715f6 
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
@hawkw
misc cleanup
2aa2202 
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
@hawkw hawkw added A-tokio-util Area: The tokio-util crate M-task Module: tokio/task labels Apr 25, 2022
@hawkw hawkw requested a review from a team April 25, 2022 21:17
@hawkw hawkw self-assigned this Apr 25, 2022
@github-actions github-actions bot added the R-loom Run loom tests on this PR label Apr 25, 2022
hawkw added 10 commits April 25, 2022 14:31
@hawkw
rutsfmt
8b5bb19
@hawkw
do a coolendrun around msrv CI
41d92d7 
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
@hawkw
placate CI readme check
c532bf2 
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
@hawkw
blerrrghhhhhhhhhh hhhhhhhhh
7947a10
@hawkw
fix remaining docs reference whoops god damn it
43875a9 
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
@hawkw
ghghhhhhhhhhhhhghghhhghghghghghhhh
fda7c12 
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
@hawkw
docs improvements
51a0a6b 
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
@hawkw
remove unneeded static bounds
838d072 
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
@hawkw
remove duplicate task IDs from debug output
f0adddb 
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
@hawkw
rustfmt
3d05600 
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
@hawkw hawkw removed the R-loom Run loom tests on this PR label Apr 26, 2022
@github-actions github-actions bot added the R-loom Run loom tests on this PR label Apr 26, 2022
@hawkw hawkw enabled auto-merge (squash) April 26, 2022 17:07
@hawkw hawkw merged commit d456706 into master Apr 26, 2022
@hawkw hawkw deleted the eliza/joinmap-2 branch April 26, 2022 17:25
@hawkw hawkw mentioned this pull request Apr 26, 2022
Closed
2 tasks
hawkw added a commit that referenced this pull request Apr 27, 2022
@hawkw
tokio: prepare to release v1.18.0
6fcaea7 
# 1.18.0 (April 27, 2022)

This release adds a number of new APIs in `tokio::net`, `tokio::signal`, and
`tokio::sync`. In addition, it adds new unstable APIs to `tokio::task` (`Id`s
for uniquely identifying a task, and `AbortHandle` for remotely cancelling a
task), as well as a number of bugfixes.

### Fixed

- blocking: add missing `#[track_caller]` for `spawn_blocking` ([#4616])
- macros: fix `select` macro to process 64 branches ([#4519])
- net: fix `try_io` methods not calling Mio's `try_io` internally ([#4582])
- runtime: recover when OS fails to spawn a new thread ([#4485])

### Added

- macros: support setting a custom crate name for `#[tokio::main]` and
  `#[tokio::test]` ([#4613])
- net: add `UdpSocket::peer_addr` ([#4611])
- net: add `try_read_buf` method for named pipes ([#4626])
- signal: add `SignalKind` `Hash`/`Eq` impls and `c_int` conversion ([#4540])
- signal: add support for signals up to `SIGRTMAX` ([#4555])
- sync: add `watch::Sender::send_modify` method ([#4310])
- sync: add `broadcast::Receiver::len` method ([#4542])
- sync: add `watch::Receiver::same_channel` method ([#4581])
- sync: implement `Clone` for `RecvError` types ([#4560])

### Changed

- update `nix` to 0.24, limit features ([#4631])
- update `mio` to 0.8.1 ([#4582])
- macros: rename `tokio::select!`'s internal `util` module ([#4543])
- runtime: use `Vec::with_capacity` when building runtime ([#4553])

### Documented

- improve docs for `tokio_unstable` ([#4524])
- runtime: include more documentation for thread_pool/worker ([#4511])
- runtime: update `Handle::current`'s docs to mention `EnterGuard` ([#4567])
- time: clarify platform specific timer resolution ([#4474])
- signal: document that `Signal::recv` is cancel-safe ([#4634])
- sync: `UnboundedReceiver` close docs ([#4548])

### Unstable

The following changes only apply when building with `--cfg tokio_unstable`:

- task: add `task::Id` type ([#4630])
- task: add `AbortHandle` type for cancelling tasks in a `JoinSet` ([#4530],
  [#4640])
- task: fix missing `doc(cfg(...))` attributes for `JoinSet` ([#4531])
- task: fix broken link in `AbortHandle` RustDoc ([#4545])
- metrics: add initial IO driver metrics ([#4507])

[#4616]: #4616
[#4519]: #4519
[#4582]: #4582
[#4485]: #4485
[#4613]: #4613
[#4611]: #4611
[#4626]: #4626
[#4540]: #4540
[#4555]: #4555
[#4310]: #4310
[#4542]: #4542
[#4581]: #4581
[#4560]: #4560
[#4631]: #4631
[#4582]: #4582
[#4543]: #4543
[#4553]: #4553
[#4524]: #4524
[#4511]: #4511
[#4567]: #4567
[#4474]: #4474
[#4634]: #4634
[#4548]: #4548
[#4630]: #4630
[#4530]: #4530
[#4640]: #4640
[#4531]: #4531
[#4545]: #4545
[#4507]: #4507
hawkw added a commit that referenced this pull request Apr 27, 2022
@hawkw
tokio: prepare to release v1.18.0
b8ffe77 
# 1.18.0 (April 27, 2022)

This release adds a number of new APIs in `tokio::net`, `tokio::signal`, and
`tokio::sync`. In addition, it adds new unstable APIs to `tokio::task` (`Id`s
for uniquely identifying a task, and `AbortHandle` for remotely cancelling a
task), as well as a number of bugfixes.

### Fixed

- blocking: add missing `#[track_caller]` for `spawn_blocking` ([#4616])
- macros: fix `select` macro to process 64 branches ([#4519])
- net: fix `try_io` methods not calling Mio's `try_io` internally ([#4582])
- runtime: recover when OS fails to spawn a new thread ([#4485])

### Added

- macros: support setting a custom crate name for `#[tokio::main]` and
  `#[tokio::test]` ([#4613])
- net: add `UdpSocket::peer_addr` ([#4611])
- net: add `try_read_buf` method for named pipes ([#4626])
- signal: add `SignalKind` `Hash`/`Eq` impls and `c_int` conversion ([#4540])
- signal: add support for signals up to `SIGRTMAX` ([#4555])
- sync: add `watch::Sender::send_modify` method ([#4310])
- sync: add `broadcast::Receiver::len` method ([#4542])
- sync: add `watch::Receiver::same_channel` method ([#4581])
- sync: implement `Clone` for `RecvError` types ([#4560])

### Changed

- update `nix` to 0.24, limit features ([#4631])
- update `mio` to 0.8.1 ([#4582])
- macros: rename `tokio::select!`'s internal `util` module ([#4543])
- runtime: use `Vec::with_capacity` when building runtime ([#4553])

### Documented

- improve docs for `tokio_unstable` ([#4524])
- runtime: include more documentation for thread_pool/worker ([#4511])
- runtime: update `Handle::current`'s docs to mention `EnterGuard` ([#4567])
- time: clarify platform specific timer resolution ([#4474])
- signal: document that `Signal::recv` is cancel-safe ([#4634])
- sync: `UnboundedReceiver` close docs ([#4548])

### Unstable

The following changes only apply when building with `--cfg tokio_unstable`:

- task: add `task::Id` type ([#4630])
- task: add `AbortHandle` type for cancelling tasks in a `JoinSet` ([#4530],
  [#4640])
- task: fix missing `doc(cfg(...))` attributes for `JoinSet` ([#4531])
- task: fix broken link in `AbortHandle` RustDoc ([#4545])
- metrics: add initial IO driver metrics ([#4507])

[#4616]: #4616
[#4519]: #4519
[#4582]: #4582
[#4485]: #4485
[#4613]: #4613
[#4611]: #4611
[#4626]: #4626
[#4540]: #4540
[#4555]: #4555
[#4310]: #4310
[#4542]: #4542
[#4581]: #4581
[#4560]: #4560
[#4631]: #4631
[#4582]: #4582
[#4543]: #4543
[#4553]: #4553
[#4524]: #4524
[#4511]: #4511
[#4567]: #4567
[#4474]: #4474
[#4634]: #4634
[#4548]: #4548
[#4630]: #4630
[#4530]: #4530
[#4640]: #4640
[#4531]: #4531
[#4545]: #4545
[#4507]: #4507
@hawkw hawkw mentioned this pull request Apr 27, 2022
Merged
hawkw added a commit that referenced this pull request Apr 27, 2022
@hawkw
tokio: prepare to release v1.18.0 (#4641)
4818343 
# 1.18.0 (April 27, 2022)

This release adds a number of new APIs in `tokio::net`, `tokio::signal`, and
`tokio::sync`. In addition, it adds new unstable APIs to `tokio::task` (`Id`s
for uniquely identifying a task, and `AbortHandle` for remotely cancelling a
task), as well as a number of bugfixes.

### Fixed

- blocking: add missing `#[track_caller]` for `spawn_blocking` ([#4616])
- macros: fix `select` macro to process 64 branches ([#4519])
- net: fix `try_io` methods not calling Mio's `try_io` internally ([#4582])
- runtime: recover when OS fails to spawn a new thread ([#4485])

### Added

- macros: support setting a custom crate name for `#[tokio::main]` and
  `#[tokio::test]` ([#4613])
- net: add `UdpSocket::peer_addr` ([#4611])
- net: add `try_read_buf` method for named pipes ([#4626])
- signal: add `SignalKind` `Hash`/`Eq` impls and `c_int` conversion ([#4540])
- signal: add support for signals up to `SIGRTMAX` ([#4555])
- sync: add `watch::Sender::send_modify` method ([#4310])
- sync: add `broadcast::Receiver::len` method ([#4542])
- sync: add `watch::Receiver::same_channel` method ([#4581])
- sync: implement `Clone` for `RecvError` types ([#4560])

### Changed

- update `nix` to 0.24, limit features ([#4631])
- update `mio` to 0.8.1 ([#4582])
- macros: rename `tokio::select!`'s internal `util` module ([#4543])
- runtime: use `Vec::with_capacity` when building runtime ([#4553])

### Documented

- improve docs for `tokio_unstable` ([#4524])
- runtime: include more documentation for thread_pool/worker ([#4511])
- runtime: update `Handle::current`'s docs to mention `EnterGuard` ([#4567])
- time: clarify platform specific timer resolution ([#4474])
- signal: document that `Signal::recv` is cancel-safe ([#4634])
- sync: `UnboundedReceiver` close docs ([#4548])

### Unstable

The following changes only apply when building with `--cfg tokio_unstable`:

- task: add `task::Id` type ([#4630])
- task: add `AbortHandle` type for cancelling tasks in a `JoinSet` ([#4530],
  [#4640])
- task: fix missing `doc(cfg(...))` attributes for `JoinSet` ([#4531])
- task: fix broken link in `AbortHandle` RustDoc ([#4545])
- metrics: add initial IO driver metrics ([#4507])

[#4616]: #4616
[#4519]: #4519
[#4582]: #4582
[#4485]: #4485
[#4613]: #4613
[#4611]: #4611
[#4626]: #4626
[#4540]: #4540
[#4555]: #4555
[#4310]: #4310
[#4542]: #4542
[#4581]: #4581
[#4560]: #4560
[#4631]: #4631
[#4582]: #4582
[#4543]: #4543
[#4553]: #4553
[#4524]: #4524
[#4511]: #4511
[#4567]: #4567
[#4474]: #4474
[#4634]: #4634
[#4548]: #4548
[#4630]: #4630
[#4530]: #4530
[#4640]: #4640
[#4531]: #4531
[#4545]: #4545
[#4507]: #4507

Signed-off-by: Eliza Weisman <eliza@buoyant.io>
crapStone pushed a commit to Calciumdibromid/CaBr2 that referenced this pull request May 1, 2022
Update Rust crate tokio to 1.18.0 (#1327)
0201aa5 
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [tokio](https://tokio.rs) ([source](https://github.com/tokio-rs/tokio)) | dependencies | minor | `1.17.0` -> `1.18.0` |
| [tokio](https://tokio.rs) ([source](https://github.com/tokio-rs/tokio)) | dev-dependencies | minor | `1.17.0` -> `1.18.0` |

---

### Release Notes

<details>
<summary>tokio-rs/tokio</summary>

### [`v1.18.0`](https://github.com/tokio-rs/tokio/releases/tokio-1.18.0)

[Compare Source](tokio-rs/tokio@tokio-1.17.0...tokio-1.18.0)

##### 1.18.0 (April 27, 2022)

This release adds a number of new APIs in `tokio::net`, `tokio::signal`, and
`tokio::sync`. In addition, it adds new unstable APIs to `tokio::task` (`Id`s
for uniquely identifying a task, and `AbortHandle` for remotely cancelling a
task), as well as a number of bugfixes.

##### Fixed

-   blocking: add missing `#[track_caller]` for `spawn_blocking` ([#&#8203;4616](tokio-rs/tokio#4616))
-   macros: fix `select` macro to process 64 branches ([#&#8203;4519](tokio-rs/tokio#4519))
-   net: fix `try_io` methods not calling Mio's `try_io` internally ([#&#8203;4582](tokio-rs/tokio#4582))
-   runtime: recover when OS fails to spawn a new thread ([#&#8203;4485](tokio-rs/tokio#4485))

##### Added

-   macros: support setting a custom crate name for `#[tokio::main]` and
    `#[tokio::test]` ([#&#8203;4613](tokio-rs/tokio#4613))
-   net: add `UdpSocket::peer_addr` ([#&#8203;4611](tokio-rs/tokio#4611))
-   net: add `try_read_buf` method for named pipes ([#&#8203;4626](tokio-rs/tokio#4626))
-   signal: add `SignalKind` `Hash`/`Eq` impls and `c_int` conversion ([#&#8203;4540](tokio-rs/tokio#4540))
-   signal: add support for signals up to `SIGRTMAX` ([#&#8203;4555](tokio-rs/tokio#4555))
-   sync: add `watch::Sender::send_modify` method ([#&#8203;4310](tokio-rs/tokio#4310))
-   sync: add `broadcast::Receiver::len` method ([#&#8203;4542](tokio-rs/tokio#4542))
-   sync: add `watch::Receiver::same_channel` method ([#&#8203;4581](tokio-rs/tokio#4581))
-   sync: implement `Clone` for `RecvError` types ([#&#8203;4560](tokio-rs/tokio#4560))

##### Changed

-   update `mio` to 0.8.1 ([#&#8203;4582](tokio-rs/tokio#4582))
-   macros: rename `tokio::select!`'s internal `util` module ([#&#8203;4543](tokio-rs/tokio#4543))
-   runtime: use `Vec::with_capacity` when building runtime ([#&#8203;4553](tokio-rs/tokio#4553))

##### Documented

-   improve docs for `tokio_unstable` ([#&#8203;4524](tokio-rs/tokio#4524))
-   runtime: include more documentation for thread_pool/worker ([#&#8203;4511](tokio-rs/tokio#4511))
-   runtime: update `Handle::current`'s docs to mention `EnterGuard` ([#&#8203;4567](tokio-rs/tokio#4567))
-   time: clarify platform specific timer resolution ([#&#8203;4474](tokio-rs/tokio#4474))
-   signal: document that `Signal::recv` is cancel-safe ([#&#8203;4634](tokio-rs/tokio#4634))
-   sync: `UnboundedReceiver` close docs ([#&#8203;4548](tokio-rs/tokio#4548))

##### Unstable

The following changes only apply when building with `--cfg tokio_unstable`:

-   task: add `task::Id` type ([#&#8203;4630](tokio-rs/tokio#4630))
-   task: add `AbortHandle` type for cancelling tasks in a `JoinSet` ([#&#8203;4530](tokio-rs/tokio#4530)],
    \[[#&#8203;4640](tokio-rs/tokio#4640))
-   task: fix missing `doc(cfg(...))` attributes for `JoinSet` ([#&#8203;4531](tokio-rs/tokio#4531))
-   task: fix broken link in `AbortHandle` RustDoc ([#&#8203;4545](tokio-rs/tokio#4545))
-   metrics: add initial IO driver metrics ([#&#8203;4507](tokio-rs/tokio#4507))

</details>

---

### Configuration

📅 **Schedule**: At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about these updates again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, click this checkbox.

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

Co-authored-by: cabr2-bot <cabr2.help@gmail.com>
Reviewed-on: https://codeberg.org/Calciumdibromid/CaBr2/pulls/1327
Reviewed-by: crapStone <crapstone@noreply.codeberg.org>
Co-authored-by: Calciumdibromid Bot <cabr2_bot@noreply.codeberg.org>
Co-committed-by: Calciumdibromid Bot <cabr2_bot@noreply.codeberg.org>
@Darksonn Darksonn mentioned this pull request May 14, 2022
Merged
@Darksonn Darksonn mentioned this pull request Jun 4, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Noah-Kennedy Noah-Kennedy Noah-Kennedy approved these changes

Assignees

@hawkw hawkw

Labels
A-tokio-util Area: The tokio-util crate M-task Module: tokio/task R-loom Run loom tests on this PR
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@hawkw @Noah-Kennedy