Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

macros: don't take ownership of futures in macros #5087

Merged
merged 9 commits into from Oct 14, 2022

Conversation

Darksonn
Copy link
Contributor

This PR mitigates the issue discussed in Surprising soundness trouble around PollFn. The issue is mitigated in two ways:

  • Never give a PollFn closure ownership of a future. Instead, we give the closure a reference to the future. This way, a reference to the closure is not a reference to a future, so even if a reference to the closure is noalias, it doesn't leak to the future.
  • Make the PollFn type unconditionally !Unpin so that rustc doesn't mark mutable references to it as noalias.

As far as I understand, either mitigation is enough on its own to stop the problem, but I don't see any harm in adding both. Our PollFn is not part of the public API of Tokio, so there's no problem with making it !Unpin. The type is only marked pub to allow use in code generated by our macros.

Adding a test for this is surprisingly difficult due to rust#52234, since we don't run miri on the tests/ folder, so I haven't done so. However, I have tested locally that the mitigations work.

@Darksonn Darksonn added A-tokio Area: The main tokio crate M-macros Module: macros in the main Tokio crate labels Oct 10, 2022
@Darksonn
Copy link
Contributor Author

This does increase the size of the futures. We could decide to only change poll_fn.rs to avoid that, though that mitigation feels like the more brittle of the two to me.

Copy link

@RalfJung RalfJung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not knowing tokio I can't check if those macro changes make any sense, but on the conceptual level this makes sense to me. I left some comments that could help clarify what exactly the reference change in the macro does.

And yes either of these changes should be enough.

tokio/src/macros/join.rs Show resolved Hide resolved
tokio/src/future/poll_fn.rs Outdated Show resolved Hide resolved
Copy link
Member

@carllerche carllerche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really understand the details of the issue... it is unfortunate that the futures get bigger :(

Copy link
Member

@hawkw hawkw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks correct to me! i commented on a couple very minor nits.

it's kind of a shame that the futures got a word larger, but...what can you do, i suppose. i guess that mitigation is worth having in case the implementation of join! or select! is ever changed to not use tokio's version of PollFn...

tokio/src/macros/join.rs Show resolved Hide resolved
@@ -64,6 +65,7 @@ async fn two_await() {
}

#[test]
#[cfg(target_pointer_width = "64")]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we be making similar assertions about the size of the futures on 32-bit platforms as well? or, calculate the expected size by adding either 8 or 4 bytes based on the value of target_pointer_width?

@@ -207,6 +207,7 @@ async fn nested() {
}

#[maybe_tokio_test]
#[cfg(target_pointer_width = "64")]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as with join, could we make a separate set of assertions about the future size on 32-bit platforms?

@Darksonn Darksonn merged commit 6929dec into master Oct 14, 2022
@Darksonn Darksonn deleted the alice/mitigate-pollfn-trouble branch October 14, 2022 07:45
crapStone pushed a commit to Calciumdibromid/CaBr2 that referenced this pull request Nov 22, 2022
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [tokio](https://tokio.rs) ([source](https://github.com/tokio-rs/tokio)) | dependencies | minor | `1.21.2` -> `1.22.0` |
| [tokio](https://tokio.rs) ([source](https://github.com/tokio-rs/tokio)) | dev-dependencies | minor | `1.21.2` -> `1.22.0` |

---

### Release Notes

<details>
<summary>tokio-rs/tokio</summary>

### [`v1.22.0`](https://github.com/tokio-rs/tokio/releases/tag/tokio-1.22.0): Tokio v1.22.0

[Compare Source](tokio-rs/tokio@tokio-1.21.2...tokio-1.22.0)

##### Added

-   runtime: add `Handle::runtime_flavor` ([#&#8203;5138])
-   sync: add `Mutex::blocking_lock_owned` ([#&#8203;5130])
-   sync: add `Semaphore::MAX_PERMITS` ([#&#8203;5144])
-   sync: add `merge()` to semaphore permits ([#&#8203;4948])
-   sync: add `mpsc::WeakUnboundedSender` ([#&#8203;5189])

##### Added (unstable)

-   process: add `Command::process_group` ([#&#8203;5114])
-   runtime: export metrics about the blocking thread pool ([#&#8203;5161])
-   task: add `task::id()` and `task::try_id()` ([#&#8203;5171])

##### Fixed

-   macros: don't take ownership of futures in macros ([#&#8203;5087])
-   runtime: fix Stacked Borrows violation in `LocalOwnedTasks` ([#&#8203;5099])
-   runtime: mitigate ABA with 32-bit queue indices when possible ([#&#8203;5042])
-   task: wake local tasks to the local queue when woken by the same thread ([#&#8203;5095])
-   time: panic in release mode when `mark_pending` called illegally ([#&#8203;5093])
-   runtime: fix typo in expect message ([#&#8203;5169])
-   runtime: fix `unsync_load` on atomic types ([#&#8203;5175])
-   task: elaborate safety comments in task deallocation ([#&#8203;5172])
-   runtime: fix `LocalSet` drop in thread local ([#&#8203;5179])
-   net: remove libc type leakage in a public API ([#&#8203;5191])
-   runtime: update the alignment of `CachePadded` ([#&#8203;5106])

##### Changed

-   io: make `tokio::io::copy` continue filling the buffer when writer stalls ([#&#8203;5066])
-   runtime: remove `coop::budget` from `LocalSet::run_until` ([#&#8203;5155])
-   sync: make `Notify` panic safe ([#&#8203;5154])

##### Documented

-   io: fix doc for `write_i8` to use signed integers ([#&#8203;5040])
-   net: fix doc typos for TCP and UDP `set_tos` methods ([#&#8203;5073])
-   net: fix function name in `UdpSocket::recv` documentation ([#&#8203;5150])
-   sync: typo in `TryLockError` for `RwLock::try_write` ([#&#8203;5160])
-   task: document that spawned tasks execute immediately ([#&#8203;5117])
-   time: document return type of `timeout` ([#&#8203;5118])
-   time: document that `timeout` checks only before poll ([#&#8203;5126])
-   sync: specify return type of `oneshot::Receiver` in docs ([#&#8203;5198])

##### Internal changes

-   runtime: use const `Mutex::new` for globals ([#&#8203;5061])
-   runtime: remove `Option` around `mio::Events` in io driver ([#&#8203;5078])
-   runtime: remove a conditional compilation clause ([#&#8203;5104])
-   runtime: remove a reference to internal time handle ([#&#8203;5107])
-   runtime: misc time driver cleanup ([#&#8203;5120])
-   runtime: move signal driver to runtime module ([#&#8203;5121])
-   runtime: signal driver now uses I/O driver directly ([#&#8203;5125])
-   runtime: start decoupling I/O driver and I/O handle ([#&#8203;5127])
-   runtime: switch `io::handle` refs with scheduler:Handle ([#&#8203;5128])
-   runtime: remove Arc from I/O driver ([#&#8203;5134])
-   runtime: use signal driver handle via `scheduler::Handle` ([#&#8203;5135])
-   runtime: move internal clock fns out of context ([#&#8203;5139])
-   runtime: remove `runtime::context` module ([#&#8203;5140])
-   runtime: keep driver cfgs in `driver.rs` ([#&#8203;5141])
-   runtime: add `runtime::context` to unify thread-locals ([#&#8203;5143])
-   runtime: rename some confusing internal variables/fns ([#&#8203;5151])
-   runtime: move `coop` mod into `runtime` ([#&#8203;5152])
-   runtime: move budget state to context thread-local ([#&#8203;5157])
-   runtime: move park logic into runtime module ([#&#8203;5158])
-   runtime: move `Runtime` into its own file ([#&#8203;5159])
-   runtime: unify entering a runtime with `Handle::enter` ([#&#8203;5163])
-   runtime: remove handle reference from each scheduler ([#&#8203;5166])
-   runtime: move `enter` into `context` ([#&#8203;5167])
-   runtime: combine context and entered thread-locals ([#&#8203;5168])
-   runtime: fix accidental unsetting of current handle ([#&#8203;5178])
-   runtime: move `CoreStage` methods to `Core` ([#&#8203;5182])
-   sync: name mpsc semaphore types ([#&#8203;5146])

[#&#8203;4948]: tokio-rs/tokio#4948

[#&#8203;5040]: tokio-rs/tokio#5040

[#&#8203;5042]: tokio-rs/tokio#5042

[#&#8203;5061]: tokio-rs/tokio#5061

[#&#8203;5066]: tokio-rs/tokio#5066

[#&#8203;5073]: tokio-rs/tokio#5073

[#&#8203;5078]: tokio-rs/tokio#5078

[#&#8203;5087]: tokio-rs/tokio#5087

[#&#8203;5093]: tokio-rs/tokio#5093

[#&#8203;5095]: tokio-rs/tokio#5095

[#&#8203;5099]: tokio-rs/tokio#5099

[#&#8203;5104]: tokio-rs/tokio#5104

[#&#8203;5106]: tokio-rs/tokio#5106

[#&#8203;5107]: tokio-rs/tokio#5107

[#&#8203;5114]: tokio-rs/tokio#5114

[#&#8203;5117]: tokio-rs/tokio#5117

[#&#8203;5118]: tokio-rs/tokio#5118

[#&#8203;5120]: tokio-rs/tokio#5120

[#&#8203;5121]: tokio-rs/tokio#5121

[#&#8203;5125]: tokio-rs/tokio#5125

[#&#8203;5126]: tokio-rs/tokio#5126

[#&#8203;5127]: tokio-rs/tokio#5127

[#&#8203;5128]: tokio-rs/tokio#5128

[#&#8203;5130]: tokio-rs/tokio#5130

[#&#8203;5134]: tokio-rs/tokio#5134

[#&#8203;5135]: tokio-rs/tokio#5135

[#&#8203;5138]: tokio-rs/tokio#5138

[#&#8203;5138]: tokio-rs/tokio#5138

[#&#8203;5139]: tokio-rs/tokio#5139

[#&#8203;5140]: tokio-rs/tokio#5140

[#&#8203;5141]: tokio-rs/tokio#5141

[#&#8203;5143]: tokio-rs/tokio#5143

[#&#8203;5144]: tokio-rs/tokio#5144

[#&#8203;5144]: tokio-rs/tokio#5144

[#&#8203;5146]: tokio-rs/tokio#5146

[#&#8203;5150]: tokio-rs/tokio#5150

[#&#8203;5151]: tokio-rs/tokio#5151

[#&#8203;5152]: tokio-rs/tokio#5152

[#&#8203;5154]: tokio-rs/tokio#5154

[#&#8203;5155]: tokio-rs/tokio#5155

[#&#8203;5157]: tokio-rs/tokio#5157

[#&#8203;5158]: tokio-rs/tokio#5158

[#&#8203;5159]: tokio-rs/tokio#5159

[#&#8203;5160]: tokio-rs/tokio#5160

[#&#8203;5161]: tokio-rs/tokio#5161

[#&#8203;5163]: tokio-rs/tokio#5163

[#&#8203;5166]: tokio-rs/tokio#5166

[#&#8203;5167]: tokio-rs/tokio#5167

[#&#8203;5168]: tokio-rs/tokio#5168

[#&#8203;5169]: tokio-rs/tokio#5169

[#&#8203;5171]: tokio-rs/tokio#5171

[#&#8203;5172]: tokio-rs/tokio#5172

[#&#8203;5175]: tokio-rs/tokio#5175

[#&#8203;5178]: tokio-rs/tokio#5178

[#&#8203;5179]: tokio-rs/tokio#5179

[#&#8203;5182]: tokio-rs/tokio#5182

[#&#8203;5189]: tokio-rs/tokio#5189

[#&#8203;5191]: tokio-rs/tokio#5191

[#&#8203;5198]: tokio-rs/tokio#5198

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about these updates again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNC4yNy4xIiwidXBkYXRlZEluVmVyIjoiMzQuMjkuMiJ9-->

Co-authored-by: cabr2-bot <cabr2.help@gmail.com>
Reviewed-on: https://codeberg.org/Calciumdibromid/CaBr2/pulls/1651
Reviewed-by: crapStone <crapstone@noreply.codeberg.org>
Co-authored-by: Calciumdibromid Bot <cabr2_bot@noreply.codeberg.org>
Co-committed-by: Calciumdibromid Bot <cabr2_bot@noreply.codeberg.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-tokio Area: The main tokio crate M-macros Module: macros in the main Tokio crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants