rt: fix `LocalSet` drop in thread local #5179

carllerche · 2022-11-09T22:15:07Z

LocalSet cleans up any tasks that have not yet been completed when it is dropped. Previously, this cleanup process required access to a thread-local. Suppose a LocalSet is stored in a thread-local itself. In that case, when it is dropped, there is no guarantee the drop implementation will be able to access the internal LocalSet thread-local as it may already have been destroyed.

The internal LocalSet thread local is mainly used to avoid writing unsafe code. All LocalState that cannot be moved across threads is stored in the thread-local and accessed on demand.

This patch moves this local-only state into the LocalSet's "shared" struct. Because this struct is Send, the local-only state is stored in UnsafeCell, and callers must ensure not to touch it from other threads.

A debug assertion is added to enforce this requirement in tests.

Fixes #5162

`LocalSet` cleans up any tasks that have not yet completed when it is dropped. Previously, this cleanup process required access to a thread-local. If a `LocalSet` is stored in a thread-local itself, when it is dropped, there is no guarantee the drop implementation will be able to access the internal `LocalSet` thread-local as it may already have been destroyed. The internal `LocalSet` thread local is mostly used to avoid writing unsafe code. All `LocalState` that cannot be moved across threads is stored in the thread-local and accessed on demand. This patch moves this local-only state into the `LocalSet`'s "shared" struct. Because this struct *is* `Send`, the local-only state is stored in `UnsafeCell` and callers must ensure to not touch it from other threads. A debug assertion is added to enforce this requirement in tests.

carllerche · 2022-11-09T22:19:43Z

The budget changes are not part of this change but triggered a panic that had to be fixed before exposing the underlying issue.

This reverts commit d8c7d27.

carllerche · 2022-11-09T23:06:46Z

I have not yet been able to get the backtrace for the panic that causes this to fail. I'll have to continue tomorrow.

carllerche · 2022-11-10T00:05:32Z

Ci is failing due to a double panic on OS X. I cannot repro the double panic while also getting the backtraces.

The second panic, however, is due to calling std::thread::current() after the thread-local used by the thread has been destroyed. The fix for that will be to stop using std::thread::current().

tokio/tests/task_local_set.rs

carllerche · 2022-11-10T16:28:56Z

CI passed except for FreeBSD which hung for some reason (not the test, I think the Ci service just failed to run).

Darksonn · 2022-11-10T17:12:57Z

tokio/tests/task_local_set.rs

+#[test]
+fn store_local_set_in_thread_local_with_runtime() {


Link to issue on this test?

Is there a relevant discussion? We don't usually link to issues afaik unless there is key context.

I guess no.

This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [tokio](https://tokio.rs) ([source](https://github.com/tokio-rs/tokio)) | dependencies | minor | `1.21.2` -> `1.22.0` | | [tokio](https://tokio.rs) ([source](https://github.com/tokio-rs/tokio)) | dev-dependencies | minor | `1.21.2` -> `1.22.0` | --- ### Release Notes <details> <summary>tokio-rs/tokio</summary> ### [`v1.22.0`](https://github.com/tokio-rs/tokio/releases/tag/tokio-1.22.0): Tokio v1.22.0 [Compare Source](tokio-rs/tokio@tokio-1.21.2...tokio-1.22.0) ##### Added - runtime: add `Handle::runtime_flavor` ([#5138]) - sync: add `Mutex::blocking_lock_owned` ([#5130]) - sync: add `Semaphore::MAX_PERMITS` ([#5144]) - sync: add `merge()` to semaphore permits ([#4948]) - sync: add `mpsc::WeakUnboundedSender` ([#5189]) ##### Added (unstable) - process: add `Command::process_group` ([#5114]) - runtime: export metrics about the blocking thread pool ([#5161]) - task: add `task::id()` and `task::try_id()` ([#5171]) ##### Fixed - macros: don't take ownership of futures in macros ([#5087]) - runtime: fix Stacked Borrows violation in `LocalOwnedTasks` ([#5099]) - runtime: mitigate ABA with 32-bit queue indices when possible ([#5042]) - task: wake local tasks to the local queue when woken by the same thread ([#5095]) - time: panic in release mode when `mark_pending` called illegally ([#5093]) - runtime: fix typo in expect message ([#5169]) - runtime: fix `unsync_load` on atomic types ([#5175]) - task: elaborate safety comments in task deallocation ([#5172]) - runtime: fix `LocalSet` drop in thread local ([#5179]) - net: remove libc type leakage in a public API ([#5191]) - runtime: update the alignment of `CachePadded` ([#5106]) ##### Changed - io: make `tokio::io::copy` continue filling the buffer when writer stalls ([#5066]) - runtime: remove `coop::budget` from `LocalSet::run_until` ([#5155]) - sync: make `Notify` panic safe ([#5154]) ##### Documented - io: fix doc for `write_i8` to use signed integers ([#5040]) - net: fix doc typos for TCP and UDP `set_tos` methods ([#5073]) - net: fix function name in `UdpSocket::recv` documentation ([#5150]) - sync: typo in `TryLockError` for `RwLock::try_write` ([#5160]) - task: document that spawned tasks execute immediately ([#5117]) - time: document return type of `timeout` ([#5118]) - time: document that `timeout` checks only before poll ([#5126]) - sync: specify return type of `oneshot::Receiver` in docs ([#5198]) ##### Internal changes - runtime: use const `Mutex::new` for globals ([#5061]) - runtime: remove `Option` around `mio::Events` in io driver ([#5078]) - runtime: remove a conditional compilation clause ([#5104]) - runtime: remove a reference to internal time handle ([#5107]) - runtime: misc time driver cleanup ([#5120]) - runtime: move signal driver to runtime module ([#5121]) - runtime: signal driver now uses I/O driver directly ([#5125]) - runtime: start decoupling I/O driver and I/O handle ([#5127]) - runtime: switch `io::handle` refs with scheduler:Handle ([#5128]) - runtime: remove Arc from I/O driver ([#5134]) - runtime: use signal driver handle via `scheduler::Handle` ([#5135]) - runtime: move internal clock fns out of context ([#5139]) - runtime: remove `runtime::context` module ([#5140]) - runtime: keep driver cfgs in `driver.rs` ([#5141]) - runtime: add `runtime::context` to unify thread-locals ([#5143]) - runtime: rename some confusing internal variables/fns ([#5151]) - runtime: move `coop` mod into `runtime` ([#5152]) - runtime: move budget state to context thread-local ([#5157]) - runtime: move park logic into runtime module ([#5158]) - runtime: move `Runtime` into its own file ([#5159]) - runtime: unify entering a runtime with `Handle::enter` ([#5163]) - runtime: remove handle reference from each scheduler ([#5166]) - runtime: move `enter` into `context` ([#5167]) - runtime: combine context and entered thread-locals ([#5168]) - runtime: fix accidental unsetting of current handle ([#5178]) - runtime: move `CoreStage` methods to `Core` ([#5182]) - sync: name mpsc semaphore types ([#5146]) [#4948]: tokio-rs/tokio#4948 [#5040]: tokio-rs/tokio#5040 [#5042]: tokio-rs/tokio#5042 [#5061]: tokio-rs/tokio#5061 [#5066]: tokio-rs/tokio#5066 [#5073]: tokio-rs/tokio#5073 [#5078]: tokio-rs/tokio#5078 [#5087]: tokio-rs/tokio#5087 [#5093]: tokio-rs/tokio#5093 [#5095]: tokio-rs/tokio#5095 [#5099]: tokio-rs/tokio#5099 [#5104]: tokio-rs/tokio#5104 [#5106]: tokio-rs/tokio#5106 [#5107]: tokio-rs/tokio#5107 [#5114]: tokio-rs/tokio#5114 [#5117]: tokio-rs/tokio#5117 [#5118]: tokio-rs/tokio#5118 [#5120]: tokio-rs/tokio#5120 [#5121]: tokio-rs/tokio#5121 [#5125]: tokio-rs/tokio#5125 [#5126]: tokio-rs/tokio#5126 [#5127]: tokio-rs/tokio#5127 [#5128]: tokio-rs/tokio#5128 [#5130]: tokio-rs/tokio#5130 [#5134]: tokio-rs/tokio#5134 [#5135]: tokio-rs/tokio#5135 [#5138]: tokio-rs/tokio#5138 [#5138]: tokio-rs/tokio#5138 [#5139]: tokio-rs/tokio#5139 [#5140]: tokio-rs/tokio#5140 [#5141]: tokio-rs/tokio#5141 [#5143]: tokio-rs/tokio#5143 [#5144]: tokio-rs/tokio#5144 [#5144]: tokio-rs/tokio#5144 [#5146]: tokio-rs/tokio#5146 [#5150]: tokio-rs/tokio#5150 [#5151]: tokio-rs/tokio#5151 [#5152]: tokio-rs/tokio#5152 [#5154]: tokio-rs/tokio#5154 [#5155]: tokio-rs/tokio#5155 [#5157]: tokio-rs/tokio#5157 [#5158]: tokio-rs/tokio#5158 [#5159]: tokio-rs/tokio#5159 [#5160]: tokio-rs/tokio#5160 [#5161]: tokio-rs/tokio#5161 [#5163]: tokio-rs/tokio#5163 [#5166]: tokio-rs/tokio#5166 [#5167]: tokio-rs/tokio#5167 [#5168]: tokio-rs/tokio#5168 [#5169]: tokio-rs/tokio#5169 [#5171]: tokio-rs/tokio#5171 [#5172]: tokio-rs/tokio#5172 [#5175]: tokio-rs/tokio#5175 [#5178]: tokio-rs/tokio#5178 [#5179]: tokio-rs/tokio#5179 [#5182]: tokio-rs/tokio#5182 [#5189]: tokio-rs/tokio#5189 [#5191]: tokio-rs/tokio#5191 [#5198]: tokio-rs/tokio#5198 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about these updates again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).  Co-authored-by: cabr2-bot <cabr2.help@gmail.com> Reviewed-on: https://codeberg.org/Calciumdibromid/CaBr2/pulls/1651 Reviewed-by: crapStone <crapstone@noreply.codeberg.org> Co-authored-by: Calciumdibromid Bot <cabr2_bot@noreply.codeberg.org> Co-committed-by: Calciumdibromid Bot <cabr2_bot@noreply.codeberg.org>

carllerche added C-bug Category: This is a bug. A-tokio Area: The main tokio crate M-task Module: tokio/task labels Nov 9, 2022

github-actions bot added the R-loom Run loom tests on this PR label Nov 9, 2022

remove extra UnsafeCell

e6988d0

Darksonn approved these changes Nov 9, 2022

View reviewed changes

carllerche added 2 commits November 9, 2022 14:50

ci debugging yay

d8c7d27

Revert "ci debugging yay"

a2d74b8

This reverts commit d8c7d27.

force populate the local thread ID on LocalSet creation

8cd2f73

Forsworns reviewed Nov 10, 2022

View reviewed changes

tokio/tests/task_local_set.rs Outdated Show resolved Hide resolved