io: Refactor out usage of Weak in the io handle #4656

hidva · 2022-05-05T17:37:05Z

Motivation

Solution

The cycle is broken by making sure nobody can register a new io resources on shutdown and by emptying the driver.

hawkw · 2022-05-05T21:33:22Z

this no longer seems to compile (i think because inner is no longer an Option?): https://github.com/tokio-rs/tokio/runs/6311895497?check_suite_focus=true

hidva · 2022-05-06T04:43:49Z

@LucioFranco Please take a look, I have done a benchmark with monoio-benchmark, and there is no performance fallback.

LucioFranco · 2022-05-06T17:27:28Z

@carllerche can you take a look at this?

LucioFranco

Looks like a good start! I would love some other eyes on this as well though cc @carllerche

tokio/src/io/driver/mod.rs

tokio/src/runtime/metrics/runtime.rs

renancloudwalk

👍

tokio/src/io/driver/mod.rs

Darksonn · 2022-05-16T17:40:54Z

tokio/src/io/driver/mod.rs

+       impl Handle {
+            pub(crate) fn metrics(&self) -> &IoDriverMetrics {
+                &self.inner.metrics
+            }
+       }


The impl Handle is currently indented by seven spaces rather than eight.

done.

Sorry, because I see the last commit is indented like this.

carllerche · 2022-05-16T18:52:08Z

tokio/src/io/driver/mod.rs

-    fn shutdown(&mut self) {}
+    fn shutdown(&mut self) {
+        if self.inner.shutdown() {
+            self.resources.for_each(|io| {


If this statement is commented out, do tests fail? i.e. if there is a memory leak, is it caught my CI? If not, we should add one.

Yes.

First, we have to comment out clear_wakers().

Second, if we commented out io.shutdown() here, we'd get a memory leak, otherwise we wouldn't.

$ valgrind --error-exitcode=1 --leak-check=full --show-leak-kinds=all ./target/debug/test-mem ==756352== LEAK SUMMARY: ==756352== definitely lost: 96 bytes in 1 blocks ==756352== indirectly lost: 20,856 bytes in 35 blocks ==756352== possibly lost: 0 bytes in 0 blocks ==756352== still reachable: 9,952 bytes in 69 blocks ==756352== suppressed: 0 bytes in 0 blocks ==756352== ==756352== For lists of detected and suppressed errors, rerun with: -s ==756352== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)

(It looks like we don't need to call clear_wakers() in Drop for Registration anymore.

carllerche

Thanks for doing this. The general direction looks good to me. I would like to double check that we have CI tests that check for memory leaks (valgrind). Given that a bug here will result in a mem leak, it would be good to check :) I left a note inline where, if commented, I would hope CI would fail.

Thanks!

carllerche · 2022-05-16T18:56:55Z

There are a few places valgrind is used in CI:

I wonder if we could run our entire CI suite w/ valgrind...

Darksonn · 2022-05-16T19:04:15Z

Regarding leak checks, see the recently merged #4696. You may want to merge in master to get this step in the PR's CI.

hidva · 2022-05-16T21:30:57Z

Regarding leak checks, see the recently merged #4696. You may want to merge in master to get this step in the PR's CI.

done

Darksonn · 2022-05-19T08:24:39Z

Thanks!

This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [tokio](https://tokio.rs) ([source](https://github.com/tokio-rs/tokio)) | dependencies | minor | `1.18.2` -> `1.19.1` | | [tokio](https://tokio.rs) ([source](https://github.com/tokio-rs/tokio)) | dev-dependencies | minor | `1.18.2` -> `1.19.1` | --- ### Release Notes <details> <summary>tokio-rs/tokio</summary> ### [`v1.19.1`](https://github.com/tokio-rs/tokio/releases/tag/tokio-1.19.1) [Compare Source](tokio-rs/tokio@tokio-1.19.0...tokio-1.19.1) ##### 1.19.1 (June 5, 2022) This release fixes a bug in `Notified::enable`. ([#4747]) [#4747]: tokio-rs/tokio#4747 ### [`v1.19.0`](https://github.com/tokio-rs/tokio/releases/tag/tokio-1.19.0) [Compare Source](tokio-rs/tokio@tokio-1.18.2...tokio-1.19.0) ##### 1.19.0 (June 3, 2022) ##### Added - runtime: add `is_finished` method for `JoinHandle` and `AbortHandle` ([#4709]) - runtime: make global queue and event polling intervals configurable ([#4671]) - sync: add `Notified::enable` ([#4705]) - sync: add `watch::Sender::send_if_modified` ([#4591]) - sync: add resubscribe method to broadcast::Receiver ([#4607]) - net: add `take_error` to `TcpSocket` and `TcpStream` ([#4739]) ##### Changed - io: refactor out usage of Weak in the io handle ([#4656]) ##### Fixed - macros: avoid starvation in `join!` and `try_join!` ([#4624]) ##### Documented - runtime: clarify semantics of tasks outliving `block_on` ([#4729]) - time: fix example for `MissedTickBehavior::Burst` ([#4713]) ##### Unstable - metrics: correctly update atomics in `IoDriverMetrics` ([#4725]) - metrics: fix compilation with unstable, process, and rt, but without net ([#4682]) - task: add `#[track_caller]` to `JoinSet`/`JoinMap` ([#4697]) - task: add `Builder::{spawn_on, spawn_local_on, spawn_blocking_on}` ([#4683]) - task: add `consume_budget` for cooperative scheduling ([#4498]) - task: add `join_set::Builder` for configuring `JoinSet` tasks ([#4687]) - task: update return value of `JoinSet::join_one` ([#4726]) [#4498]: tokio-rs/tokio#4498 [#4591]: tokio-rs/tokio#4591 [#4607]: tokio-rs/tokio#4607 [#4624]: tokio-rs/tokio#4624 [#4656]: tokio-rs/tokio#4656 [#4671]: tokio-rs/tokio#4671 [#4682]: tokio-rs/tokio#4682 [#4683]: tokio-rs/tokio#4683 [#4687]: tokio-rs/tokio#4687 [#4697]: tokio-rs/tokio#4697 [#4705]: tokio-rs/tokio#4705 [#4709]: tokio-rs/tokio#4709 [#4713]: tokio-rs/tokio#4713 [#4725]: tokio-rs/tokio#4725 [#4726]: tokio-rs/tokio#4726 [#4729]: tokio-rs/tokio#4729 [#4739]: tokio-rs/tokio#4739 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about these updates again. --- - [x] If you want to rebase/retry this PR, click this checkbox. --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). Co-authored-by: cabr2-bot <cabr2.help@gmail.com> Reviewed-on: https://codeberg.org/Calciumdibromid/CaBr2/pulls/1394 Reviewed-by: crapStone <crapstone@noreply.codeberg.org> Co-authored-by: Calciumdibromid Bot <cabr2_bot@noreply.codeberg.org> Co-committed-by: Calciumdibromid Bot <cabr2_bot@noreply.codeberg.org>

hidva changed the title ~~io: Refactor out usage of Weak in the io handle~~ [WIP]io: Refactor out usage of Weak in the io handle May 5, 2022

Darksonn added A-tokio Area: The main tokio crate M-runtime Module: tokio/runtime labels May 5, 2022

Darksonn requested a review from LucioFranco May 5, 2022 18:47

github-actions bot added the R-loom Run loom tests on this PR label May 6, 2022

hidva changed the title ~~[WIP]io: Refactor out usage of Weak in the io handle~~ io: Refactor out usage of Weak in the io handle May 6, 2022

LucioFranco requested changes May 6, 2022

View reviewed changes

tokio/src/io/driver/mod.rs Outdated Show resolved Hide resolved

tokio/src/io/driver/mod.rs Outdated Show resolved Hide resolved

tokio/src/io/driver/mod.rs Show resolved Hide resolved

tokio/src/runtime/metrics/runtime.rs Show resolved Hide resolved

renancloudwalk approved these changes May 7, 2022

View reviewed changes

tokio/src/io/driver/mod.rs Show resolved Hide resolved

hidva requested a review from LucioFranco May 9, 2022 13:15

LucioFranco approved these changes May 16, 2022

View reviewed changes

LucioFranco requested review from Darksonn and carllerche May 16, 2022 14:43

Darksonn reviewed May 16, 2022

View reviewed changes

tokio/src/io/driver/mod.rs Outdated Show resolved Hide resolved

Darksonn reviewed May 16, 2022

View reviewed changes

carllerche reviewed May 16, 2022

View reviewed changes

hidva added 3 commits May 17, 2022 04:37

io: Refactor out usage of Weak in the io handle

36633f0

io: Format the code manually

c535837

io: Format the code manually

43fce89

Darksonn approved these changes May 19, 2022

View reviewed changes

Darksonn merged commit 931a777 into tokio-rs:master May 19, 2022

Darksonn mentioned this pull request Jun 3, 2022

chore: prepare Tokio v1.19.0 #4738

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

io: Refactor out usage of Weak in the io handle #4656

io: Refactor out usage of Weak in the io handle #4656

hidva commented May 5, 2022

hawkw commented May 5, 2022

hidva commented May 6, 2022

LucioFranco commented May 6, 2022

LucioFranco left a comment

renancloudwalk left a comment

Darksonn May 16, 2022

hidva May 16, 2022

carllerche May 16, 2022

hidva May 16, 2022 •

edited

carllerche left a comment

carllerche commented May 16, 2022

Darksonn commented May 16, 2022

hidva commented May 16, 2022

Darksonn commented May 19, 2022

io: Refactor out usage of Weak in the io handle #4656

io: Refactor out usage of Weak in the io handle #4656

Conversation

hidva commented May 5, 2022

Motivation

Solution

hawkw commented May 5, 2022

hidva commented May 6, 2022

LucioFranco commented May 6, 2022

LucioFranco left a comment

Choose a reason for hiding this comment

renancloudwalk left a comment

Choose a reason for hiding this comment

Darksonn May 16, 2022

Choose a reason for hiding this comment

hidva May 16, 2022

Choose a reason for hiding this comment

carllerche May 16, 2022

Choose a reason for hiding this comment

hidva May 16, 2022 • edited

Choose a reason for hiding this comment

carllerche left a comment

Choose a reason for hiding this comment

carllerche commented May 16, 2022

Darksonn commented May 16, 2022

hidva commented May 16, 2022

Darksonn commented May 19, 2022

hidva May 16, 2022 •

edited