Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add spawn_pinned to tokio-util #3370

Merged
merged 47 commits into from Jan 27, 2022
Merged

Add spawn_pinned to tokio-util #3370

merged 47 commits into from Jan 27, 2022

Commits on Jan 3, 2021

  1. Add Jetbrains IDE .idea folder to gitignore

    @AzureMarker
    AzureMarker committed Jan 3, 2021
    Copy the full SHA
    9d0ee8d View commit details
    Browse the repository at this point in the history

  2. tokio-util: add (incomplete) spawn_pinned support 

    The pinned workers don't shut down when the tokio runtime does, so the
included test never completes.

The worker count is currently hard-coded with a fixme.

PinnedPool does not reset, so if a second runtime is started after the
first finishes, spawn_pinned will not work (the first issue about not
shutting down the workers prevents this scenario from happening atm).
    @AzureMarker
    AzureMarker committed Jan 3, 2021
    Copy the full SHA
    34ccb19 View commit details
    Browse the repository at this point in the history

  3. Fix clippy warnings

    @AzureMarker
    AzureMarker committed Jan 3, 2021
    Copy the full SHA
    1260d6f View commit details
    Browse the repository at this point in the history

  4. Fix the rest of the clippy warnings

    @AzureMarker
    AzureMarker committed Jan 3, 2021
    Copy the full SHA
    4501084 View commit details
    Browse the repository at this point in the history

  5. Ignore the spawn_pinned test for now because it doesn't terminate

    @AzureMarker
    AzureMarker committed Jan 3, 2021
    Copy the full SHA
    cfdbb8b View commit details
    Browse the repository at this point in the history

Commits on Jan 4, 2021

  1. Replace global pool with owned handles to individual pools

    @AzureMarker
    AzureMarker committed Jan 4, 2021
    Copy the full SHA
    9dd38b9 View commit details
    Browse the repository at this point in the history

  2. Add docs

    @AzureMarker
    AzureMarker committed Jan 4, 2021
    Copy the full SHA
    af54d47 View commit details
    Browse the repository at this point in the history

Commits on Jan 5, 2021

  1. Enforce a pool size of at least 1

    @AzureMarker
    AzureMarker committed Jan 5, 2021
    Copy the full SHA
    a621591 View commit details
    Browse the repository at this point in the history

  2. Track # of tasks in each worker and use least burdened in spawn_pinned

    @AzureMarker
    AzureMarker committed Jan 5, 2021
    Copy the full SHA
    0df5844 View commit details
    Browse the repository at this point in the history

  3. Add another test

    @AzureMarker
    AzureMarker committed Jan 5, 2021
    Copy the full SHA
    27df2b2 View commit details
    Browse the repository at this point in the history

Commits on Sep 28, 2021

  1. Revert "Add Jetbrains IDE .idea folder to gitignore" 

    This reverts commit 9d0ee8d.
    @AzureMarker
    AzureMarker committed Sep 28, 2021
    Copy the full SHA
    e454f4f View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into feature/spawn-pinned

    @AzureMarker
    AzureMarker committed Sep 28, 2021
    Copy the full SHA
    c073781 View commit details
    Browse the repository at this point in the history

Commits on Oct 24, 2021

  1. Merge branch 'master' into feature/spawn-pinned

    @AzureMarker
    AzureMarker committed Oct 24, 2021
    Copy the full SHA
    0189798 View commit details
    Browse the repository at this point in the history

Commits on Oct 29, 2021

  1. Replace spawn_blocking with std::thread::spawn

    @AzureMarker
    AzureMarker committed Oct 29, 2021
    Copy the full SHA
    764414c View commit details
    Browse the repository at this point in the history

  2. Enable all features on the worker runtime

    @AzureMarker
    AzureMarker committed Oct 29, 2021
    Copy the full SHA
    ff14408 View commit details
    Browse the repository at this point in the history

  3. Fix giving more than one task to the least burdened worker

    @AzureMarker
    AzureMarker committed Oct 29, 2021
    Copy the full SHA
    ca7f8e8 View commit details
    Browse the repository at this point in the history

  4. Remove downcasting in spawn_pinned

    @AzureMarker
    AzureMarker committed Oct 29, 2021
    Copy the full SHA
    9486e32 View commit details
    Browse the repository at this point in the history

  5. Add spawn_pinned_nonblocking

    @AzureMarker
    AzureMarker committed Oct 29, 2021
    Copy the full SHA
    2939338 View commit details
    Browse the repository at this point in the history

Commits on Oct 30, 2021

  1. Fix doc link to spawn_pinned

    @AzureMarker
    AzureMarker committed Oct 30, 2021
    Copy the full SHA
    ba8b6be View commit details
    Browse the repository at this point in the history

Commits on Nov 21, 2021

  1. Simplify spawn_pinned test

    @AzureMarker
    AzureMarker committed Nov 21, 2021
    Copy the full SHA
    c4cce06 View commit details
    Browse the repository at this point in the history

  2. Move LocalPoolHandle creation into LocalPoolHandle::new

    @AzureMarker
    AzureMarker committed Nov 21, 2021
    Copy the full SHA
    09839ad View commit details
    Browse the repository at this point in the history

  3. Simplify bounds on spawn_pinned functions

    @AzureMarker
    AzureMarker committed Nov 21, 2021
    Copy the full SHA
    24ca0c7 View commit details
    Browse the repository at this point in the history

  4. Create local pool runtimes on user's thread in case it panics

    @AzureMarker
    AzureMarker committed Nov 21, 2021
    Copy the full SHA
    5444dda View commit details
    Browse the repository at this point in the history

  5. Tweak how join handle send error is dropped

    @AzureMarker
    AzureMarker committed Nov 21, 2021
    Copy the full SHA
    45b5435 View commit details
    Browse the repository at this point in the history

  6. Fix some compile errors

    @AzureMarker
    AzureMarker committed Nov 21, 2021
    Copy the full SHA
    75b9bb1 View commit details
    Browse the repository at this point in the history

Commits on Nov 25, 2021

  1. Make spawn_pinned async and name blocking version spawn_pinned_blocking

    @AzureMarker
    AzureMarker committed Nov 25, 2021
    Copy the full SHA
    e5972b0 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into feature/spawn-pinned

    @AzureMarker
    AzureMarker committed Nov 25, 2021
    Copy the full SHA
    4ce300f View commit details
    Browse the repository at this point in the history

  3. Make spawn_pinned synchronous again, but immediately send the task 

    This also makes spawn_pinned_blocking pretty redundant, so it is
removed.
    @AzureMarker
    AzureMarker committed Nov 25, 2021
    Copy the full SHA
    4622c2a View commit details
    Browse the repository at this point in the history

  4. Fix broken link in docs

    @AzureMarker
    AzureMarker committed Nov 25, 2021
    Copy the full SHA
    e8e7af9 View commit details
    Browse the repository at this point in the history

Commits on Nov 26, 2021

  1. Synchronously return join handle in spawn_pinned 

    Also added more error handling and comments. There are still two more
error handling issues:
1. `create_task` could panic, which kills the worker. Using
   `catch_unwind` limits the types you can use for `create_task` too
   much. We could use AssertUnwindSafe here, but probably shouldn't.
2. If a worker dies for some reason, the pool doesn't heal (recreate the
   worker). Healing the pool would require more mutable state and
   mutual exclusion, which might get complicated.
    @AzureMarker
    AzureMarker committed Nov 26, 2021
    Copy the full SHA
    bd32346 View commit details
    Browse the repository at this point in the history

  2. Simplify channel types

    @AzureMarker
    AzureMarker committed Nov 26, 2021
    Copy the full SHA
    7769e0a View commit details
    Browse the repository at this point in the history

Commits on Dec 3, 2021

  1. Use catch_unwind and AssertUnwindSafe to protect workers from panics

    @AzureMarker
    AzureMarker committed Dec 3, 2021
    Copy the full SHA
    b64edff View commit details
    Browse the repository at this point in the history

Commits on Dec 4, 2021

  1. Call create_task in spawn_local to avoid using catch_unwind ourselves

    @AzureMarker
    AzureMarker committed Dec 4, 2021
    Copy the full SHA
    bb6bdb1 View commit details
    Browse the repository at this point in the history

  2. Directly resume panic unwinding if the task panics

    @AzureMarker
    AzureMarker committed Dec 4, 2021
    Copy the full SHA
    ff7ad94 View commit details
    Browse the repository at this point in the history

Commits on Dec 5, 2021

  1. Automatically decrement job count via guard (Drop impl)

    @AzureMarker
    AzureMarker committed Dec 5, 2021
    Copy the full SHA
    5e21d68 View commit details
    Browse the repository at this point in the history

  2. Propagate task cancellation in spawn_pinned

    @AzureMarker
    AzureMarker committed Dec 5, 2021
    Copy the full SHA
    b7c4da9 View commit details
    Browse the repository at this point in the history

  3. Propagate task cancellation in one more spot (very unlikely to occur)

    @AzureMarker
    AzureMarker committed Dec 5, 2021
    Copy the full SHA
    2067c61 View commit details
    Browse the repository at this point in the history

  4. Minor renaming/formatting

    @AzureMarker
    AzureMarker committed Dec 5, 2021
    Copy the full SHA
    16e1521 View commit details
    Browse the repository at this point in the history

Commits on Dec 21, 2021

  1. Add a test to ensure tasks are balanced across workers

    @AzureMarker
    AzureMarker committed Dec 21, 2021
    Copy the full SHA
    62f5ec7 View commit details
    Browse the repository at this point in the history

  2. Use a robust abort handling mechanism

    @AzureMarker
    AzureMarker committed Dec 21, 2021
    Copy the full SHA
    c130046 View commit details
    Browse the repository at this point in the history

Commits on Dec 28, 2021

  1. Fix typo (missing word)

    @AzureMarker
    AzureMarker committed Dec 28, 2021
    Copy the full SHA
    8aaa3ca View commit details
    Browse the repository at this point in the history

Commits on Dec 29, 2021

  1. Return job count guard when incrementing worker

    @AzureMarker
    AzureMarker committed Dec 29, 2021
    Copy the full SHA
    9214302 View commit details
    Browse the repository at this point in the history

  2. Add a debug assert to verify the job count doesn't go negative

    @AzureMarker
    AzureMarker committed Dec 29, 2021
    Copy the full SHA
    1bd5bf2 View commit details
    Browse the repository at this point in the history

  3. Check against panic message in tests

    @AzureMarker
    AzureMarker committed Dec 29, 2021
    Copy the full SHA
    34d3646 View commit details
    Browse the repository at this point in the history

  4. Remove sleeps in spawn_pinned tests

    @AzureMarker
    AzureMarker committed Dec 29, 2021
    Copy the full SHA
    2937f7e View commit details
    Browse the repository at this point in the history

  5. Update tokio-util/src/task/spawn_pinned.rs 

    Co-authored-by: Alice Ryhl <aliceryhl@google.com>
    @AzureMarker @Darksonn
    AzureMarker and Darksonn committed Dec 29, 2021
    Copy the full SHA
    5d9bbde View commit details
    Browse the repository at this point in the history

  6. Fix some compile errors after applying suggestion

    @AzureMarker
    AzureMarker committed Dec 29, 2021
    Copy the full SHA
    fb11c45 View commit details
    Browse the repository at this point in the history