Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support shuffle-based groupby aggregations in dask_cudf #11800

Merged
merged 22 commits into from
Sep 28, 2022
Merged

Support shuffle-based groupby aggregations in dask_cudf #11800

merged 22 commits into from
Sep 28, 2022

Commits on Aug 22, 2022

  1. add explicit-comms shuffle option for sort_values and shuffle

    @rjzamora
    rjzamora committed Aug 22, 2022
    Configuration menu
    Copy the full SHA
    edca3aa View commit details
    Browse the repository at this point in the history

  2. use npartitions=4 in test

    @rjzamora
    rjzamora committed Aug 22, 2022
    Configuration menu
    Copy the full SHA
    750a30b View commit details
    Browse the repository at this point in the history

  3. use npartitions=4 in test

    @rjzamora
    rjzamora committed Aug 22, 2022
    Configuration menu
    Copy the full SHA
    76d8faa View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2022

  1. add shuffle-based groupby

    @rjzamora
    rjzamora committed Sep 20, 2022
    Configuration menu
    Copy the full SHA
    4f30fe1 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'upstream/branch-22.10' into ec-shuffle-… 

    …option
    @rjzamora
    rjzamora committed Sep 20, 2022
    Configuration menu
    Copy the full SHA
    b3de1a8 View commit details
    Browse the repository at this point in the history

  3. use dask-config to support explicit-comms shuffle

    @rjzamora
    rjzamora committed Sep 20, 2022
    Configuration menu
    Copy the full SHA
    eccf5a6 View commit details
    Browse the repository at this point in the history

  4. move _shuffle_context

    @rjzamora
    rjzamora committed Sep 20, 2022
    Configuration menu
    Copy the full SHA
    b01eef9 View commit details
    Browse the repository at this point in the history

  5. fix test

    @rjzamora
    rjzamora committed Sep 20, 2022
    Configuration menu
    Copy the full SHA
    694b25b View commit details
    Browse the repository at this point in the history

  6. add test coverage

    @rjzamora
    rjzamora committed Sep 20, 2022
    Configuration menu
    Copy the full SHA
    04d80b3 View commit details
    Browse the repository at this point in the history

  7. remove commented lines

    @rjzamora
    rjzamora committed Sep 20, 2022
    Configuration menu
    Copy the full SHA
    4c2530b View commit details
    Browse the repository at this point in the history

  8. align with groupby benchmark again

    @rjzamora
    rjzamora committed Sep 20, 2022
    Configuration menu
    Copy the full SHA
    a611c29 View commit details
    Browse the repository at this point in the history

  9. use ec shuffle directly for repartitioning

    @rjzamora
    rjzamora committed Sep 20, 2022
    Configuration menu
    Copy the full SHA
    9137662 View commit details
    Browse the repository at this point in the history

  10. Merge branch 'ec-shuffle-option' into shuffle-groupby

    @rjzamora
    rjzamora committed Sep 20, 2022
    Configuration menu
    Copy the full SHA
    885506f View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2022

  1. Merge remote-tracking branch 'upstream/branch-22.10' into shuffle-gro… 

    …upby
    @rjzamora
    rjzamora committed Sep 27, 2022
    Configuration menu
    Copy the full SHA
    be1bd3b View commit details
    Browse the repository at this point in the history

  2. remove stale code

    @rjzamora
    rjzamora committed Sep 27, 2022
    Configuration menu
    Copy the full SHA
    482b21a View commit details
    Browse the repository at this point in the history

  3. simplify _shuffle_aggregate a bit

    @rjzamora
    rjzamora committed Sep 27, 2022
    Configuration menu
    Copy the full SHA
    35f7fb3 View commit details
    Browse the repository at this point in the history

  4. use rearrange_by_column

    @rjzamora
    rjzamora committed Sep 27, 2022
    Configuration menu
    Copy the full SHA
    437cf66 View commit details
    Browse the repository at this point in the history

Commits on Sep 28, 2022

  1. add explicit-comms test coverage

    @rjzamora
    rjzamora committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    dbbdc7c View commit details
    Browse the repository at this point in the history

  2. simplify further

    @rjzamora
    rjzamora committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    e732baf View commit details
    Browse the repository at this point in the history

  3. fix args bug

    @rjzamora
    rjzamora committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    6f56cef View commit details
    Browse the repository at this point in the history

  4. add note

    @rjzamora
    rjzamora committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    b698e9e View commit details
    Browse the repository at this point in the history

  5. revert changes related to explicit-comms

    @rjzamora
    rjzamora committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    7ca1996 View commit details
    Browse the repository at this point in the history