Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Shuffle-based groupby aggregation for high-cardinality groups #9302

Merged
merged 14 commits into from Aug 17, 2022
Merged

Shuffle-based groupby aggregation for high-cardinality groups #9302

merged 14 commits into from Aug 17, 2022

Commits on Jul 22, 2022

  1. copy experimental files into fresh branch

    @rjzamora
    rjzamora committed Jul 22, 2022
    Configuration menu
    Copy the full SHA
    9f2f01d View commit details
    Browse the repository at this point in the history

Commits on Jul 23, 2022

  1. use split_every to guide repartitioning

    @rjzamora
    rjzamora committed Jul 23, 2022
    Configuration menu
    Copy the full SHA
    a24d4d9 View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2022

  1. add math import

    @rjzamora
    rjzamora committed Jul 24, 2022
    Configuration menu
    Copy the full SHA
    3082e06 View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2022

  1. handle split_every None

    @rjzamora
    rjzamora committed Jul 25, 2022
    Configuration menu
    Copy the full SHA
    d6c697b View commit details
    Browse the repository at this point in the history

  2. avoid shuffle for split_out==1

    @rjzamora
    rjzamora committed Jul 25, 2022
    Configuration menu
    Copy the full SHA
    e6afb2c View commit details
    Browse the repository at this point in the history

Commits on Aug 15, 2022

  1. add test coverage for shuffle=True

    @rjzamora
    rjzamora committed Aug 15, 2022
    Configuration menu
    Copy the full SHA
    7965ac6 View commit details
    Browse the repository at this point in the history

  2. fix test_shuffle_aggregate

    @rjzamora
    rjzamora committed Aug 15, 2022
    Configuration menu
    Copy the full SHA
    9825cb1 View commit details
    Browse the repository at this point in the history

  3. strip shuffle-based groupby out of aca (and define stand-alone function)

    @rjzamora
    rjzamora committed Aug 15, 2022
    Configuration menu
    Copy the full SHA
    11089fc View commit details
    Browse the repository at this point in the history

  4. set shuffle default to 'tasks' for groupby aggregation

    @rjzamora
    rjzamora committed Aug 15, 2022
    Configuration menu
    Copy the full SHA
    786e76e View commit details
    Browse the repository at this point in the history

  5. remove extra sort arg

    @rjzamora
    rjzamora committed Aug 15, 2022
    Configuration menu
    Copy the full SHA
    937eef7 View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2022

  1. Merge remote-tracking branch 'upstream/main' into shuffle-groupby

    @rjzamora
    rjzamora committed Aug 16, 2022
    Configuration menu
    Copy the full SHA
    9772f49 View commit details
    Browse the repository at this point in the history

  2. add sort=True support for split_out>1 when shuffle=True

    @rjzamora
    rjzamora committed Aug 16, 2022
    Configuration menu
    Copy the full SHA
    24591f9 View commit details
    Browse the repository at this point in the history

  3. remove kwargs

    @rjzamora
    rjzamora committed Aug 16, 2022
    Configuration menu
    Copy the full SHA
    8f89208 View commit details
    Browse the repository at this point in the history

Commits on Aug 17, 2022

  1. add new docstring for aggs

    @rjzamora
    rjzamora committed Aug 17, 2022
    Configuration menu
    Copy the full SHA
    b4cf919 View commit details
    Browse the repository at this point in the history