Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement parallel collect to array for non-Copy elements #817

Merged
merged 15 commits into from May 28, 2020
Merged

Commits on May 17, 2020

  1. Copy the full SHA
    ad856bd View commit details
    Browse the repository at this point in the history
  2. FIX: Use Zip::fold_while for final reduction in parallel array view

    When fold_with is used, use Zip::fold_while to fold the array view's
    parallel iterator. Note that in some cases, the IntoIterator of
    the view is used instead.
    bluss committed May 17, 2020
    Copy the full SHA
    511f8b2 View commit details
    Browse the repository at this point in the history
  3. FIX: Make is_contiguous pub(crate)

    The reason this method is not yet public, is that it's not accurate
    (false negatives) for less common layouts. It's correct for C/F i.e
    row/col major layouts.
    bluss committed May 17, 2020
    Copy the full SHA
    192a166 View commit details
    Browse the repository at this point in the history
  4. Copy the full SHA
    35e89f8 View commit details
    Browse the repository at this point in the history
  5. FEAT: Factor out traits SplitAt and SplitPreference

    To be used by Zip and parallel Zip
    bluss committed May 17, 2020
    Copy the full SHA
    84295c4 View commit details
    Browse the repository at this point in the history
  6. FEAT: Add ParalleIterator ParallelSplits

    This iterator is for internal use; it produces the splits of a Zip
    (it splits the Zip the same way as the regular parallel iterator for
    Zip, but here the whole Zip is the produced item of the iterator.)
    
    This is helpful as a building block for other operations.
    bluss committed May 17, 2020
    Copy the full SHA
    3ab67eb View commit details
    Browse the repository at this point in the history

Commits on May 19, 2020

  1. Copy the full SHA
    ee97dbf View commit details
    Browse the repository at this point in the history
  2. FEAT: Add internal Zip::last_producer

    This method is useful for parallel Zip.
    bluss committed May 19, 2020
    Copy the full SHA
    fe2ebf6 View commit details
    Browse the repository at this point in the history
  3. FEAT: Implement generic parallel collect

    Allow non-copy elements by implementing dropping partial results from
    collect (needed if there is a panic with unwinding during the
    apply-collect process).
    
    It is implemented by:
    
    1. allocate an uninit output array of the right size and layout
    2. use parallelsplits to split the Zip into chunks processed in parallel
    3. for each chunk keep track of the slice of written elements
    4. each output chunk is contiguous due to the layout being picked to
    match the Zip's preferred layout
    5. Use reduce to merge adjacent partial results; this ensures
    we drop all the rests correctly, if there is a panic in any thread
    bluss committed May 19, 2020
    Copy the full SHA
    4277296 View commit details
    Browse the repository at this point in the history
  4. FIX: In ParallelSplits, count maximum number of splits

    Instead of requiring to use the size in elements of the thing-to-split,
    simply use a counter for the number of splits.
    bluss committed May 19, 2020
    Copy the full SHA
    6d43c13 View commit details
    Browse the repository at this point in the history
  5. Copy the full SHA
    e3ebf8c View commit details
    Browse the repository at this point in the history
  6. Copy the full SHA
    efcd607 View commit details
    Browse the repository at this point in the history
  7. Copy the full SHA
    8ed9ac3 View commit details
    Browse the repository at this point in the history

Commits on May 23, 2020

  1. FIX: Use Partial instead of PartialArray

    Partial is just a contiguous slice, and much simpler than PartialArray;
    Partial is all that's needed, because the area written will always be
    contiguous.
    bluss committed May 23, 2020
    Copy the full SHA
    d02b757 View commit details
    Browse the repository at this point in the history
  2. FEAT: Combine common parts of apply_collect and par_apply_collect

    Factor out the common part of the parallel and and regular apply_collect
    implementation; the non-parallel part came first and ended up more
    complicated originally. With the parallel version in place, both
    can use the same main operation which is implemented in the methods
    Zip::collect_with_partial.
    bluss committed May 23, 2020
    Copy the full SHA
    e472612 View commit details
    Browse the repository at this point in the history