New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement parallel collect to array for non-Copy elements #817
Commits on May 17, 2020
-
-
FIX: Use Zip::fold_while for final reduction in parallel array view
When fold_with is used, use Zip::fold_while to fold the array view's parallel iterator. Note that in some cases, the IntoIterator of the view is used instead.
-
FIX: Make is_contiguous pub(crate)
The reason this method is not yet public, is that it's not accurate (false negatives) for less common layouts. It's correct for C/F i.e row/col major layouts.
-
-
FEAT: Factor out traits SplitAt and SplitPreference
To be used by Zip and parallel Zip
-
FEAT: Add ParalleIterator ParallelSplits
This iterator is for internal use; it produces the splits of a Zip (it splits the Zip the same way as the regular parallel iterator for Zip, but here the whole Zip is the produced item of the iterator.) This is helpful as a building block for other operations.
Commits on May 19, 2020
-
-
FEAT: Add internal Zip::last_producer
This method is useful for parallel Zip.
-
FEAT: Implement generic parallel collect
Allow non-copy elements by implementing dropping partial results from collect (needed if there is a panic with unwinding during the apply-collect process). It is implemented by: 1. allocate an uninit output array of the right size and layout 2. use parallelsplits to split the Zip into chunks processed in parallel 3. for each chunk keep track of the slice of written elements 4. each output chunk is contiguous due to the layout being picked to match the Zip's preferred layout 5. Use reduce to merge adjacent partial results; this ensures we drop all the rests correctly, if there is a panic in any thread
-
FIX: In ParallelSplits, count maximum number of splits
Instead of requiring to use the size in elements of the thing-to-split, simply use a counter for the number of splits.
-
-
-
Commits on May 23, 2020
-
FIX: Use Partial instead of PartialArray
Partial is just a contiguous slice, and much simpler than PartialArray; Partial is all that's needed, because the area written will always be contiguous.
-
FEAT: Combine common parts of apply_collect and par_apply_collect
Factor out the common part of the parallel and and regular apply_collect implementation; the non-parallel part came first and ended up more complicated originally. With the parallel version in place, both can use the same main operation which is implemented in the methods Zip::collect_with_partial.