Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optional parallelization #1098

Open
robertvazan opened this issue Oct 30, 2023 · 0 comments
Open

Optional parallelization #1098

robertvazan opened this issue Oct 30, 2023 · 0 comments

Comments

@robertvazan
Copy link

My code needs to be able to run both sequentially and in parallel. Currently I have to implement separate sequential and parallel branches like this:

let converted = if parallelize {
    templates
        .par_iter()
        .map(|&t| TemplateEngine::instance().convert(t))
        .collect()
} else {
    templates
        .iter()
        .map(|&t| TemplateEngine::instance().convert(t))
        .collect()
};

I can let the calling code specify custom ThreadPool (or even better install it around the call), but this at best allows reducing thread count to one. I need to be able to run the code in current thread in case there's thread-local data that needs to be accessed. Having a way to locally disable parallelization would also help with #591.

Ideally, I would like my code to be either sequential by default or using global thread pool by default and let callers override this by installing their own thread pool or disabling parallelization completely.

There's rayon-cond that lets me avoid the duplication, but I still have to expose parallelization flag in the API, which is inconsistent with having ThreadPool specified via thread-local install.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant