…CPUS

(cherry picked from commit 37686f8)

Uses the existing env variable to restrict the number of threads used for parallel proving.

Cleaned up version of #81

(cherry picked from commit 0c1868d)

pick:
- Only the changes to bellman::multicore.
- Added lazy_static and rayon dependencies.

(cherry picked from commit dca4c5a)

pick: Only the changes to bellman::multicore.

(cherry picked from commit c55d5e4)

Changes during pick:
- Excluded GPU-related changes (not upstreamed yet).
- Added fix for change to Field::double API.
- Fixed no-multicore Worker impl.

Co-authored-by: Jack Grigg <thestr4d@gmail.com>

If a waiter is used from within a thread pool it might deadlock. Don't
do that.

(cherry picked from commit dc13606)

The Worker/Waiter model creates a Worker that requests that threads
be spawned to complete some work.  Since there is no bound to the
number of requests for work, it's possible to exhaust memory with
pending closures waiting to be processed.

This was not an issue when 'everything was bounded by a single thread
pool', however that setup has race conditions built into it.

We have recently spent time splitting that model out, and this should
be the final step required.

This code places a limit on the number of requests that can be queued
in an attempt to lessen the chance of memory exhaustion due to pending
requests.  It does this by placing a limit that is a simple multiple of
the number of cores available (and in testing is far below observed
crash criteria).  When the number of requests for spawning new threads
reaches that limit, instead of adding a new request, we block on execution
to help clear the backlog.

(cherry picked from commit 74947c0)

pick: Also includes log change from b481a34

(cherry picked from commit 007487f)

…dpool


This occurring is a programming error.

Co-authored-by: Daira Hopwood <daira@jacaranda.org>

Co-authored-by: Daira Hopwood <daira@jacaranda.org>

Co-authored-by: Daira Hopwood <daira@jacaranda.org>

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

… is expected to play more nicely

with other uses of rayon in the same process (avoiding excess parallelism).

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Co-authored-by: str4d <thestr4d@gmail.com>

…icore) implementation.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

Use global rayon threadpool for multicore