fetch/unpack layers concurrently

[cgwalters]

Today we fetch layers serially, which is unnecessarily slow. I think we should start by fetching and unpacking at least 2 (possibly 3) layers in parallel.

This would crucially allow interleaving the fetching of each layer with the storage process.

[cgwalters]

Looked briefly at this, we probably want containers/containers-image-proxy-rs#31 to enable concurrency on the proxy. That said, it'd also work OK to wrap the proxy in a Mutex I think.

[cgwalters]

Came across https://docs.rs/futures/0.3.24/futures/stream/trait.StreamExt.html#method.buffer_unordered to help ensure we only fetch e.g. max of 2 at a time.

[jdoss]

We came across this issue last week. We are layering decently sized container on top of FCOS and in development it layers quickly but on various cloud providers it is painfully slow. We are still digging into why the provisioning times are so vastly different but we think that this might be the issue.

Any chance of concurrency for layers getting added soon?