profiling with iter_batched_ref

[jmayclin]

Hello! I'm using criterion to profile TLS handshakes. Since the initial setup object is relatively expensive (loading certs, etc) I'm doing the setup using iter_batched_ref.

I want to generate flamegraphs for those benchmarks, and I want that to only include the actual benchmarking routine (the handshake), but it looks like the profiler is also capturing samples of the benchmark setup code (cert loading).

From looking at the profile function, I think that this is expected behavior?

1. is this expected behavior, or am I just missing some configuration?
2. would you be open to a PR changing this behavior?

If you're open to a PR, do you have any guidance? I haven't looked at it deeply, but my thinking is that you could have something like

while insufficient_samples {
    let inputs = batch_inputs();
    profiler.start_profiling();
    inputs.bench(...)
    profiler.stop_profiling();
}
// this method does not currently exist ...
profiler.done_profiling();

I'm thinking that the criterion::profiler::Profiler trait will need some sort of done_profiling() method to indicate when start and stop will no longer be called?

Alternately, we could do a single iteration of the profiling loop so that no changes are needed to the Profiler trait, but this would mean that some situations might have very low-fidelity flamegraphs due to the limited number of samples. But I would view low-fidelity & accurate as preferable to the current "inaccurate" ones?