Skip to content

Commit

Permalink
tracing: reorganize benchmarks for comparability (#2178)
Browse files Browse the repository at this point in the history
Currently, `tracing`'s benchmark suite benchmarks the same behaviors
(e.g. creating a span, recording an event, etc) across a handful of
cases: with no default dispatcher, with a global default, and with a
scoped (thread-local) default. We use criterion's `benchmark_group` to
represent each kind of dispatcher, and `bench_function` for each
behavior being measured.

This is actually kind of backwards relative to how Criterion is
_supposed_ to be used. `bench_function` is intended for comparing
different implementations of the *same* thing, with the
`benchmark_group` representing what's being compared. If we inverted the
structure of these benchmarks, Criterion would give us nicer plots that
would allow comparing the performance of each dispatch type on the same
task.

This PR reorganizes the benchmarks so that each behavior being tested
(such as entering a span or recording an event) is a `benchmark_group`,
and each dispatch type (none, global, or scoped) is a `bench_function`
within that group. Now, Criterion will generate plots for each group
comparing the performance of each dispatch type in that benchmark.

For example, we now get nice comparisons like this:
![image](https://user-images.githubusercontent.com/2796466/175659314-835664ac-a8cf-4b07-91ee-f8cfee77bfbb.png)

Unfortunately, this required splitting each benchmark type out into its
own file. This is because, once we set the global default dispatcher
within one benchmark group, it would remain set for the entire lifetime
of the process --- there would be no way to test something else with no
global default. But, I think this is fine, even though it means we now
have a bunch of tiny files: it also allows us to run an individual
benchmark against every combination of dispatch types, without having to
run unrelated benches. This is potentially useful if (for example)
someone is changing only the code for recording events, and not spans.
  • Loading branch information
hawkw committed Jun 24, 2022
1 parent 181bdbe commit ebea0e4
Show file tree
Hide file tree
Showing 14 changed files with 369 additions and 429 deletions.
30 changes: 27 additions & 3 deletions tracing/Cargo.toml
Expand Up @@ -68,15 +68,39 @@ attributes = ["tracing-attributes"]
valuable = ["tracing-core/valuable"]

[[bench]]
name = "subscriber"
name = "baseline"
harness = false

[[bench]]
name = "no_subscriber"
name = "dispatch_get_clone"
harness = false

[[bench]]
name = "global_subscriber"
name = "dispatch_get_ref"
harness = false

[[bench]]
name = "empty_span"
harness = false

[[bench]]
name = "enter_span"
harness = false

[[bench]]
name = "event"
harness = false

[[bench]]
name = "span_fields"
harness = false

[[bench]]
name = "span_no_fields"
harness = false

[[bench]]
name = "span_repeated"
harness = false

[badges]
Expand Down
24 changes: 24 additions & 0 deletions tracing/benches/baseline.rs
@@ -0,0 +1,24 @@
use criterion::{black_box, criterion_group, criterion_main, Criterion};

fn bench(c: &mut Criterion) {
use std::sync::atomic::{AtomicUsize, Ordering};

let mut group = c.benchmark_group("comparison");
group.bench_function("relaxed_load", |b| {
let foo = AtomicUsize::new(1);
b.iter(|| black_box(foo.load(Ordering::Relaxed)));
});
group.bench_function("acquire_load", |b| {
let foo = AtomicUsize::new(1);
b.iter(|| black_box(foo.load(Ordering::Acquire)))
});
group.bench_function("log", |b| {
b.iter(|| {
log::log!(log::Level::Info, "log");
})
});
group.finish();
}

criterion_group!(benches, bench);
criterion_main!(benches);
15 changes: 15 additions & 0 deletions tracing/benches/dispatch_get_clone.rs
@@ -0,0 +1,15 @@
use criterion::{black_box, criterion_group, criterion_main, Criterion};

mod shared;

fn bench(c: &mut Criterion) {
shared::for_all_dispatches(&mut c.benchmark_group("Dispatch::get_clone"), |b| {
b.iter(|| {
let current = tracing::dispatcher::get_default(|current| current.clone());
black_box(current);
})
});
}

criterion_group!(benches, bench);
criterion_main!(benches);
16 changes: 16 additions & 0 deletions tracing/benches/dispatch_get_ref.rs
@@ -0,0 +1,16 @@
use criterion::{black_box, criterion_group, criterion_main, Criterion};

mod shared;

fn bench(c: &mut Criterion) {
shared::for_all_dispatches(&mut c.benchmark_group("Dispatch::get_ref"), |b| {
b.iter(|| {
tracing::dispatcher::get_default(|current| {
black_box(&current);
})
})
});
}

criterion_group!(benches, bench);
criterion_main!(benches);
43 changes: 43 additions & 0 deletions tracing/benches/empty_span.rs
@@ -0,0 +1,43 @@
use criterion::{black_box, criterion_group, criterion_main, Criterion};

mod shared;

fn bench(c: &mut Criterion) {
let mut group = c.benchmark_group("empty_span");
shared::for_all_dispatches(&mut group, |b| {
b.iter(|| {
let span = tracing::span::Span::none();
black_box(&span);
})
});
group.bench_function("baseline_struct", |b| {
b.iter(|| {
let span = FakeEmptySpan::new();
black_box(&span);
})
});
}

struct FakeEmptySpan {
inner: Option<(usize, std::sync::Arc<()>)>,
meta: Option<&'static ()>,
}

impl FakeEmptySpan {
fn new() -> Self {
Self {
inner: None,
meta: None,
}
}
}

impl Drop for FakeEmptySpan {
fn drop(&mut self) {
black_box(&self.inner);
black_box(&self.meta);
}
}

criterion_group!(benches, bench);
criterion_main!(benches);
16 changes: 16 additions & 0 deletions tracing/benches/enter_span.rs
@@ -0,0 +1,16 @@
use criterion::{criterion_group, criterion_main, Criterion};
use tracing::{span, Level};

mod shared;

fn bench(c: &mut Criterion) {
shared::for_all_dispatches(&mut c.benchmark_group("enter_span"), |b| {
let span = span!(Level::TRACE, "span");
b.iter(|| {
let _span = span.enter();
})
});
}

criterion_group!(benches, bench);
criterion_main!(benches);
12 changes: 12 additions & 0 deletions tracing/benches/event.rs
@@ -0,0 +1,12 @@
use criterion::{criterion_group, criterion_main, Criterion};

mod shared;

fn bench(c: &mut Criterion) {
shared::for_all_recording(&mut c.benchmark_group("event"), |b| {
b.iter(|| tracing::info!("hello world!"))
});
}

criterion_group!(benches, bench);
criterion_main!(benches);
136 changes: 0 additions & 136 deletions tracing/benches/global_subscriber.rs

This file was deleted.

0 comments on commit ebea0e4

Please sign in to comment.