Are PUT benchmarks in op-guide doing inserts on a single key or on a range of sequential keys?

[Ngalstyan4]

Hi all

I am confused about the PUT performance table in the performance section of etcd op guide

The table indicates that writes are done on different keys (There is a "Number of Keys" column). But the commands cited below the table write all values to the same key since key-space-size argument to the benchmark script defaults to 1.
Because the benchmark commands have '--sequential-keys' flag, I think the intention was to provide benchmark commands which would write to ${NUMBER_OF_INSERTS} different keys.

The commands below show that the provided PUT benchmark commands write to a single key

#template from the website: benchmark --endpoints=${HOST_1} --target-leader --conns=1 --clients=1 put --key-size=8 --sequential-keys --total=10000 --val-size=256
$./benchmark --endpoints=http://localhost:22379 --target-leader --conns=100 --clients=10     put --key-size=8 --sequential-keys --total=100000 --val-size=256

# after the position number the first 8 bytes are the key and the last two are two LFs
$../../bin/etcdctl --endpoints=http://localhost:22379 get ""  --prefix --keys-only| xxd
00000000: 0000 0000 0000 0000 0a0a                 ..........

# this one writes to 4 different keys
$./benchmark --endpoints=http://localhost:22379 --target-leader --conns=100 --clients=10     put --key-size=8 --sequential-keys --total=100000 --key-space-size 4  --val-size=256
$../../bin/etcdctl --endpoints=http://localhost:22379 get ""  --prefix --keys-only| xxd
00000000: 0000 0000 0000 0000 0a0a 0200 0000 0000  ................
00000010: 0000 0a0a 0400 0000 0000 0000 0a0a 0600  ................
00000020: 0000 0000 0000 0a0a                      ........

Were the benchmarks run key-space-size set appropriately or did all writes actually go to the same key?
At least in my setup the performance is not affected much by setting key-space-size appropriately. But still, would be good to know whether there is an issue in the table or the cited benchmark commands below.

[tjungblu]

I didn't check the history of the docs, but it might happen that the defaults/args have changed over time. I think that the benchmark results are also fairly outdated (etcd 3.2.0, there aren't even any n-4 instances on GCP anymore), so maybe it's not useful to compare against.

I think the benchmark CLI itself is not used widely, I've been benchmarking mostly with the rw-benchmark script [0] which exercises a bigger range of keys and clients.

[0] https://github.com/etcd-io/etcd/blob/main/tools/rw-heatmaps/rw-benchmark.sh

[serathius]

Feel free to file an issue proposing to update/remove outdated docs.