Making the default tool set lighter - soliciting comments

[ndokos]

There have been repeated requests to e.g. not include perf or pidstat in the default tool set. These tools produce voluminous output and it's not clear that they are useful to most people. See issues #1679, #1714 and #2331.

The trouble, as usual, is backward compatibility: if we get rid of them in the default tool set, we risk the possibility of people performing long runs and then finding out that the data they expected to be present, are not.

But we are at the bullet-biting point now and we have PR #2390 that proposes leaving such tools out of the default tool set.

This discussion has three purposes:

• to describe what the PR implements with the hope that the description, which will be widely disseminated, will prevent unpleasant surprises;

• to describe what the user can do to restore the status quo or otherwise define their preferred tool set;

• to solicit comments about the proposed PR.

---

The PR proposes three predefined tool sets: light, medium and heavy in addition to the legacy-tool-set (the legacy value of the default tool set). It then redefines the default tool set to be medium.

The proposed light tool set consists of just vmstat, the rationale being that it provides some CPU, memory and IO stats that might be enough for basic runs.

The proposed medium tool set consists of what is in the light set and in addition, some tools for networking stats and some more detailed stats in the other areas. It consists of vmstat, iostat and sar. As mentioned above, this would be the default tool set.

The proposed heavy tool set consists of what is in the medium tool set and in addition, mpstat, perf, pidstat, proc-interrupts, proc-vmstat and turbostat.

Finally, the legacy tool set consists of what is in the default tool set currently: iostat, mpstat, pidstat, proc-interrupts, proc-vmstat, sar, turbostat and perf.

We would appreciate comments and discussion on the above proposal.

---

What follows is a description of some ways to revert to the status quo or do something slightly different if you don't like the default.

If you register a tool set without specifying it further (i.e. pbench-register-tool-set without argument or with only --remote arguments), you would get the new default tool set.

You can use the --toolset argument of pbench-register-tool-set to register a different set, e.g. pbench-register-tool-set --toolset=heavy ..., or perhaps the legacy set: pbench-register-tool-set --toolset=legacy .... If you script pbench invocations, that is probably the easiest thing to do.

You might instead edit the agent config file and redefine the default tool set to be something else, but we don't recommend that: you'd have to remember to do that every time you upgrade. See below for a (slightly) better option.

If you want to define your own custom tool set as the default, you can forego pbench-register-tool-set and use pbench-register-tool to register each tool individually.

Another option that would allow the use of pbench-register-tool-set is to add another config file on top of the existing ones:

• create a file, e.g. /home/me/.config/pbench/pbench-agent.cfg that looks like this:

[DEFAULT]
my-tool-set = sar, mpstat, perf

[pbench/tools]
default-tool-set = %(my-tool-set)s

[config]
path = /opt/pbench-agent/config
files = pbench-agent.cfg, pbench-agent-default.cfg

• make sure that the environment variable _PBENCH_AGENT_CONFIG points to the file above: export _PBENCH_AGENT_CONFIG=/home/me/.config/pbench/pbench-agent.cfg by modifying /etc/profile.d/pbench-agent.sh - but note that with this method too, you will have to remember to do this step after every upgrade.

To check what e.g. the default-tool-set includes, say:

pbench-config default-tool-set pbench/tools

In the 0.69 (or earlier) pbench-agent, this should read:

getconf.py default-tool-set pbench/tools