Add help_short field for Python tools and use it to generate the help text

[krishnan-chandra]

Closes #20912.

This actually wasn't too bad! I didn't realize the trick with classproperty would work as I intended, but it did. Looking at the individual commits should help separate out the repetitive work (changing help to help_short) from the more substantive changes.

[krishnan-chandra]

Sample output:

./pants help black
There is no pantsd metadata at /Users/krishnanchandra/Projects/pants/.pants.d/pids/b054a5b87b87/pantsd.
20:02:59.97 [INFO] Initializing scheduler...
20:03:33.31 [INFO] Scheduler initialized.

`black` subsystem options
-------------------------

The Black Python code formatter (https://black.readthedocs.io/).

This version of Pants uses `black` version 23.3.0 by default. Use a dedicated lockfile and the `install_from_resolve` option to control this.

[huonw]

Thanks for doing this separately.

I'd like to get additional eyes on this.

I think @benjyw is probably across this area the most.

[huonw]

I imagine subclasses of PythonToolBase are fairly common for people with their own custom plugins. A search like https://github.com/search?q=pants++PythonToolBase+-path%3A%2Fsrc%5C%2Fpython%5C%2Fpants%5C%2F%2F+-org%3Apantsbuild+language%3Apython&type=code indicates a bunch of ones in open source code.

In a perfect world, we wouldn't break those existing plugins with this change, i.e. so that people can upgrade easily without making changes. My theory is we don't break them, because existing classes will still override help, thus not use this class's version (and thus not read help_short)... but I'm not exactly sure how to verify.

Given I think it should be fine, one approach would be to land it and then see if people report problems.

[benjyw]

I concur that I think this won't break.

[huonw]

@benjyw: background: this comes out of #20901, where @krishnan-chandra added info about the default version of each tool but within the install_from_resolve help text. The desire and change here is to have the info even more obvious as part of the "intro".

The specific implementation approach comes out of the thread #20901 (comment) and in particular the list of options we discussed there:

1. keep this approach, with a new method on the Subsystem class which lets "leaf" subclasses just set help while still giving superclasses flexibility for manipulating it
2. a new field manipulated by the Python tool helpers, that Python tool subclasses need to set
3. a class method that Python tool subclasses can call when formatting their own help text
4. omit this information about the default version from the subsystem help string and have it just in places that don't cause the subclassing fiddle, i.e. option help text like install_from_resolve.
   1. Variant: omit it for now so we can land this PR with just the parsing & display in option text and no change to the top-level help text, and then consider how to display it elsewhere as follow-up

The original #20901:

• used option 1 first: added help_extended to Subsystem with a default implementation that deferred directly to help, then the PythonToolRequirementBase overrode it to append additional content to each subclass's help.
• was cut down to 4i to separate out this more controversial change and thus land Add tool version into install_from_resolve arg documentation #20901 sooner

Thus PR then tries option 2.

[benjyw]

Seems like a reasonable approach to me!

[benjyw]

@huonw I'm fine to merge this but will wait to give you a last say.

[huonw]

Really nice, thank you for iterating!