Skip to content

Commit

Permalink
Elaborate on what AST changes Black might perform
Browse files Browse the repository at this point in the history
  • Loading branch information
ambv committed Apr 28, 2021
1 parent 682077c commit b39999d
Show file tree
Hide file tree
Showing 3 changed files with 37 additions and 7 deletions.
5 changes: 3 additions & 2 deletions CHANGES.md
Expand Up @@ -6,8 +6,9 @@

- Fix crash if the user configuration directory is inaccessible. (#2158)

- Clarify that _Black_ may change the AST, especially when cleaning up docstrings.
(#2159)
- Clarify
[circumstances](https://github.com/psf/black/blob/master/docs/the_black_code_style.md#pragmatism)
in which _Black_ may change the AST (#2159)

#### _Packaging_

Expand Down
10 changes: 5 additions & 5 deletions README.md
Expand Up @@ -237,11 +237,11 @@ is that **until the formatter becomes stable, you should expect some formatting
change in the future**. That being said, no drastic stylistic changes are planned,
mostly responses to bug reports.

Also, as a temporary safety measure, _Black_ will check that the reformatted code still
produces a valid AST that is mostly equivalent to the original. This slows it down. If
you're feeling confident, use `--fast`. In a few contexts, Black does make changes to
the AST: it cleans up whitespace in docstrings, adds or removes parentheses in some
`del` statements, and may move around type comments.
Also, as a safety measure which slows down processing, _Black_ will check that the
reformatted code still produces a valid AST that is effectively equivalent to the
original (see the
[Pragmatism](https://github.com/psf/black/blob/master/docs/the_black_code_style.md#pragmatism)
section for details). If you're feeling confident, use `--fast`.

## The _Black_ code style

Expand Down
29 changes: 29 additions & 0 deletions docs/the_black_code_style.md
Expand Up @@ -464,3 +464,32 @@ exception to this rule is r-strings. It turns out that the very popular
default by (among others) GitHub and Visual Studio Code, differentiates between
r-strings and R-strings. The former are syntax highlighted as regular expressions while
the latter are treated as true raw strings with no special semantics.

### AST before and after formatting

When run with `--safe`, _Black_ checks that the code before and after is semantically
equivalent. This check is done by comparing the AST of the source with the AST of the
target. There are three limited cases in which the AST does differ:

1. _Black_ cleans up leading and trailing whitespace of docstrings, re-indenting them if
needed. It's been one of the most popular user-reported features for the formatter to
fix whitespace issues with docstrings. While the result is technically an AST
difference, due to the various possibilities of forming docstrings, all realtime use
of docstrings that we're aware of sanitizes indentation and leading/trailing
whitespace anyway.

2. _Black_ manages optional parentheses for some statements. In the case of the `del`
statement, presence of wrapping parentheses or lack of thereof changes the resulting
AST but is semantically equivalent in the interpreter.

3. _Black_ might move comments around, which includes type comments. Those are part of
the AST as of Python 3.8. While the tool implements a number of special cases for
those comments, there is no guarantee they will remain where they were in the source.
Note that this doesn't change runtime behavior of the source code.

To put things in perspective, the code equivalence check is a feature of _Black_ which
other formatters don't implement at all. It is of crucial importance to us to ensure
code behaves the way it did before it got reformatted. We treat this as a feature and
there are no plans to relax this in the future. The exceptions enumerated above stem
from either user feedback or implementation details of the tool. In each case we made
due diligence to ensure that the AST divergence is of no practical consequence.

0 comments on commit b39999d

Please sign in to comment.