Black thinks that a file containing a syntax error is well formatted

[inglesp]

Describe the bug

Black thinks that a file containing a syntax error is well formatted.

To Reproduce

With a file containing just this code:

{
    f"{",
}

Black does not give an error:

$ black bad.py 
All done! ✨ 🍰 ✨
1 file left unchanged.

But there is a syntax error in the file:

$ python file.py 
  File "/home/inglesp/file.py", line 2
    f"{",
        ^
SyntaxError: f-string: expecting '}'

Expected behavior

I would expect Black to complain!

Environment

• Black's version: 21.10b0
• OS and Python version: Linux/Python 3.1.10

Additional context

Black does complain as expected with the following similar files:

{
    f"{"
}

{
f"{",
}

It also complains as expected if there is an extra newline at the end of the file, but that doesn't render in GHFM.

[isidentical]

On the cases where black reformatted the source code (in the 2 additional examples), it complains due to the underlying ast.parse() call (when used with --safe, which is implied by default). So if there is nothing to reformat, then it simply doesn't ever try to parse some of the extra stuff (e.g f-strings, which are ignored by the lib2to3) so it never notices this.

Not a black maintainer but I think this is the appropriate case since this is simply not black's job. If you use pre-commit, then you can use this action to ensure everything is parse-able (check-ast).

[inglesp]

As an outsider to the project, this didn't make much sense:

So if there is nothing to reformat, then it simply doesn't ever try to parse some of the extra stuff (e.g f-strings, which are ignored by the lib2to3) so it never notices this.

But having dug in, I think I understand better.

Black's algorithm is:

• Parse input string with lib2to3
• Produce output string by reformatting according to Black's rules
• In safe mode, if the input string and output string are different, check that they parse to the same AST

lib2to3 doesn't handle f-strings, and so it is the AST parsing that finds the SyntaxError.

In this case, my example is unchanged after reformatting, and so the input and output strings are the same, so Black doesn't need to check that they parse to the same AST, and Black thinks the file is correctly formatted.

Is that correct?

I don't know whether it's reasonable to expect Black to detect syntax errors -- but I do think it's surprising that it sometimes does and sometimes doesn't.

[felix-hilden]

That's fair, and it does seem a bit surprising. But we only promise to format correct Python.

[isidentical]

@JelleZijlstra I guess this can be closed after #2645

[JelleZijlstra]

Thanks, agree that #2645 addresses this. (For this specific case, we should catch this error in the future if we end up formatting the expression bits of f-strings.)