INTERNAL ERROR when fmt:off and on do not match

[ppwwyyxx]

Howdy! Sorry you're having trouble. To expedite your experience,
provide some basics for me:

Operating system:
Python version: 3.7
Black version: black, version 19.3b0
Does also happen on master: yes

class A():
    def __init__(self, cfg):
        # fmt: off
        b = 1
        # fmt: off

class B():
    def __init__(self, cfg):
        # fmt: off
        a = 1
        # fmt: on
        assert a == cfg

When running locally, I observe:

error: cannot format a.py: INTERNAL ERROR: Black produced invalid code: expected an indented block (<unknown>, line 10). Please report a bug on https://github.com/ambv/black/issues.  This invalid output might be helpful: /tmp/blk_9w1d83ne.log
All done! 💥 💔 💥
1 file failed to reformat.

when running online with master, I observe wrong indentation in the output:

[mcsitter]

This still happens. I do not know exactly how ast parsing works, but maybe i can add some info so this can be resolved. The error message is also the same.

The invalid output as shown in the log that gets linked in the error message includes a traceback and may be more than shown above:

The _ast27.parse function is implemented in CPython and lies in the _ast27.cpython file in the typed_ast site-package. I do not know whether the code in the log is coming from black or from typed_ast. I have trouble looking into its code base. I found however this comment in typed_ast: Python 3.8.

I reproduced the issue with the current pipfile defined on master, which is on Python version 3.8, and on 3.7 by running it in a environment created by pipenv install --python 3.7 --dev. Both create the same error message and log file.

The same error message appears on this example as well:

def foo():
    # fmt: off
    pass
bar = 1

In the log is no traceback however, just some output i do not understand. Instead of assigning 1 to bar you can also insert a function or class definition or any other assignment. I do now know whether this is still in the scope of the reported bug. Both appear to be issues on parsing the code.

Here is what i think happens: The parts with # fmt: off do not get passed to the ast parsing and this can mess things up. Maybe one could make sure that # fmt: on is inserted at the end of function and class definitions since adding # fmt: on or '# fmt: off' or combinations of those into a following class or function definition, does not solve the issue.

If anyone can help out how to tackle this i would greatly appreciate it. Even finding out whether it is actually a problem with black or the ast parser would be nice! It seems to be maintained quite regularly.

[ichard26]

Duplicate of #569.