Error trying to format line with parameter assignment and generator

[tabrezm]

Describe the bug

Black throws an error trying to format files containing a parameter assignment with a generator. This is valid Python code.

error: cannot format util.py: Cannot parse: 200:43:     if any(match := pattern_error.match(s) for s in buffer):
Oh no! 💥 💔 💥
1 file failed to reformat.

To Reproduce

Code sample:

    if any(match := pattern_error.match(s) for s in buffer):
        if match.group(2) == data_not_available:
            # Error OK to ignore.
            pass

Expected behavior

Black formats the file without errors. It seems to handle parameter assignment elsewhere just fine:

            if not (next_line := next(buffer_iter, None)):
                # Buffer is incomplete, so we return early and wait to be
                # reinvoked with a complete buffer.
                return status

Environment (please complete the following information):

• Version: main
• OS and Python version: Mac/Python 3.9.7

Does this bug also happen on main?

Yes

Additional context

None

[vbarbaresi]

I confirm this syntax seems valid in 3.9 (it wasn't in Python 3.8.6)

Though it looks like this syntax is not supported by the current CPython parser.
I looked into CPython parser unit tests:
https://github.com/python/cpython/blob/v3.9.10/Lib/lib2to3/tests/test_parser.py#L654-L655
The assignment in any() is wrapped in parentheses and it doesn't pass the tests if you remove the parentheses.
Maybe this should be fixed in CPython and/or black's lib2to3 fork?

By the way the documentation about the usage of parentheses in assignment expression is still a work in progress python/cpython#23291

To workaround the issue: wrap the assignment (match := pattern_error.match(s)) in parentheses, or define the generator expression in a variable.

[betafcc]

up for 2022-10-09 on black 22.10.0:

from typing import TypeVar, AsyncIterable

A = TypeVar("A")

async def enumerate(fa: AsyncIterable[A]) -> AsyncIterable[tuple[int, A]]:
    i = -1
    async for a in fa:
        yield i := i + 1, a

Result:

> black --version
black, 22.10.0 (compiled: yes)
Python (CPython) 3.10.6

> black dirty_bits/helpers/aitertools.py
error: cannot format dirty_bits/helpers/aitertools.py: Cannot parse: 165:15:         yield i:= i + 1, a

Oh no! 💥 💔 💥
1 file failed to reformat.

The parens workaround in #1898 does work:

async def enumerate(fa: AsyncIterable[A]) -> AsyncIterable[tuple[int, A]]:
    i = -1
    async for a in fa:
        yield (i := i + 1), a

[ichard26]

@isidentical would you be interested in taking a look at this issue?

[isidentical]

@isidentical would you be interested in taking a look at this issue?

Yep, will look into it! Thanks for CC'ing me.

[isidentical]

@betafcc, the example you have shared already fails with the standard Python parser since the named expression has to be wrapped with parens in that case. But the bug still persists for generator expressions passed as call arguments (any(x := y for _ in _)), so hopefully #3327 should fix it.