Black produced invalid code: invalid syntax with --experimental-string-processing

[Horstage]

I hope this helps to make the experimental string processing a bit less experimental :-)

Environment

Python 3.8.6
black, version 20.8b1
Separate venv (via pipx) where only black and its' dependencies are installed

Executed Command

black myfile.py --experimental-string-processing

myfile.py

print(f"Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam. {[f'{i}' for i in range(10)]}")

Error Mesage

black myfile.py --experimental-string-processing
error: cannot format myfile.py: INTERNAL ERROR: Black produced invalid code: invalid syntax (<unknown>, line 77). Please report a bug on https://github.com/psf/black/issues.  This invalid output might be helpful: /tmp/blk_tqh1ju35.log
Oh no! 💥 💔 💥
1 file failed to reformat.

Process finished with exit code 123

Log file

  File "[..]/python3.8/site-packages/black/__init__.py", line 6141, in assert_equivalent
    dst_ast = parse_ast(dst)
  File "[..]/python3.8/site-packages/black/__init__.py", line 6052, in parse_ast
    return ast27.parse(src)
  File "[..]/python3.8/site-packages/typed_ast/ast27.py", line 50, in parse
    return _ast27.parse(source, filename, mode)
print(
    f"Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam {[f'{i}'"
    " for i in range(10)]}"
)

[JelleZijlstra]

Looks like we should make it so it doesn't try to break the string in the middle of an f-string expression part. cc @bbugyi200.

[bbugyi200]

@JelleZijlstra Thanks for tagging me on this.

I did design #1132 with f-expressions in mind (i.e. I added logic to prevent splitting on f-expressions), but the existing logic apparently does not handle f-expressions which contain other f-expressions well.

This bug should be fixed with #1809.