PEP604-style union with python<3.10 after __future__ import

[supersergiy]

Is your feature request related to a problem? Please describe.
We can't use typeguard + PEP604-style union with python <3.10. A good fraction of packages don't state explicit support for 3.10 yet, and having to choose between typeguard and PEP604 is very hard.

Example:

from __future__ import annotations

from typeguard import typechecked

@typechecked
def foo(a: int | str): ...

foo(1) # MyPy OK; TypeGuard: unsupported operand type(s) for |: 'type' and 'type'

Describe the solution you'd like
One way is to translate annotations to pre-PEP604 format before passing them to get_type_hints. Here's a quick Lark prototype:

from lark import Lark, Transformer

class UnionTransformer(Transformer):
    def typ(self, children):
        return "".join(children)

    def pep604_union(self, children):
        return "Union[" + ", ".join(children) + "]"

    def qualification(self, children):
        return "[" + ", ".join(children) + "]"

    def string(self, children):
        return children[0].value

    def name(self, children):
        return children[0].value

    def number(self, children):
        if len(children) == 2:  # minus sign
            return f"-{children[1].value}"
        else:
            return str(children[0].value)

HINT_PARSER = Lark(
"""
?hint: pep604_union | typ
pep604_union: typ ("|" typ)+

typ: name (qualification)? | qualification | number | string
qualification: "[" hint ("," hint)* "]"
number: (minus)? (DEC_NUMBER | HEX_NUMBER | BIN_NUMBER | OCT_NUMBER)
?minus: "-"

%import python.name
%import python.string
%import python.DEC_NUMBER
%import python.HEX_NUMBER
%import python.BIN_NUMBER
%import python.OCT_NUMBER
%import common.WS
%ignore WS
""",
    start="hint",
)

def translate_type_hint(hint: str) -> str:
    tree = HINT_PARSER.parse(hint)
    return UnionTransformer(tree).transform(tree)

hint = 'int | str | Union[Optional[      Literal["querty\\"\\\'\\""] ]| Literal[-1], None]'
print(translate_type_hint(hint)) # Union[int, str, Union[Union[Optional[Literal["querty\"\'\""]], Literal[-1]], None]]

Grammar:

?hint: pep604_union | typ  
pep604_union: typ ("|" typ)+

typ: name (qualification)? | qualification | number | string 
qualification: "[" hint ("," hint)* "]"
number: (minus)? (DEC_NUMBER | HEX_NUMBER | BIN_NUMBER | OCT_NUMBER)
?minus: "-"

%import python.name
%import python.string
%import python.DEC_NUMBER
%import python.HEX_NUMBER 
%import python.BIN_NUMBER 
%import python.OCT_NUMBER
%import common.WS
%ignore WS

[agronholm]

I was too hasty. You're talking about adding support for UnionType on Python < 3.10.

[agronholm]

That parser you wrote made this easy to implement. Thank you!

[agronholm]

Now I'm wondering: could we have done this with just AST manipulation, instead of involving third party dependencies? After all, the problem with PEP 604 is not incompatible syntax, but rather the lack of class-level __or__().

1. Compile expression into an AST
2. Modify AST to turn PEP 604 unions into Union
3. Compile AST to code
4. Evaluate code

[agronholm]

Here's a 20 min coodle (code-doodle):

from ast import (
    BinOp, Index, Load, Name, NodeTransformer, Subscript, Tuple, fix_missing_locations,
    parse)
from typing import Any, Dict, FrozenSet, List, Set, Union


class UnionTransformer(NodeTransformer):
    def visit_BinOp(self, node: BinOp) -> Any:
        self.generic_visit(node)
        return Subscript(
            value=Name(id='Union', ctx=Load()),
            slice=Index(
                Tuple(elts=[node.left, node.right], ctx=Load()),
                ctx=Load()
            ),
            ctx=Load()
        )


type_mappings = {"dict": Dict, "list": List, "tuple": Tuple, "set": Set, "frozenset": FrozenSet}
parsed = parse("str | int | dict[str, int]", "<string>", "eval")
UnionTransformer().visit(parsed)
fix_missing_locations(parsed)
code = compile(parsed, "<string>", "eval")
print(eval(code, globals(), type_mappings))

Result when run:

typing.Union[str, int, typing.Dict[str, int]]

I don't think your parser translates the built-in collection types to their Typing equivalents, but this will (with just a two line change, no less!).

[agronholm]

@supersergiy Would you mind looking at #291?