mypy unable to narrow type of tuple elements in case clause in pattern matching

[exoriente]

Bug Report

When using match on a tuple, mypy is unable to apply type narrowing when a case clause specifies the type of the elements of the tuple.

To Reproduce

Run this code through mypy:

class MyClass:
    def say_boo(self) -> None:
        print("Boo!")


def function_without_mypy_error(x: object) -> None:
    match x:
        case MyClass():
            x.say_boo()


def function_with_mypy_error(x: object, y: int) -> None:
    match x, y:
        case MyClass(), 3:
            x.say_boo()

Expected Behavior

mypy exists with Succes.

I would expect mypy to derive the type from the case clause in the second function, just like it does in the first function. Knowing the first element of the matched tuple is of type MyClass it should allow for a call to .say_boo().

Actual Behavior

mypy returns an error:

example.py:15: error: "object" has no attribute "say_boo"

Your Environment

I made a clean environment with no flags or ini files to test the behavior of the example code given.

• Mypy version used: mypy 0.940
• Mypy command-line flags: none
• Mypy configuration options from mypy.ini (and other config files): none
• Python version used: Python 3.10.2
• Operating system and version: macOS Monterrey 12.2.1

[tmke8]

Yeah, mypy doesn't narrow on tuples. I commented on it here: #12267 (comment) But apparently the worry is that it is too computationally expensive.

EDIT: hmm, maybe it's not the same problem actually

[cpbotha]

I just ran into this exact problem in a codebase I'm converting to mypy.

I worked around the issue (for now) by adding a superfluous assert isinstance(x, MyClass) just above x.say_boo().

[alexpovel]

I believe the issue I came across fits here. For example:

from datetime import date
from typing import Optional


def date_range(
    start: Optional[date],
    end: Optional[date],
    fmt: str,
    sep: str = "-",
    ongoing: str = "present",
) -> str:
    match (start, end):
        case (None, None):  # A
            return ""
        case (None, date() as end):  # B
            return f"{sep} {end.strftime(fmt)}"  # C
        case (date() as start, None):
            return f"{start.strftime(fmt)} {sep} {ongoing}"
        case (date() as start, date() as end):
            return f"{start.strftime(fmt)} {sep} {end.strftime(fmt)}"
        case _:
            raise TypeError(f"Invalid date range: {start} - {end}")


print(date_range(date(2020, 1, 1), date(2022, 10, 13), "%Y-%m-%d"))

(This not-quite-minimal working example should run as-is).

After line A, end cannot possibly be None. mypy understands guard clauses like if a is not None: ..., after which it knows to drop None from the type union of a. So I hoped it could deduce it in this case as well, but no dice. I can see that it might introduce a lot of complexity.

To circumvent, I cast to date explicitly in line B, such that line C doesn't yield a type checking attribute error anymore (error: Item "None" of "Optional[date]" has no attribute "strftime" [union-attr]). Note that date(end) didn't work (for mypy), it had to be date() as end.

While this works and line C is then deduced correctly, the resulting code is quite verbose. I suspect checking all combinations of start and end for None is verbose in any case (before match, this would have been a forest of conditionals), but wondered if this could be solved more succinctly.

Another suspicion I have is that structural pattern matching in Python is intended more for matching to literal cases, not so much for general type checking we'd ordinarily use isinstance and friends for (?). At least that's what it looks like from PEP 636.

---

EDIT: More context here:

• https://mypy-lang.blogspot.com/2022/03/mypy-0940-released.html
• https://mypy.readthedocs.io/en/stable/literal_types.html#exhaustiveness-checking with its assert_never

[arseniiv]

I have what appears the same issue (mypy 1.2, Python 3.11):

A value of type Token: = str | tuple[Literal['define'], str, str] | tuple[Literal['include'], str] | tuple[Literal['use'], str, int, int] is matched upon in this manner:

match token:
    case str(x):
        ...
    case 'define', def_var, def_val:
        reveal_type(token)
        ...
    case 'include', inc_path:
        reveal_type(token)
        ...
    case 'use', use_var, lineno, column:
        reveal_type(token)
        ...
    case _:
        reveal_type(token)
        assert_never(token)

In each case the type revealed is the full union, without only a str which is succesfully filtered out by the first case str(x), and then there’s an error in assert_never(token) because nothing had been narrowed.

Also def_var and other match variables are all typed as object and cause errors in their own stead—but that here at least can be fixed by forcefully annotating them like str(def_var) and so on.

Also for some reason I don’t see any errors from mypy in VS Code on this file, but I see them in droves when running it myself. 🤔

[loic-simon]

I encountered a similar issue with mapping unpacking, I suppose it fits here:

values: list[str] | dict[str, str] = ...

match values:
    case {"key": value}:
        reveal_type(value)

produces

Revealed type is "Union[Any, builtins.str]"

Interestingly, if I replace the first line by values: dict[str, str] or even values: str | dict[str, str], I got the expected

Revealed type is "builtins.str"

Looks like mixing list and dict causes an issue here?

[wrzian]

Ran into this with a two boolean unpacking

def returns_int(one: bool, two: bool) -> int:
    match one, two:
        case False, False: return 0
        case False, True: return 1
        case True, False: return 2
        case True, True: return 3

Don't make me use three if-elses and double the linecount 😢

[alenzo-arch]

I see a a number of others have provided minimal reproducible examples so I am including mine as is just for another reference point.

MyPy Version: 1.5.0
Error: Argument 1 to "assert_never" has incompatible type "tuple[float | None, float | None, float | None]"; expected "NoReturn"

from enum import StrEnum
from typing import assert_never

from pydantic import BaseModel


class InvalidExpectedComparatorException(Exception):
    def __init__(
        self,
        minimum: float | None,
        maximum: float | None,
        nominal: float | None,
        comparator: "ProcessStepStatus.MeasurementElement.ExpectedNumericElement.ComparatorOpts | None",
        detail: str = "",
    ) -> None:
        """Raised by `ProcessStepStatus.MeasurementElement.targets()` when the combination of an ExpectedNumericElement
        minimum, maximum, nominal, and comparator do not meet the requirements in IPC-2547 sec 4.5.9.
        """
        # pylint: disable=line-too-long
        self.minimum = minimum
        self.maximum = maximum
        self.nominal = nominal
        self.comparator = comparator
        self.detail = detail
        self.message = f"""Supplied nominal, min, and max do not meet IPC-2547 sec 4.5.9 spec for ExpectedNumeric.
detail: {detail}
nominal:  {nominal}
minimum: {minimum}
maximum: {maximum}
comparator: {comparator}
"""
        super().__init__(self.message)


class ExpectedNumericElement(BaseModel):
    """An expected numeric value, units and decade with minimum and maximum values
    that define the measurement tolerance window. Minimum and maximum limits shall be in the
    same units and decade as the nominal. When nominal, minimum and/or maximum attributes are
    present the minimum shall be the least, maximum shall be the greatest and the nominal shall
    fall between these values.
    """

    nominal: float | None = None
    units: str | None = None
    decade: float = 0.0  # optional in schema but says default to 0
    minimum: float | None = None
    maximum: float | None = None
    comparator: "ComparatorOpts | None" = None
    position: list[str] | None = None

    class ComparatorOpts(StrEnum):
        EQ = "EQ"
        NE = "NE"
        GT = "GT"
        LT = "LT"
        GE = "GE"
        LE = "LE"
        GTLT = "GTLT"
        GELE = "GELE"
        GTLE = "GTLE"
        GELT = "GELT"
        LTGT = "LTGT"
        LEGE = "LEGE"
        LTGE = "LTGE"
        LEGT = "LEGT"


expected = ExpectedNumericElement()

if expected.comparator is None:
    comp_types = ExpectedNumericElement.ComparatorOpts
    # If no comparator is expressed the following shall apply:
    match expected.minimum, expected.maximum, expected.nominal:
        case float(), float(), float():
            # If both limits are present then the default shall be GELE. (the nominal is optional).
            comparator = comp_types.GELE
        case float(), float(), None:
            # If both limits are present then the default shall be GELE. (the nominal is optional).
            comparator = comp_types.GELE
        case float(), None, float():
            # If only the lower limit is present then the default shall be GE.
            comparator = comp_types.GE
        case None, float(), float():
            # If only the upper limit is present then the default shall be LE.
            comparator = comp_types.LE
        case None, None, float():
            # If only the nominal is present then the default shall be EQ.
            comparator = comp_types.EQ
        case None as minimum, None as maximum, None as nom:
            raise InvalidExpectedComparatorException(
                minimum, maximum, nom, expected.comparator, "No minimum, maximum or, nominal present."
            )
        case float() as minimum, None as maximum, None as nom:
            raise InvalidExpectedComparatorException(
                minimum, maximum, nom, expected.comparator, "Minimum without maximum or nominal."
            )
        case None as minimum, float() as maximum, None as nom:
            raise InvalidExpectedComparatorException(
                minimum, maximum, nom, expected.comparator, "Maximum without minimum"
            )
        case _:
            # NOTE: Known issue with mypy here narrowing types for tuples so assert_never doesn't work
            # https://github.com/python/mypy/issues/14833
            # https://github.com/python/mypy/issues/12364
            assert_never((expected.minimum, expected.maximum, expected.nominal))

            # raise InvalidExpectedComparatorException(
            #    expected.minimum,
            #    expected.maximum,
            #    expected.nominal,
            #   expected.comparator,
            #    "Expected unreachable",
            # )

[Vinh-CHUC]

Out of curiosity is this planned to be addressed at some point? Is there a roadmap doc somewhere? This seems to be quite a fundamental issue that prevents using mypy in a codebase with pattern matching

[JelleZijlstra]

Mypy is a volunteer-driven project. There is no roadmap other than what individual contributors are interested in.

[loic-simon]

I tried to dig into this. If I got it right, there are several similar-but-distinct issues here:

1. exoriente & wrzian cases: mypy doesn't narrow individual items inside a sequence pattern case block. I drafted a patch doing this, it seems to work well, I'll open a PR soon 😄
2. alexpovel case: mypy doesn't narrow individual items after a non-match with sequence pattern(s). This looks more complex to achieve since the type of each item depend of already narrowed types for all other items 😢
3. arseniiv & my cases: mypy doesn't properly reduce union types when visiting a sequence pattern. I think we need to special-case them, I may try that next 😁

[loic-simon]

Just got bitten by this again, @JelleZijlstra do you think you could take a look at the linked PR, or ping anyone who could? Can I do something to help? 🙏