int is not a Number?

[tjltjl]

n : Number = 5
produces

test_compiler.py:18: error: Incompatible types in assignment (expression has type "int", variable has type "Number")

Which it probably shouldn't because isinstance(n, Number) == True

[tjltjl]

(Talking about numbers.Number, naturally)

[ilevkivskyi]

Thank you for reporting this!

It looks like this is mostly a typeshed issue. Structursl subtyping (see PR #3132) will help, but it looks like still some changes will be necessary in typeshed, since we can't declare Number a protocol (it is too trivial currently).

@JelleZijlstra what do you think?

[gvanrossum]

Not sure we need to keep this open, if it can be fixed in typeshed once protocols land. (Maybe there should be an omnibus issue for things we can change once it lands?)

Also see PEP 484, which has this to say:

PEP 3141 defines Python's numeric tower, and the stdlib module numbers implements the corresponding ABCs ( Number , Complex , Real , Rational and Integral ). There are some issues with these ABCs, but the built-in concrete numeric classes complex , float and int are ubiquitous (especially the latter two :-).

Rather than requiring that users write import numbers and then use numbers.Float etc., this PEP proposes a straightforward shortcut that is almost as effective: when an argument is annotated as having type float , an argument of type int is acceptable; similar, for an argument annotated as having type complex , arguments of type float or int are acceptable. This does not handle classes implementing the corresponding ABCs or the fractions.Fraction class, but we believe those use cases are exceedingly rare.

[JelleZijlstra]

I am not too familiar with the numeric tower, but maybe we should just go with PEP 484's recommendation and avoid the numbers ABCs.

numbers.Number in particular doesn't seem very useful since it declares no members, so logically mypy shouldn't allow you to do anything with a variable declared as a Number. It looks like the others can be made into Protocols easily when that lands.

What are concrete use cases where one would use the numbers ABCs instead of int/float/complex? Maybe numpy numeric types?

[gvanrossum]

What are concrete use cases where one would use the numbers ABCs instead of int/float/complex? Maybe numpy numeric types?

It's come up a few times before, the only places I found that were using it were just trying to be hyper-correct. E.g. numpy supports it but barely mentions it in its docs.

[Warning: Luddite mode on]
IMO PEP 3141 was a flight of fancy and has never produced much useful effect. Pythoneers are generally to close to practice to care about the issue, most of the time the code just works through the magic of duck typing, and when it doesn't people typically say "well that's too fancy anyways". :-)

[JukkaL]

I'm not sure if the other ABCs beyond Number are very straightforward either. It would be an interesting exercise for somebody to write a prototype stub (using the protocol PR) that only supports one operation (such as addition) for Complex, Real and friends. It would have to work with int, float and user-defined classes -- and probably also with mixtures of those when it makes sense.

However, I don't believe that the numeric tower is going to be very useful for type checking. The current types seem to be good enough for the vast majority of use cases, and just providing another set of types for essentially the same purpose seems like unnecessary complexity. In my opinion, a much more promising (and somewhat related) project would be figuring out how to type check code using numpy.

[ilevkivskyi]

In my opinion, a much more promising (and somewhat related) project would be figuring out how to type check code using numpy.

I agree this looks very interesting. This would require some simple dependent types, since fixed size arrays/matrices are everywhere in mypy. The rules for those are non-trivial, e.g. matrix[n, m] * matrix[m, k] == matrix[n, k]. But I think fixed size arrays will be quite useful on its own even outside numpy.

[tjltjl]

It would be really cool at least to document it a bit more clearly than the PEP 484 quote, which I read as

"You don't need to use the numbers.* classes but you can"

when it appears that the real situation is

"The numbers.* classes do not work with typing".

I (quite reasonably?) thought that isinstance(x, cls) implies that you can have the

 x : cls

type declaration.

Are there other places where this is not the case?

[tjltjl]

(or even just a warning in mypy if one of the numbers.* types is used?)

[thanatos]

Just noting that I seem to have a use-case affected by this; I'm adding annotations for use w/ mypy to some code that takes latitude/longitude values; so,

def foo(lat, lng):
    # type: (?, ?) -> None

I started with float, but passing an integer is okay. That led me to Union[float, int], and subsequently #2128; however, Decimal is also okay — ideally, I just work in whatever types get input to the function and let duck typing do its thing, but Union[float, int, Decimal…] is not what I really want to say. So, numbers.Real, and that brings one to this bug.

(I agree with tjltjl's reasoning that if the specification is x: Cls, then passing a foo, where isinstance(foo, Cls), should work.)

[ilevkivskyi]

numbers.Real doesn't require any special actions, it will be fixed automatically when protocols PR lands (unlike numbers.Number that also might require some changes in typeshed).

[Zac-HD]

@ilevkivskyi, has that PR landed? Because I have just encountered the same problem with Mypy 0.590:

def bug(n: Real = 1) -> Real:
    return n + 1
# error: Incompatible default for argument "n" (default has type "int", argument has type "Real")

Motivating example is HypothesisWorks/hypothesis#200; we have several test strategies for some type of number within a range, and the bounding values can be specified as int/float/fraction - so it would be really nice to show Real instead of that union!

[ilevkivskyi]

Yes, but those types are still not labeled as protocols. I personally don't see any harm in labelling them as protocols. Note that the other type is Any for all methods, but I don't think this will introduce any additional unsafety as compared to current situation. The problem however is that Number can't be made a protocol (because it is empty), but it is a base class for all other numbers and protocols can't inherit from non-protocols, so at least for now they stay as ABCs, i.e. this will need a .register() support, see #2922

[pkch]

One consideration around the numeric tower is that when a function returns numpy.float, it would be nice to mark it as returning numbers.Real, since

(1) I don't want to force people to fix type annotations because they happened to return a float in the future
(2) You could see both numpy.float and python float returned from the same function for convenience.

[ilevkivskyi]

Raised priority to high since this is constantly coming.

[gvanrossum]

What solution do you propose?

[ilevkivskyi]

One possible option is to add a bit of lie to typeshed, see python/typeshed#3108.

[JukkaL]

Another option is to reject the use of Number etc. as types (and maybe give a link to documentation that explains why this isn't supported and what to do instead).

[ilevkivskyi]

Another option is to reject the use of Number etc. as types (and maybe give a link to documentation that explains why this isn't supported and what to do instead).

This would be a bit sad, especially taking into account this is a 12-th most liked issue on the tracker. TBH I forgot the details of the discussion, but if the typeshed attempt will work out are there any reasons not to do this?

[JukkaL]

but if the typeshed attempt will work out are there any reasons not to do this?

I'm wondering if the types will be useful for their intended purpose. If not, the effort may be wasted.

[posita]

@mdickinson, I attempted to reproduce (and respond) to your post on discuss.python.org. Apologies if imposing that venue change is disruptive. No disrespect is intended.

[uhasker]

Since the discussion regarding the number tower is now at discuss.python.org I would suggest that within this thread we continue to talk about a possible mypy hotfix only.

My suggestion would be (as outlined above) to have a warning that the abstract numeric base classes shouldn't be used for typing. This would mean that whenever the user attempts to use Number, Real etc, a warning along the lines of "Using abstract numeric base classes for typing is (currently) a bad idea, see e.g. issue #3186" (probably with some nicer wording) is displayed. The warning should also probably contain suggestions for workarounds (like using SupportsInt etc).

[NeilGirdhar]

his would mean that whenever the user attempts to use Number, Real etc, a warning along the lines of "Using abstract numeric base classes for typing is (currently) a bad idea,

But it's not always a bad idea. It's fine to use it for single dispatch (as a type annotation that guides the dispatch), for example. Do we want to trade some false positives for other false positives?

The ideal thing would be to detect the error that int isn't matching Integral, for example. However, that might be harder than just implementing #2922 so that it does.

[cfcohen]

I commented on this ticket (or one of its many duplicates) a couple of years ago. I've been trying to cobble together a library that interoperates between a number of domains (e.g. floats, Fraction, Decimal, numpy, gmp, sympy, etc.). Because the interfaces for these domains are so inconsistent, I desperately need the assistance of mypy type checking to alert me to differences in the APIs of the various libraries. e.g. Sympy doesn't support trunc(). I don't really see how to get there practically without the ability to place related implementations in a group with a name like numbers.Real. I could have invented my own, but that didn't completely solve the problem either because I need some cooperation from the language on int, float, etc. And that primarily came from the poorly supported PEP3141.

Sensing that the commitment to supporting PEP3141 was weak at best, about two years ago I produced my own mypy patch, with the expectation that I would muddle through until this issue was resolved in some more principled way. Two years later, it's less than clear that a consensus solution will ever emerge...

So, here's what I did: 09d8367

I don't claim that this fix is correct for any definition, just that it works for me, and I think it might be helpful for others. I'm open to commentary on why it's the wrong fix or discussion about why this kind of a solution shouldn't be adopted more broadly, which will probably be rooted in some reasonable argument about half-solutions. I also welcome feedback of the form "It would be better if it did this instead." The reason why I haven't proposed any serious solutions to the problem is exactly because I know how thorny the problem really is, and how inadequate my knowledge is to truly solve it. That said, practical problems need practical solutions, and as hacky as this patch is, it was the solution that worked for me. At least for a while. I'm less thrilled about maintaining it forever...

[oscarbenjamin]

e.g. Sympy doesn't support trunc()

As far as I can tell trunc works with SymPy:

>>> import sympy
>>> e = sympy.sqrt(2)
>>> e
sqrt(2)
>>> import math
>>> math.trunc(e)
1

Please open a SymPy issue if I've misunderstood (the thread here is long enough that it doesn't need any further discussion on this point).

[theRealProHacker]

If I may add the discussion.

1. Solution Protocols

As already stated, using Number as a protocol doesn't really make sense right now, because it doesn't tell you what a Number should be doing (it doesn't have any members). One solution would be to change that and add some common functions like

• __add__ and __sub__
• __mul__ and __div__
• __neg__
• __abs__
• __int__, __float__, __complex__
  That would make anything a number that defines these. However, there is a problem with this solution alone. Take this example

class MutableFloat:
    value: float
    def __init__(self, value):
        self.set(value)
    def set(self, value):
        self.value = float(value)
    forbidden = {
        "__new__", "__init__", "__getattribute__", "__getattr__", "__setattr__", "__delattr__",
        "__dir__", "__class__", "__init_subclass__", "__subclasshook__"
    }
    for name in float().__dir__():
        if name in forbidden or not name.startswith("__"): continue
        func = getattr(float, name)
        if not callable(func): continue
        change = "if type(x) is float: self.value=x" if name.startswith("__i") and name != "__int__" else ""
        s = f"""
def {name}(self, *args, **kwargs):
    args = [arg.value if type(arg) is MutableFloat else arg for arg in args]
    x = float.{name}(self.value, *args, **kwargs)
    {change}
    return x
        """.strip()
        exec(s)

It isn't beautiful, but it works (I haven't tested it completely) and there are actual use cases. It's basically a float proxy, so it is a Number, but the type checker wouldn't know because I didn't want to write down all the possible methods and used the power of python to make my life easier. This is probably a bad example, but what I am trying to bring close to you is that there needs to be a way to distinguish a number, not only with a protocol.

I think the real problem is that in the documentation it says that all numbers inherit from Number, which is wrong. So I suggest we should fix that.
And I totally agree with @tjltjl

I (quite reasonably?) thought that isinstance(x, cls) implies that you can have the

 x : cls

The above implication not holding true is IMO not acceptable for a proper language. And I feel like there is not enough will to solve this issue if this is already 5 years old. I know this sounds harsh, but I just want to express how absurd this is.

Are there other places where this is not the case?

I would love this question to be answered.

2. Solution: User defines what numbers mean to them

Several comments noted that what a number really is depends on the person and the use case. Should a Vector (complex numbers are basically just 2-Vectors) be a number? Can floats be considered real?
The last question for example could be answered with no: If you'd pick a random real number, you would never pick a number that can be represented as a float
Or it could be answered with yes: floats are a good enough simulation of the real numbers and I don't really care if my result is wrong in the 9th digit because I am just using Python for education and I don't want to explain to my students why floats aren't actually real.

In conclusion, we could just say everyone defines for themselves what a number is. This could also just be temporary solution

Here, I want to show a simple solution for most projects:

1. Make a file called my_types.py (or whatever name that doesn't collide with other things)
2. Write this into the file

Number = int, float, Decimal # and whatever else you like
NumberType = int | float | Decimal # I know it feels weird to write the same thing twice just with different delimiters, but this might just be the life of a python developer for some time. 

# 3. Now you can import Number and NumberType in another file and use them together
def test_Numbers(x: None|NumberType)->NumberType:
    if isinstance(x, Number):
        reveal_type(x)
        y: NumberType = x
    else:
        y = 0
    return y

To make this process easier, Python could support two canonical function that convert a Union into a tuple, and the reverse and are understood by type checkers. Another similar possibility would be allowing Union types as parameter for isinstance. And as I see this is being developed but isn't working on my machine with Python 3.10.1 yet.
This intertwining of static and dynamic type checking is a feature that is sensible, practical, easy to understand and requested.
For example here

Note: The revealed type of x is Union[builtins.float, _decimal.Decimal]. Probably because you could just write x: float = 0. So int|float is reduced to float.
Note 2: Looking at the above note, it feels ridiculous why ´x: float = 0is correct butx: Number = 0` is not.

3. Solution: Special case in type checking

The comment above actually shows us how to do it. Just like float also accepts int and complex allows both float and int, the Types Number, Real, ... should just allow the types where isinstance(type(),Type) is True. I would suggest this as an immediate hotfix. This should be pretty easy and would fix the problem.

4. Solution: Nominal Subtyping

The easiest and maybe just the best way to do it is good old nominal subtyping. Just like before, it is decided on a case to case basis what a number is. However, not by the end user, but by the library makers (The Python language just gives guidelines on what a Number should provide)
Right now, there is an incoherence between the theoretical state of the language and the practical state. Theoretically, int should be a subtype of Number by the definition of Number in theory/in the documentation. From my understanding, isinstance knows from int.__instancecheck__ that int is a subtype of Number even though it isn't actually: int.__mro__ == (<class 'int'>, <class 'object'>).

Making theory and practice coherent again would mean to make int actually subclass Number. Numpy could then for example just subclass from Numbers too. If a user has a special use-case where the library definition of a Number isn't good for him, he can still just use solution 2 and make his own number class.

Summary

Hotfix

• Add special cases to mypy so that the fundamental typing rules aren't violated anymore
• Developers using numbers like the issue creator (x: Number = 0) should be provided with the information that they can just use x: float = 0. This will probably solve most further issues raised from inexperienced developers that might for example come from JavaScript, where Numbers is just one type that mangles ints and floats together.
• Change the documentation of the numbers module

Additional Notes to Prototyping

What shouldn't happen is that we get types like UnsignedNumber or IndivisableNumber and small splitters of Number types that are difficult to put in a simple tree structure.
An Example where extreme subtyping for example is extremely annoying (it might seem off-topic, but it's just an example):

# I define this here. 
style_mapping = Mapping[str, Style] # Style could be anything

# somewhere totally different
d1: style_mapping = {
  "example": Style()
}
# somewhere different again, maybe in a function
d2: style_mapping = {
  "example": Style()
}
d3 = ChainMap(d1, d2)
# error because ChainMap wants to have a MutableMapping

I will never edit anything in this ChainMap. It's just for viewing the dicts without copying them.
There would be different solutions to this problem (eg I could make an ImmutableChainMap). Some might even seem simple, but they all involve me writing more code just because the ChainMap only takes a MutableMapping, when really it could also just take an immutable mapping with no problem and just raise an error when it gets changed.

I would really love some feedback on these ideas. I can also take some hits 😁.
However, I believe that it would make sense to decide on one primary idea because
There should be one– and preferably only one –obvious way to do it.

[NeilGirdhar]

@theRealProHacker It's also been pointed out in a few places that the number 3 solution here: #2922 (comment) would solve the Numbers problem. This would require no special cases. You would just register the appropriate numerical types and they would work.

Solution 1: protocols

I feel like the reason people want to use protocols is because protocols get around the registration problem. The protocol is automatically bound as a parent class. But the issue is that protocols are really inferior to explicit registration. You'd have to be so careful when designing your class to match the protocol, and the protocol could never change or else everyone counting on the automatic registration would suddenly find that their numerical class isn't a number anymore. It's a bad solution.

Solution 2

This is what I do now, but it's not a good solution compared with just fixing the registration.

3. Solution: Special case in type checking

Would be fine as a hotfix. I still think the registration fix would be the best long term solution.

Solution 4: int is a subtype of Number even though it isn't actually:

int is registered as a subtype, which is good enough. Unfortunately, the registration isn't visible to MyPy. But it is visible at runtime.

Making theory and practice coherent again would mean to make int actually subclass Number.

I don't think that's necessary (or even desirable). Registration is just as good as inheritance.

[cfcohen]

I agree that ABCMeta.register() support #2922 is a better solution than the one I used, and I had assumed that at some point in the future it would be the fix that actually resolved this issue. I just couldn't figure out how to make that change myself. If I've correctly understood the solutions proposed in that issue, approach three would provide a practically workable solution without introducing performance problems or creating complex multi-pass algorithms. Such an approach would be required to support my other extensions to Numbers (see cfcohen@eee71ca).

As for the comment on sympy, it wasn't meant to be a bug report -- it was just an example of the most recent problem that mypy had helped me find. While math.trunc() works for number-like values, it doesn't work for symbols, producing TypeError: can't truncate symbols and expressions and from the sympy stubs I'm using: error: Argument 1 to "trunc" has incompatible type "Basic"; expected "SupportsTrunc". It would have been convenient for me if Trunc() was an operator like Add() tor Pow() in sympy. While this sympy behavior isn't that surprising in retrospect, it's easy to get confused by the observation above where math.trunc() does work for some sympy values when combined with the mypy type Union[int, float, fractions.Fraction, decimal.Decimal, Numbers.Rational, sympy.Basic] to miss the detail about trunc() not being supported on symbols. I commented on this despite the length of this thread because I think it helps elucidate the value of having a more robust definition of what methods are required on which classes in the Numbers tower. I was surprised for example, that there seems to be no requirement that Integers are Rationals are ordered. :-( Perhaps this aspect of the discussion belongs in the Python thread about what to do about PEP3141.

[theRealProHacker]

I experimented a bit and tried to build a plugin as a hotfix. I would suggest that either this or 09d8367 should be used as a hotfix. Additionally, I would still suggest that if someone uses anything from numbers he is linked to https://peps.python.org/pep-0484/#the-numeric-tower. Something like:

Note: Instead of using the numbers module check out peps.python.org/pep-0484/#the-numeric-tower

I think that would be a huge and simple improvement right now. 😁

from mypy.plugin import Plugin as _Plugin, ClassDefContext, TypeInfo
import numbers
from numbers import __all__
from decimal import Decimal
import os

tdata: dict[str, tuple[type, int]] = {
    f"numbers.{cls}":(getattr(numbers, cls), -1-i) for i,cls in enumerate(__all__)
}

bdata: dict[str, type] = { # these are all the builtin numbers I am aware of
    "builtins.int": int,
    "builtins.float": float,
    "builtins.complex": complex,
    "_decimal.Decimal": Decimal
}

bstore: dict[str, TypeInfo] = {}
tstore: dict[str, TypeInfo] = {}

_key = "number-plugin-done"
def is_done(info: TypeInfo):
    rv = _key in info.metadata
    info.metadata[_key] = {}
    return rv

abspath = os.path.abspath(__file__)
prep = "import sys\n"

class Plugin(_Plugin):
    refreshed = False
    def get_customize_class_mro_hook(self, name: str):
        # refreshing to trick mypys caching. That however leads to inefficiency.
        # So this should be integrated into mypy in such a way that it runs every 
        # time without forcing everything else to restart
        if not self.refreshed:
            self.refreshed = True
            with open(abspath) as file:
                s = file.read()
            with open(abspath, "w") as file:
                file.write(s.removeprefix(prep) if s.startswith(prep) \
                    else prep + s)
        # handling Numbers
        if name.startswith("numbers."):
            tcls, index = tdata[name]
            def tinner(ctx: ClassDefContext):
                tinfo = ctx.cls.info
                if is_done(tinfo): return
                tstore[name] = tinfo
                for bname, binfo in bstore.items():
                    if not issubclass(bdata[bname], tcls): break
                    binfo.mro.insert(index, tinfo)
            return tinner
        # handling numbers
        if not name in bdata: return
        def binner(ctx: ClassDefContext):
            binfo = ctx.cls.info
            if is_done(binfo): return
            bstore[name] = binfo
            bcls = bdata[name]
            for tname, tinfo in tstore.items():
                tcls, index = tdata[tname]
                if not issubclass(bcls, tcls): continue
                binfo.mro.insert(index, tinfo)
        return binner


def plugin(version: str):
    # TODO:
    # if version >= fix_version: return _Plugin
    return Plugin

Additionally, a small test file that should run without errors.

from numbers import Number, Complex
from decimal import Decimal

class OwnNumber(int): pass

x: Number = 0

y: list[Number] = [
    1, 1.0, 1+0j, Decimal(1.0), OwnNumber(1.0)
]

def main(x: Number|str)->str:
    if isinstance(x,Number):
        return str(x).replace(".",",")
    else:
        return x.strip()

def add_numbers(x: Number, y: Number):
    # theoretically numbers have no attributes you can rely on but practically 
    # we know that they will probably be interoperable with other numbers
    # this ignore is required because of the theoretic aspect. 
    # We need to settle on one thing that all numbers should have in common. 
    # I would suggest that they must have a possibility to convert to a `complex`, `float` *or* `int` 
    # and thats it. Everything else is optional because it can be deduced.
    return x + y # type: ignore 

def add_numbers_(x: Complex, y: Complex)->Complex:
    # this works although the revealed type of x+y is Any
    # here I would suggest that type.__add__(anything) should always return type
    # for example adding a Complex to a Real should result in a Complex
    return x+y

[tjltjl]

As the original reporter of the bug, I wholeheartedly agree on the suggestion "if anyone uses anything from numbers, ...". This would have fixed the original problem right away.

[hauntsaninja]

As of #15137, mypy will now issue a much more helpful diagnostic when this comes up.

There is no plan to make naive use of numbers.Number type check:

1. no major python static type checker supports ABC registration, see e.g. ABCMeta.register support #2922
2. you'd need to lie a bunch to make things work because there are incompatibilities in the classes see e.g. int is not a Number? #3186 (comment)
3. the end result would still be very unsound and so the value proposition is dicey, see e.g. int is not a Number? #3186 (comment)

If you have a use case, I recommend using your own Protocol to describe exactly what behaviour you expect.

If for some reason a Protocol cannot do what you want it to do, please open an issue at https://github.com/python/typing

If you're interested in designing some future of interoperability of numeric types or discussing the future of the numbers module, this thread https://discuss.python.org/t/numeric-generics-where-do-we-go-from-pep-3141-and-present-day-mypy/17155/12 is a good place to start.

Since there are no clear action items for mypy that don't have better more specific issues tracking them, I'm going to take the bold step of closing this issue.

This is a long issue with a lot of history. If you feel the need to post something, please read this thread through first.