The First Annual Meeting of the Extraordinary League of Runtime Typers

[leycec]

Here! Here! The dishonourable @leycec presiding. You may now be thinking: "This is a scam, right?", "Who are you and why are you rooting around in my inbox?", "You annoy me, peasant.", "@beartype is bad and you should feel bad!", "Oh, Gods. That guy.", and "I already dislike you on principle."

These are reasonable thoughts. Nonetheless! Please, monocled gentle-people. There are matters afoot. They affect all members of the runtime type-checking community. This means you. Of course, runtime type-checking is mad-cap, pell-mell, and hazardous to mental health and physical well-being at the best of times. There are only a few of us. We're not legion. Nobody even knows about us. Okay, everybody knows about Pydantic. But the rest of us are like typing ninjas. Nobody expects... oh forget it. We're all tired of that line. Sadly, you are about to become even more tired.

Only three of us author general-purpose runtime type-checkers. Only a handful more author special-purpose runtime type-checkers. You have been called here tonight, because you are one of these authors.

Please, Gods. Say Something Interesting Already.

Python 3.13 is about to break forward compatibility with your type-checker. Python 3.13 – the next stable release of Python – is slated for release on October 1st of this year. Python 3.13 will unconditionally enable an old "feature" previously only enabled through arcane syntax so annoying that basically nobody used it, except PyTorch. PyTorch gotta walk its own way. That feature is baaaaaack. What's back? More on that in a hot minute.

The more important first question is: "Who is affected by the breakage?" After all, if it's not you, none of this matters. You can just go back to sleep and lapse into the sugar coma you came from. Sadly, it's you.

Unaffected: Most Packages, But Not Yours

Most Python packages are unaffected. Most means 99.999999999%. I just made that number up, but it's probably true. This is the good news. Python 3.13 preserves compatibility for most use cases and users.

Sadly, yours is not one of them. This is the bad news.

Most Python packages only annotate things with type hints. They don't actually consume, introspect, modify, or otherwise handle type hints as actual runtime objects. Thankfully, these use cases and users are fine. These packages should continue doing exactly what they are doing. Whatever they're doing is fine and will continue to be fine across the Python 3.13 threshold.

Consider @felixchenier at kineticstoolkit/kineticstoolkit#198, who wonders what exactly this means for end users. I hesitated in responding before, because I genuinely didn't know. I was woefully dumb. Also, this topic is nuanced and terrifying. But then I read and implemented support for the PEPs we shall introduce shortly. Now, I pretend to know. @felixchenier and other concerned citizens, your answer is:

"Nuffin'. Absolutely nuffin'. You don't gotta do nuffin'. Laziness prevails."

This is a big nothing-burger for almost everybody. But what about the rest of us? ...heh.

Affected: Hardly Any Packages, But You Wrote One of Them

Affected Python packages include those either directly accessing the __annotations__ dunder dictionary or directly calling the standard inspect.get_annotations() or typing.get_type_hints() functions. Because I have summoned you, your package is one of these. Specifically:

• @agronholm, I see some 4 references to __annotations__ across 2 files in typeguard and another 3 references to typing.get_type_hints() across 1 file.
• @cosmicBboy, I have no idea who principally maintains pandera – but you've committed a lot recently. So, you're it. Congrats. I see some 8 references to __annotations__ across 4 files in pandera and another 9 references to typing.get_type_hints() across 4 files. pandera is really exposed to this issue.
• @patrick-kidger, I see some 3 references to __annotations__ across 1 file in jaxtyping and 2 references to typing.get_type_hints() across 1 file. Excellent showing!
• @samuelcolvin, I see some 30-odd references to __annotations__ across 17 files in @pydantic alone and another 4 references to __annotations__ across 3 files in `pydantic-core. @pydantic is extremely exposed to this issue.
• @wesselb, I see some 3 references to __annotations__ across 1 file in Plum and another 2 references to typing.get_type_hints() across 1 file. Excellent showing!

You definitely did the best out of everybody, @patrick-kidger and @wesselb. Your pain is minimal. Still, there is pain.

I mostly didn't bother grepping for inspect.get_annotations() calls. Almost nobody accesses annotations indirectly that way. Why bother, right? Python 3.13 says: "Hi! You should've bothered. I'm about to break you."

Uhh.... I Still Don't Get It. What Was This About Again?

Right. Right. Getting there. So, we're agreed that we're all about to suffer. Everyone is in this together, which certainly makes me feel fuzzy and/or warm. Communal suffering. Nothing quite like it.

Python 3.13 enables a new annotation-specific feature called "bare forward references." "What's that? Sounds horrid," you're thinking. A "bare forward reference" is an unquoted reference to a type that has yet to be declared. The most common forward reference is a self-reference to the class currently being declared. Examples include:

@your_runtime_checker  # <-- this is your decorator that is about to break
def muh_func(muh_arg: MuhClass | int | str) -> str:  # <-- "MuhClass" is the bare forward reference here
    return repr(muh_arg)

@your_runtime_checker  # <-- your decorator again; still about to break
def MuhClass(object):
    def muh_method(self, muh_arg: MuhClass | int | str) -> str:  # <-- "MuhClass" is again the bare forward reference here
        return repr(muh_arg)

Both references to MuhClass above are bare forward references. In both contexts, the MuhClass class has yet to be declared in its lexical scope. Of course, in the latter context, the MuhClass class is technically available to @your_runtime_checker – but that doesn't particularly matter, because @your_runtime_checker has no say in the matter. "Wut!? The heck it don't!", you are now thinking. Bear with me a moment, please. When resolving annotations, lexical scope is the only thing that matters. Sadly, your decorator is irrelevant there.

Allow me to now explicate. Python 3.13's support for bare forward references requires fundamentally new technology unavailable in prior Python versions. To support bare forward references, your runtime type-checker must be manually refactored by you to add this support. This support does not come for free. If you ignore this issue and do nothing, your runtime type-checker will fail to support bare forward references. "Who cares? So what? I'm a tough dude who wears shades at night," you are now thinking.

Sadly, everybody wants bare forward references. Everybody hates PEP 484-style quoted forward references, PEP 585-style typing.ForwardRef()-encapsulated forward references, and the PEP 673-style typing.Self singleton. Everybody just wants to reference classes in type hints regardless of whether those classes have been declared yet. This is the grimdark reality we now find ourselves swimming in.

Let's up the grimdark. If you do nothing, your runtime type-checker will raise one unreadable exception for each type hint containing one or more bare forward references. The unreadability of the exception depends on whether the problematic type hints in question occur at global or local scope. Notably:

• For each global type hint containing one or more bare forward references, your runtime type-checker will raise an unreadable NameError exception resembling:

  NameError: name 'SomeUndefinedClass' is not defined

• For each local type hint containing one or more bare forward references, your runtime type-checker will raise an unreadable UnboundLocalError exception resembling:

  # Either this...
  UnboundLocalError: cannot access local variable 'SomeUndefinedClass' where it is not associated with a value
  
  # ...or this.
  UnboundLocalError: cannot access free variable 'SomeUndefinedClass' where it is not associated with a value in enclosing scope.

In all three cases, note that the exception message fails to emit the name of the module, callable, or class annotated by that problematic type hint. Obviously, the traceback contains that context – but consider the intersection of logging, web and mobile apps, and your runtime type-checker. That context may be stripped, truncated, or otherwise missing by the time your devoted userbase finally heaves a collective sigh and submits an issue to your tracker.

In other words, you absolutely must do something. If you ignore these prophetic words I am intoning in a deep baritone, destruction and mayhem shall be thy constant companions. Woe! Woe unto thy codebase.

I'm Becoming Concerned By What I'm Hearing. Also, You're Creeping Me Out.

...right? Thankfully, all is not lost. The codebase you save from Python 3.13 might be your very own.

You have three choices. To quote somebody who was famous but is now dead:

Vetinari: “There is always a choice."

Lipwig: "You mean I could choose certain death?"

Vetinari: "A choice nevertheless – or perhaps an alternative. You see, I believe in freedom."

Choice 1: Do It Yourself, Because Ain't Nobody Else Gonna

Choice 1 is the easy way. The easy way preserves independence. You don't need to add any additional optional or mandatory dependencies to your package. Instead, you "just" need to:

1. Define a new private utility function in your package for memoizing (i.e., caching expensive) function calls. If you don't already have a memoization decorator, ^{...you prolly do, but just sayin'} you have two sub-choices here:
   • Steal @beartype's. Stealing @beartype's stuff is generally the best possible thing you can do, usually. I'm permissive and jovial. Steal our stuff and release it under your own license with or without attribution. That's totally fine. Seriously. Specifically, steal our @callable_cached decorator. As for as I know, @callable_cached is the fastest possible general-purpose memoization decorator in pure Python. For efficiency, it prohibits keyword arguments. On the other hand, it even memoizes exceptions – which does cost a bit, but is basically mandatory for sane memoization in the general case.
   • Use Python's. The standard @functools.cache decorator under Python ≥ 3.12 and @functools.lru_cache(maxsize=None) decorator under older Python versions. Both are considerably slower than @callable_cached and fail to memoize exceptions – both of which sorta defeat the point of memoization. But... hey. You do you. Both of these decorators are certainly fine for casual use in a first-draft implementation. It's like when your significant other casually says "...fine!" and then sighs loudly. 😮‍💨
2. Define a new private utility function in your package for safely accessing type hints. Let us call it... oh, I don't know, gimme_dose_hints(). Yes. You do need to define a new private utility function in your package for reasons that will become clear shortly. No, it doesn't need to be called gimme_dose_hints(). It probably shouldn't be, in fact.
3. Globally refactor every direct access of the __annotations__ dunder dictionary or direct call to the standard inspect.get_annotations() or typing.get_type_hints() functions into a call to your gimme_dose_hints() function.

The implementation vaguely resembles (and I am wildly waving my hands here):

from collections.abc import Callable
from sys import version_info

# If the active Python interpreter targets Python >= 3.13...
if version_info >= (3, 13):
    # Defer version-specific imports.
    from inspect import (
        FORWARDREF,  # <-- *ONLY DEFINED UNDER PYTHON >= 3.13, YO*
        get_annotations,
    )

    # Memoization is basically mandatory under Python >= 3.13 when retrieving
    # annotations dictionaries possibly containing one or more bare forward
    # references. Tedious and tiresome explanation follows below.
    @memoize_me
    def gimme_dose_hints(obj: type | Callable) -> dict[str, object]:
        '''
        PEP 649-compliant utility function safely retrieving the dictionary of
        type hints annotating the passed type or callable in a safe manner
        portably supporting bare forward references.
        '''

        return get_annotations(obj, format=FORWARDREF)  # <-- *KEYWORD-ONLY PARAMETER, YO*
else:
    # Note the lack of memoization, which is unneeded for older Python versions.
    def gimme_dose_hints(obj: type | Callable) -> dict[str, object]:

        return obj.__annotations__  # <-- still fine for older Python versions
        #return get_annotations(obj)  # <-- also fine, but note *NO* "format" parameter

"I hate Choice 1," you were almost about to think. But you stopped when you realized that Choices 2 and 3 were probably a lot worse. They are. So, what is going on here? Why is memoization required?

What is going on here is PEP 649. PEP 649 deprecates PEP 563 (i.e., from __future__ import annotations) by globally and unconditionally enabling a completely different internal mechanism called "annotation scopes" that semantically achieves a similar purpose. For unknown reasons that likely reduce to "cause it was easy," the implementors of PEP 649 chose to implement the portion of PEP 649 handling bare forward references in pure Python.

Even that wouldn't have been so bad – except that this pure-Python implementation is outrageously, incredibly, and excrutiatingly slow. Why? Because this pure-Python implementation is doing insane things you do not want to think about:

In a nutshell, this technique involves running a Python-compiler-generated __annotate__ function in an exotic runtime environment. Its normal globals dict is replaced with what’s called a “fake globals” dict. A “fake globals” dict is a dict with one important difference: every time you “get” a key from it that isn’t mapped, it creates, caches, and returns a new value for that key (as per the __missing__ callback for a dictionary). That value is a an instance of a novel type referred to as a “stringizer”.

A “stringizer” is a Python class with highly unusual behavior. Every stringizer is initialized with its “value”, initially the name of the missing key in the “fake globals” dict. The stringizer then implements every Python “dunder” method used to implement operators, and the value returned by that method is a new stringizer whose value is a text representation of that operation.

When these stringizers are used in expressions, the result of the expression is a new stringizer whose name textually represents that expression...

Bringing it all together: if we run a Python-generated __annotate__() function, but we replace its globals with a “fake globals” dict, all undefined symbols it references will be replaced with stringizer proxy objects representing those symbols, and any operations performed on those proxies will in turn result in proxies representing that expression. This allows __annotate__ to complete, and to return an annotations dict, with stringizer instances standing in for names and entire expressions that could not have otherwise been evaluated.

In practice, the “stringizer” functionality will be implemented in the ForwardRef object currently defined in the typing module. ForwardRef will be extended to implement all stringizer functionality; it will also be extended to support evaluating the string it contains, to produce the real value (assuming all symbols referenced are defined). This means the ForwardRef object will retain references to the appropriate “globals”, “locals”, and even “closure” information needed to evaluate the expression.

Your face should now be contorting into a rictus grin. Note that the "exotic runtime environment" mentioned above is a dynamically synthesized data structure with non-trivial space and time complexity that the inspect.get_annotations(obj, format=FORWARDREF) function internally creates and then discards each time you call it. This worst-case complexity happens each time you pass an object containing one or more bare forward references to your gimme_dose_hints() function.

If I had to estimate the worst-case time complexity of your unmemoized gimme_dose_hints() function under Python ≥ 3.13, I'd ballpark it at O(∞!∞). That is, infinity factorial infinity. Kinda big number, huh? Don't be that guy. Memoize the bad PEP away.

I wouldn't bother with an LRU cache, either. Just go unbounded. The amount of space consumed by type hints and the total number of type hints across an app is very small by compare to the remainder of the app – especially for those of us in data science and web apps.

Choice 2: How Could Things Get Any Worse

To introduce Choice 2, I now need to discuss the failings of Choice 1. Actually, there's only one failing – but you're not gonna like it. You're really not gonna like it. Let's back up to our initial example of bare forward references. Remember this? @leycec remembers this:

@your_runtime_checker  # <-- this is your decorator that is about to break
def muh_func(muh_arg: MuhClass | int | str) -> str:  # <-- "MuhClass" is the bare forward reference here
    return repr(muh_arg)

@your_runtime_checker  # <-- your decorator again; still about to break
def MuhClass(object):
    def muh_method(self, muh_arg: MuhClass | int | str) -> str:  # <-- "MuhClass" is the bare forward reference here
        return repr(muh_arg)

If you did something, that now works under Python ≥ 3.13. Congrats. I salute you before passing out.

But... "What about Python ≤ 3.12"? I can hear your quavering voice already. It sounds like my own. The answer, of course, is that the above code raises SyntaxError exceptions in Python ≤ 3.12. But users want that code to work reliably across all Python versions! So... "What do users do?"

You already know the answer to this, don't you? Somewhere deep inside, you know. The truth is squirming. It's creeping unbidden across your brain like that intestinal worm you got after visiting the Dominican Republic last summer that just won't go away.

The answer is PEP 563. Ironically, although PEP 649 deprecates PEP 563, PEP 649 also makes PEP 563 much more popular by legitimizing what PEP 563 was trying but failed to do a decade ago. This means that everybody is about to add the following to every single module in their codebase:

from __future__ import annotations  # <-- ...heh. Now everything "works" under all Python versions. \o/

@your_runtime_checker  # <-- this is your decorator that is about to break
def muh_func(muh_arg: MuhClass | int | str) -> str:  # <-- "MuhClass" is the bare forward reference here
    return repr(muh_arg)

@your_runtime_checker  # <-- your decorator again; still about to break
def MuhClass(object):
    def muh_method(self, muh_arg: MuhClass | int | str) -> str:  # <-- "MuhClass" is the bare forward reference here
        return repr(muh_arg)

Do you see what just happened there? Everybody just enabled PEP 563 via from __future__ import annotations. In other words, not only is PEP 563 not going away for the foreseeable future, but PEP 563 just got a huge testosterone-enhancing shot in the arm and is back for one more swing with your codebase. PEP 563 is now legitimate. It's here to stay. It's going to crop up everywhere.

You are now thinking: "...fine. This is fine. I... I can deal with this. I can cope. I'm hugging a teddy bear right now. I'll just tell everybody not to do that. Hah! Dumb users lose again."

Yeah. That's not gonna happen. Well, that might happen. Let's be honest. But those dumb users might just decide that your package is even dumber. They might just decide to switch to a different runtime type-checker that actually does support PEP 563 rather than abandon bare forward references – which, as we all know by now, everybody wants. PEP 563 can no longer be ignored, because PEP 563 is effectively being globally enabled.

Can you do something about that? You can. But you're not going to like it. You're going to hate it. And then you're going to try to punch me repeatedly in my GitHub avatar.

Remember that gimme_dose_hints() function we defined above? What if we augmented that function to transparently support PEP 563 under Python ≤ 3.12? You should now be thinking: "Impossible, bro. Tried that. Couldn't do it. Nobody can. Can't be done. Gave up. Went home. Did something productive with our lives... like you should be doing right now."

Behold! The impossible made possible:

from collections.abc import Callable
from sys import version_info

# If the active Python interpreter targets Python >= 3.13...
if version_info >= (3, 13):
    # Defer version-specific imports.
    from inspect import (
        FORWARDREF,  # <-- *ONLY DEFINED UNDER PYTHON >= 3.13, YO*
        get_annotations,
    )

    # Memoization is basically mandatory under Python >= 3.13 when retrieving
    # annotations dictionaries possibly containing one or more bare forward
    # references. Tedious and tiresome explanation follows below.
    @memoize_me
    def gimme_dose_hints(obj: type | Callable) -> dict[str, object]:
        '''
        PEP 649-compliant utility function safely retrieving the dictionary of
        type hints annotating the passed type or callable in a safe manner
        portably supporting bare forward references.
        '''

        return get_annotations(obj, format=FORWARDREF)  # <-- *KEYWORD-ONLY PARAMETER, YO*
else:
    # Attempt to import PEP 563 resolution functionality from an optional dependency.
    # This dependency is almost certainly installed by something else in your app's
    # dependency stack. It's likely to be available in many Python environments.
    # Pretend this is less painful than it actually is.
    try:
        from beartype.peps import resolve_pep563  # <-- your rictus grin is now threatening to swallow your whole head

        # Note the lack of memoization, which is unneeded for older Python versions.
        def gimme_dose_hints(obj: type | Callable) -> dict[str, object]:

            # Parse PEP 563-postponed string type hints back into actual type
            # hints. Unbelievably, this actually works in all possible use cases
            # regardless of type hint complexity. It's best to pretend.
            resolve_pep563(obj)

            return obj.__annotations__  # <-- still fine for older Python versions
            #return get_annotations(obj)  # <-- also fine, but note *NO* "format" parameter
    # If that optional dependency is unavailable, fallback to a less ideal
    # implementation that fails to support PEP 563 but at least sorta works.
    except ImportError:
        # Note the lack of memoization, which is unneeded for older Python versions.
        def gimme_dose_hints(obj: type | Callable) -> dict[str, object]:

            return obj.__annotations__  # <-- still fine for older Python versions
            #return get_annotations(obj)  # <-- also fine, but note *NO* "format" parameter

That's right. @beartype fully supports PEP 563... and has for a year or two. That's pretty much all I did for a year, honestly. I want that year back. Alas, all I have is PEP 563 support.

You can now profit from my loss by "piggybacking" your own runtime type-checker onto @beartype. But you do not want to require @beartype, of course. You only want to parasitically feed off @beartype's salty tears.

That's where our public beartype.peps.resolve_pep563() API comes in. @wesselb's Plum has supported PEP 563 for just as long as @beartype itself by calling that function. It works. It's battle-hardened. It does everything everybody wants.

You are now thinking: "I hate you. Bait-and-switch, much? I ain't calling nuffin'. I'm gonna copy-paste that right out of @beartype into {insert-my-better-package-here}." Yeah. That's not gonna work. I wish it would, honestly. Then you'd be more likely to do this. But PEP 563 support is extraordinarily non-trivial. I'm pretty sure it took in upwards of 100,000 lines of pure-Python to get that to work. Copy-pasting is probably infeasible. Just sayin'.

You are now thinking: "...I am going to live on a mountain, curl up into a little ball, and pretend that none of this ever happened."

Excellent decision! That's basically what I did. Just substitute "mountain" for "smelly wetlands in Canada." It's less picturesque, but more full of edible food. Assuming your definition of edible food includes insects and things that go <squelch-squelch>.

Choice 3: The Omega of Bad Choices

Actually... there is no Choice 3. Not yet, anyway. If Choice 3 did exist, it would look something like this:

• Require @beartype as a mandatory dependency. ^...heh.
• Call a public @beartype function to just do all of this for you already. Instead of defining your own gimme_dose_hints() function, you'd just call @beartype's beartype.gimme_dose_hints(). That function does not exist. But it might. Does anybody besides @wesselb want something like that? Since @wesselb be does, I'll be doing this anyway – but I might not necessarily publicly advertise it unless somebody else expresses interest.

Some of You Are Doing Something PEP 649 Hates

Some of you go a step further than simply accessing (i.e., getting, reading) annotation dictionaries. You're also modifying (i.e., setting, writing) annotation dictionaries. Unfortunately, PEP 649 really hates that.

Technically, you can still attempt to modify annotation dictionaries – but not directly. Python ≥ 3.13 will silently ignore any attempts to modify the __annotations__ dunder dictionary. Sure, you can do it. But you just made life non-deterministic for your userbase, because the __annotations__ dunder dictionary is now just a volatile cache that can be overwritten by any number of standard things – including the new __annotate__() dunder method, the inspect.get_annotations() function, and the typing.get_type_hints() function.

In other words, you probably want to think twice. Do you really need to modify the __annotations__ dunder dictionary? If not, you should stop doing that. If you really do need to modify that dictionary, your life just became even harder and more exhausting. Why? Because you now need to dynamically redefine the __annotate__() dunder method on all passed objects. Your redefined __annotate__() method must then perform the desired modifications, isolated to that method.

Now consider what this means if someone else (e.g., some other runtime type-checker) has already redefined the __annotate__() before you got there. What then? What about function composition? Good luck. That's what.

In short, it sucks. It really sucks. Consider finding another way – like, any other way – to do what you want. Because you really do not want to try modifying the __annotations__ dunder dictionary. It might even be the case that you can no longer reliably and portably do so in a manner consistent with function composition. I'm not sure, honestly. I wouldn't try it. But... if you're still convinced you want to fling yourself body-and-soul down that murder hole, that's certainly a "choice."

I Am So Tired. Please. Stop.

...yeah. We're all tired. I'm tired of Python repeatedly punching both itself and the runtime type-checking community in the face. But what can you do?

Sometimes, a man just gotta lie down.

[samuelcolvin]

Well I love the introduction, but don't like the body so much.

I'll discuss with the team next week and get back to you.

Either way, you should all come to the typing summit at pycon US, it works be nice to not be the only runtime wonk at the party.

You should probably invite the msgspec guy too. He'll probably reply with a msgspec Vs pydantic benchmark, as that seems to be his fetish, but fetishists should be welcome too.

[leycec]

...but don't like the body so much.

...heh. Can't say I disagree. When I first realized what the CPython dev community was foisting on all of us, I was livid for sleepless weeks on end. Now, I'm simply sad.

PEP 649 is like watching a runner with blade-legs run a 26-mile marathon faster than competitors with actual legs. Somehow, the runner's doing it! The crowd cheers. Confetti sprays everywhere. Then the runner stumbles a yard from the finish. A blade shatters on a pebble. The runner sprawls into the dirt, clutching a ruptured knee. The crowd goes silent. The runner crawls in abject desperation over the finish line – dead last. Yet in your heart of hearts, you know that they won.

PEP 649 is like that. Larry Hastings was so close to awesome. Then he made the default format=inspect.VALUE instead of format=inspect.FORWARDREF, breaking everything for everyone.

Now, I can only clutch my knee.

I'll discuss with the team next week and get back to you

I should note that I'm just some rando runtime typing devbro. I live in a cabin in the woods. Like all who live in cabins in the woods, I have no authority and even less wisdom.

Still, it seemed irresponsible not to strike up a discussion with everybody about this. I'm only surprised that CPython devs themselves didn't get the ball rolling. Especially after PEP 563 almost decapitated Pydantic and FastAPI for the Python 3.10 release.

Yet history repeats. Thus, I remain sad.

Either way, you should all come to the typing summit at pycon US...

I'm painfully introverted and even more painfully poor. But you're so nice! What a wonderful invitation. May you and all the other attendees party like it's Python 1.0. 🥳

...be nice to not be the only runtime wonk at the party.

Runtime wonk. So now I know what I finally am.

He'll probably reply with a msgspec Vs pydantic benchmark, as that seems to be his fetish, but fetishists should be welcome too.

🤣

[jcrist]

You should probably invite the msgspec guy too. He'll probably reply with a msgspec Vs pydantic benchmark, as that seems to be his fetish, but fetishists should be welcome too.

What a weird and rude thing to say.

[samuelcolvin]

It was meant (in keeping with the thread) to be a tongue in cheek comment. Apologies if it offended. Let's try to keep our sense of humour.

[alexmojaki]

I wouldn't bother with an LRU cache, either. Just go unbounded. The amount of space consumed by type hints and the total number of type hints across an app is very small by compare to the remainder of the app – especially for those of us in data science and web apps.

This doesn't work if your_runtime_checker is called on some locally defined class/function. In a long-running web app, this could be a serious memory leak.

[leycec]

Sure. Totally. Absolutely. You could even go so far as to make caching user-configurable. That said, the truly optimal solution is probably just to guarantee that you only call the aforementioned gimme_dose_hints() getter at most once per object. Constrain yourself to that and you don't even need or benefit from memoization, right?

@beartype basically can't do that, because I'm wrestling 200,000 lines of spaghetti code over here. Nothing is guaranteed. Thus, I memoize indiscriminantly. These are the days of our lives. 😮‍💨

[patrick-kidger]

Use a weakref? Not all types are weakref'able, but many are.

[patrick-kidger]

Blech. Thank you @leycec for this heads-up.

Q1: mutating __annotations__

FWIW, in jaxtyping we're very careful never to mutate the __annotations__ dictionary directly:

fn.__annotations__['foo'] = bar

because that would be mutating an object we don't own.

We do mutate our own objects that sit inside of __annotations__ dictionaries:

class MyJaxtypingClass:
    pass

foo = fn.__annotations__['foo']
if isinstance(foo, MyJaxtypingClass):
    foo.bar = baz

When you talk about mutating __annotations__ dictionaries, do you know if both are now verboten, or if the second is safe?

Q2: PEP 563

Ech, you're right, this is about to come back with a vengeance. This is going to be an absolute nightmare to resolve -- PEP649 was supposed to finally end the difficulty of handling stringified hints, and instead it's just going to make things worse.

Do you think we could raise this with the Python folks directly, and ask for a from __future__ import pep_649_annotations backport for earlier Python versions? Then at least we can tell people to use that. Otherwise PEP649 is going to make things worse, not better.

[leycec]

When you talk about mutating __annotations__ dictionaries, do you know if both are now verboten, or if the second is safe?

Both are now verboten. Sucks, huh? Verily, and I quote:

When either __annotations__ or __annotate__ is updated on an object, the other of those two attributes is now out-of-date and should also either be updated or deleted (set to None, in the case of __annotate__ which cannot be deleted). In general, the semantics established in the previous section ensure that this happens automatically. However, there’s one case which for all practical purposes can’t be handled automatically: when the dict cached by o.__annotations__ is itself modified, or when mutable values inside that dict are modified.

I stand corrected. That seems to imply that:

1. You can safely modify the __annotations__ dictionary "all-at-once" with a direct assignment.
2. The __annotate__() method will then automatically detect that and re-apply its own internal transformations if necessary to that dictionary. Actually, that's not what happens. Directly assigning to the __annotations__ dictionary implicitly destroys the __annotate__() method, which Python then sets to None. wuuut 😨

Maybe? It's brutal stuff. If I'm reading that right, then the only truly verboten design pattern appears to be modifying the type hints cached inside the __annotations__ dictionary. Not sure. I would hesitate doing either. The optimal approach seems to be to redefine the __annotate__() method.

Maybe. It's brutal stuff. More brutality:

Since this can’t be handled in code, it must be handled in documentation. This PEP proposes amending the documentation for inspect.get_annotations (and similarly for typing.get_type_hints) as follows:

If you directly modify the __annotations__ dict on an object, by default these changes may not be reflected in the dictionary returned by inspect.get_annotations when requesting either SOURCE or FORWARDREF format on that object. Rather than modifying the __annotations__ dict directly, consider replacing that object’s __annotate__ method with a function computing the annotations dict with your desired values. Failing that, it’s best to overwrite the object’s __annotate__ method with None to prevent inspect.get_annotations from generating stale results for SOURCE and FORWARDREF formats.

Personally, overwriting the object’s __annotate__() method with None strikes me as fundamentally insane. Nobody should ever do that. If somebody ever does that, then the original annotations have effectively been destroyed and can never be recovered. Or is the implication that CPython internally detects when __annotate__() is None and dynamically recreates a new __annotate__() method or something? But the problem with that is the __annotate__() method that you just overwrite with None might have been defined by a third-party package. If you overwrite that with None, you just destroyed their useful work. That can't be undone. Congrats. What an utter mess.

In my humble and clueless opinion, it's best to only redefine an object's __annotate__() to be a valid callable invoking the original __annotate__(). Thankfully, the PEP gives a useful example of this:

Here’s how a third-party wrapper class might implement __annotate__. In this example, the wrapper works like functools.partial, pre-binding one parameter of the wrapped callable, which for simplicity must be named arg:

def __annotate__(self, format):
    ann = inspect.get_annotations(self.wrapped_fn, format)
    if 'arg' in ann:
        del ann['arg']
    return ann

I sigh with sadness. 😮‍💨

UPDATE: Oh. I see. This is, indeed, brutal. There are now four distinct things that are all confusing, because they ambiguously share the same or similar names:

• A new __annotate__ data descriptor defined on the metaclass.
• A new __annotate__() method, which can inexplicably also be None. When you reassign it, the __annotate__ data descriptor sets the cached __annotation__ dictionary to None.
• The existing __annotation__ data descriptor defined on the metaclass.
• The existing __annotation__ dictionary is now just an ephemeral cached thing, which can inexplicably also be None. When you reassign it, the __annotation__ data descriptor sets the __annotate__() method to None.

It's all still madness. There's no way any of this survives the harsh light of third-party packages. Wrapping the __annotate__() method like above still makes the most sense to me. Every other approach strikes me as differing degrees of blood, fire, and eldritch madness.

Ech, you're right, this is about to come back with a vengeance. This is going to be an absolute nightmare to resolve -- PEP649 was supposed to finally end the difficulty of handling stringified hints, and instead it's just going to make things worse.

Unintended consequences. I hadn't actually thought things through, either – until I wrote this. Then I realized what was really about to happen. It's even worse than our worst nightmares of a dystopian world without JAX.

Do you think we could raise this with the Python folks directly, and ask for a from __future__ import pep_649_annotations backport for earlier Python versions?

Excellent ideas. My dark suspicion ^{...based on the bonkers-intense "The stringizer and the fake globals environment" section of PEP 649} is that a backport is probably infeasible. PEP 649 mandates very extreme changes to CPython – both at the low C level and at the higher Python level. Any hypothetical from __future__ import pep_649_annotations backport would need to:

• Conditionally define the newly added inspect.FORMAT and inspect.FORWARDREF globals but only when that future is enabled.
• Conditionally modify the behaviour of the inspect.get_annotations() and typing.get_type_hints() functions accordingly, which must also now accept optional format parameters but only when that future is enabled.
• Conditionally define __annotate__() code objects on all annotated classes and callables.
• Fundamentally modify the CPython parser to enable all of this insanity – possibly including modifications to the PEG-based language that underlies CPython.
• And so on and so on.

It's a huge ask, honestly. I'm unsure whether it's technically (yet alone pragmatically) feasible to backport this technology. If it was, it probably would have happened already. PEP 649 barely passed peer review as it is – and that was only after years of heated deliberation. The assumption appears to be that PEP 563 is the closest analogue for older Python versions. If you want a seamless transition from PEP 563 to 649, you enable from __future__ import annotations. Sucks, huh?

This is probably a close approximation of CPython's final will and testament on the issue. It's not quite written in stone – but it basically is. The first alpha release of CPython 3.13.0 was pushed out a few weeks ago, if I recall. The release schedule only accelerates from here.

Suspiciously, none of us in the runtime community were informed about any of this. I weep. 😭

[provinzkraut]

My dark suspicion ...based on the bonkers-intense "The stringizer and the fake globals environment" section of PEP 649 is that a backport is probably infeasible.

I asked this about a year ago, worrying about the same things now being discussed here, and the answer was pretty much what you said:

That's very unlikely to happen.

summary for those who don't have or don't want to read stuff on discord:

A: I understand the concern 🙂
A: Just difficult to see a better way of dealing with it, I think

P: A future import for PEP 649 would help

A: How would it help?

P: It would then be possible to at least only support one style of deferred annotations

A: As in, you'd want the future import backported to <=3.12? That's very unlikely to happen. PEP 649 will be a huge change, and generally even minor, risk-free features aren't backported to older Python versions. Only bugfixes.

---

This is probably a close approximation of CPython's final will and testament on the issue. It's not quite written in stone – but it basically is. The first alpha release of CPython 3.13.0 was pushed out a few weeks ago, if I recall. The release schedule only accelerates from here.

Yeah, that's pretty much what I gathered from the few exchanges I've had around this topic. It's gonna be a pain, no matter what 🤷‍♀️

Suspiciously, none of us in the runtime community were informed about any of this. I weep. 😭

Honestly, I find this baffling too. It feels like runtime typing is still seen as an afterthought when it comes to these things, even though projects like Pydantic and FastAPI which are fundamentally built around runtime typing are what introduced a huge chunk of the Python community to type annotations or convinced them of their usefulness in real life applications.. It's a bit of a shame really :/

[leycec]

@jcrist: @samuelcolvin and I politely summon you! Because @msgpack accesses the __annotations__ dictionary, you are exposed to this issue. This isn't our fault. We weep as well. 😭

[jcrist]

Thanks for the polite ping (and fun writeup)! I've opened an issue in our repo to track this (jcrist/msgspec#651).

AFAICT none of this has actually landed in CPython's dev branch yet (and I couldn't find a PR anywhere for it). Is there code for this outside of Larry Hasting's old repo (currently still branched against 3.10)? Or are we only dealing with hypothetical APIs as described in the PEP?

Because @msgpack accesses the annotations dictionary

small nit - it's msgspec, not msgpack. msgpack is a format that we support though!

[leycec]

Thanks for the polite ping (and fun writeup)! I've opened an issue in our repo to track this (jcrist/msgspec#651).

You're most welcome. That's excellent!

Or are we only dealing with hypothetical APIs as described in the PEP?

...lol. That's a huge relief, honestly. PEP 649 isn't quite ready for primetime. ^{<- understatement} If this PEP goes through as is without modification, the breakage across the Python ecosystem will prove quite intense.

Still, I'm kinda surprised there's not even a prospective PR. This is intense stuff and it's still earmarked for inclusion in Python 3.13, according to the PEP. It's kinda hard to see this making Python 3.13. Phew!

small nit - it's msgspec, not msgpack. msgpack is a format that we support though!

Gah! Sorry about that. Emoji cat hangs its head in shame. 😿

[patrick-kidger]

I've started a discussion on this in the Python forums here.

[leycec]

Superbness. I should participate. Yet, I remain tired. Would you mind conveying two principal concerns? This is a strange request, I know. But they're probably similar to your principal concerns: ^...probably

• To preserve compatibility, the new optional format parameter accepted by the inspect.get_annotations() and typing.get_type_hints() functions must default to FORWARDREF rather than VALUE. The current default of VALUE breaks compatibility by unpredictably raising unreadable NameError exceptions on accessing the __annotations__ dictionary, which prevents __annotations__ from being safely accessed, which (...waitforit) breaks compatibility.
• It's unclear how to safely modify type hints – or whether type hints even can be safely modified anymore. Pain points include:
  • How can two or more third-party packages safely modify __annotations__ and/or __annotate__() without destroying each other's work?
  • The fact that __annotations__ and __annotate__() silently destroy each other is concerning. Ideally, one of these dunder attributes should be the definitive source of truth concerning annotations and not destroyable; the other dunder attribute should just be a cache that is destroyed and then silently recreated whenever the first dunder attribute is modified.
  • None of these dunder attributes should be destroyable, really. None values do not make sense for either __annotations__ and __annotate__(). Am I missing something? I'm probably missing something.

Sorry. I know I'm being annoying. I know this entire issue is annoying. Thanks so much for taking point on this, @patrick-kidger. I now retreat to a face in the clouds. 😶‍🌫️

[patrick-kidger]

Tagging @pingsutw + @ddl-rliu + @wild-endeavor for Flytekit, as I believe you're doing some nontrivial runtime interaction with Python annotations as well. (I can see what looks like quite a complicated TypeEngine!)

[wesselb]

Whoa, @leycec, this is fantastically written. :)

What you wrote about, however, is much less great. 😭 It seems that all our lives are about to become a lot more painful.

Thanks for the heads up regarding Plum's exposure. That's good to know and an action item that's moved high up my priority list.

That's where our public beartype.peps.resolve_pep563() API comes in. @wesselb's Plum has supported PEP 563 for just as long as @beartype itself by calling that function. It works. It's battle-hardened. It does everything everybody wants.

To anyone reading this, I can confirm that beartype.peps.resolve_pep563() works very reliably. In Beartype we trust!