[Docos] Add a new FAQ entry documenting how to define custom type hints via __instancecheck__() and __instancecheck_str__()

[sylvorg]

Hello!

Is there any way to modify the behavior of is_bearable, die_if_unbearable, and is_subhint using a class magic / dunder method, such as __bearabilitycheck__ and __subhintcheck__?

Thank you kindly for the help!

[leycec]

Gah! So sorry for the extreme delay, favourite undergrad genius @sylvorg. It's been one of those weeks. We know the kind. The kind that make you go cross-eyed with consternation as your furrowed brow starts to dig worry lines into your pallid grey cheeks.

You're almost in luck, though. @beartype does half of what you want. The other half? Nadda. I got nuthin'.

But let's start with the good news:

Is there any way to modify the behavior of is_bearable, die_if_unbearable... using a class magic / dunder method...?

Yes! So much yes. @beartype fully complies with PEP 3119 – Introducing Abstract Base Classes. This includes:

• The beartype.door.is_bearable() and beartype.door.die_if_unbearable() functions.
• The @beartype.beartype decorator.
• All beartype.door import hooks (e.g., beartype_this_package())

All of those things allow you to define your own custom type hints as third-party classes whose metaclasses define:

• The standard __instancecheck__() and __subclasscheck__() dunder methods.
• A @beartype-specific __instancecheck_str__() dunder method. Whereas __instancecheck__() returns a bool describing whether or not the passed instance satisfies the passed class, __instancecheck_str__() returns a str describing in human-readable language why the passed instance violates the passed class.

This sort of thing gets really brutal really fast. Here is what I am trying but failing to say:

#!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
#*YOU DEFINE THESE IN YOUR THIRD-PARTY PACKAGE.*
#!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
class MuhTruthDetector(type):
    def __instancecheck__(cls, instance: object) -> bool:
        return bool(getattr(cls, 'muh_truthiness', None))
    def __instancecheck_str__(cls, instance: object) -> str:
        return "I see your beady little eyes peeping in the darkness."
class MuhTruth(object, metaclass=MuhTruthDetector):
    def __new__():
        raise ValueError("Don't mess with the guy in shades.")
    muh_truthiness: bool = True
class MuhLies(object, metaclass=MuhTruthDetector):
    def __new__():
        raise ValueError("I wear my sunglasses at night.")
    muh_truthiness: bool = False

#!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
#*YOUR USERS DEFINE THESE IN THEIR SEPARATE THIRD-PARTY PACKAGES.*
#!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
class TruthIsGood(object):
    muh_truthiness: bool = True
class LiesIsBad(object):
    muh_truthiness: bool = False

#!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
#*YOUR USERS THEN DO STUFF LIKE THIS.*
#!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
from beartype.door import die_if_unbearable, is_bearable

truth_is_good = TruthIsGood()
lies_is_bad = LiesIsBad()
assert is_bearable(truth_is_good, MuhTruth) is True
assert is_bearable(lies_is_bad, MuhLies) is False
die_if_unbearable(truth_is_good, MuhTruth)
die_if_unbearable(lies_is_bad, MuhLies)

...which raises the expected type-checking violation:

beartype.roar.BeartypeDoorHintViolation: Die_if_unbearable() value
<__main__.LiesIsBad object at 0x7f772e017650> violates type hint
<class '__main__.MuhLies'>, as I see your beady little eyes peeping in the darkness.

Note the custom type-checking violation message "I see your beady little eyes peeping in the darkness." 👀

The key takeaway is that the MuhTruth and MuhLies classes that you define are purely virtual abstract classes. They aren't intended to be instantiated or subclassed by anyone – not even you. They're only intended to be used as type hints (e.g., passed as the second parameter to the is_bearable() and die_if_unbearable() functions). They're custom type hints that you yourself define and that @beartype then implicitly supports.

That's the good news. Now we get to the bad news:

Is there any way to modify the behavior of ...is_subhint using a class magic / dunder method...?

Nuffin'. Ain't got nuffin. 😭

[sylvorg]

Hello hello hello, and no worries! 😸

Could you then help me go about creating a custom subscriptable type as mentioned here? I tried to come up with me own solution below, but, well...

from typing import TypeVar, Generic, get_origin, get_args
from beartype.door import is_bearable

T = TypeVar('T')

class Lazy(Generic[T]):
    def __isinstance__(self, value):
        return self.__args__[0](value)

print(get_origin(Lazy[lambda i: i < 1])) # Lazy
print(get_args(Lazy[lambda i: i < 1])) # lambda i: i < 1
print(is_bearable(0, Lazy[lambda i: i < 1])) # True
print(is_bearable(1, Lazy[lambda i: i < 1])) # False

[leycec]

...brutal. You can do what you want to do – but you'll have to manually roll out support yourself. typing.Generic and types.GenericAlias can no longer help you, because you're now off the deep end. Thankfully, this is Python. If you're willing to shed enough blood and tears pounding in frustration on the keyboard, you can do whatever you want:

from beartype import beartype
from collections.abc import Callable
from functools import cache
from typing import NoReturn

FunkyChecker = Callable[[object], bool]

@beartype
class FunkyTypeHint(object):
    @classmethod
    @cache
    def __class_getitem__(cls, func: FunkyChecker) -> type['_FunkedTypeHint']:
        subcls = type(
            f'_FunkedTypeHintSubclass_{id(func)}',
            (_FunkedTypeHint,),
            {},
        )

        subcls.__args__ = (func,)
        subcls.__func__ = func
        return subcls


@beartype
class _FunkedTypeHintMetaclass(type):
    def __instancecheck__(
        cls: type['_FunkedTypeHint'], instance: object) -> bool:
        return cls.__func__(instance)

    #FIXME: Define as needed for pain.
    # def __instancecheck_str__(cls, instance: object) -> str:
    #     return repr(instance)


@beartype
class _FunkedTypeHint(object, metaclass=_FunkedTypeHintMetaclass):
    __args__: tuple = None  # type: ignore
    __func__: FunkyChecker = None  # type: ignore
    __origin__ = FunkyTypeHint  # type: ignore

    def __new__() -> NoReturn:
        raise ValueError('"_FunkedTypeHint" not instantiatable.')


from typing import TypeVar, Generic, get_origin, get_args
from beartype.door import is_bearable

T = TypeVar('T')

IsNonpositive = FunkyTypeHint[lambda i: isinstance(i, int) and i < 1]
print(IsNonpositive.__origin__) # FunkyTypeHint
print(IsNonpositive.__args__) # lambda i: i < 1
print(is_bearable(0, IsNonpositive)) # True
print(is_bearable(1, IsNonpositive)) # False

...which prints:

<class '__main__.FunkyTypeHint'>
(<function <lambda> at 0x7fc7ff752480>,)
True
False

Lotsa black magic here. Everything works – except integration with the typing.get_origin() and typing.get_args() getters. But that's absolutely fine. Those getters suck. More specifically, those getters only support official type hints. However, most type hints of interest to @beartype users are unofficial type hints. Examples include jaxtyping, pandera, jax.typing, numpy.typing, and so on. Since you're defining unofficial third-party type hints, those getters hate you. Nobody should be calling those getters. This includes you and everybody else.

Instead, just access the __args__ and __origin__ fields directly. This is Python. We do what we want here – regardless of what CPython devs think. 💪 🐻

[sylvorg]

Gah! So sorry for the extreme delay, favourite static-typing genius @leycec. It's been one of those 2 weeks. We know the kind. The kind that make you go cross-eyed with consternation as your furrowed brow starts to dig worry lines into your pallid grey cheeks.

😹

I'm currently trying to modify your solution to work with get_args and get_origin; I'll get back to you when it works! Shouldn't take too long!

[sylvorg]

Okay, I may be sounding like a broken record at this point but the following:

from abc import ABCMeta
from beartype.door import is_bearable
from types import GenericAlias
from typing import Generic, TypeVar, get_origin, get_args

T = TypeVar("T")

class LazyMeta(ABCMeta):
    def __instancecheck__(cls, value):
        return cls.__args__[0](value)

class LazyAlias(GenericAlias):
    def __instancecheck__(self, value):
        return self.__args__[0](value)

class Lazy(Generic[T], metaclass=LazyMeta):
    @classmethod
    def __class_getitem__(cls, *params):
        return LazyAlias(cls, *params)

print(get_origin(Lazy[lambda i: i < 1]), get_origin(Lazy[lambda i: i < 1]) is Lazy) # Lazy
print(get_args(Lazy[lambda i: i < 1])) # lambda i: i < 1
print(isinstance(0, Lazy[lambda i: i < 1])) # True
print(isinstance(1, Lazy[lambda i: i < 1])) # False

# TODO: Failing with the error below
print(is_bearable(0, Lazy[lambda i: i < 1])) # True
print(is_bearable(1, Lazy[lambda i: i < 1])) # False

Works with isinstance but not is_bearable, failing with the following traceback:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /mnt/wsl/sylvorg/sylvorg/sylveon/siluam/oreo/./test25.py:130 in <module>                         │
│                                                                                                  │
│   127 print(get_args(Lazy[lambda i: i < 1])) # lambda i: i < 1                                   │
│   128 print(isinstance(0, Lazy[lambda i: i < 1])) # True                                         │
│   129 print(isinstance(1, Lazy[lambda i: i < 1])) # False                                        │
│ ❱ 130 print(is_bearable(0, Lazy[lambda i: i < 1])) # True                                        │
│   131 print(is_bearable(1, Lazy[lambda i: i < 1])) # False                                       │
│   132                                                                                            │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │      ABCMeta = <class 'abc.ABCMeta'>                                                         │ │
│ │         dirs = <function dirs at 0x7fce78a059e0>                                             │ │
│ │      Generic = <class 'typing.Generic'>                                                      │ │
│ │ GenericAlias = <class 'types.GenericAlias'>                                                  │ │
│ │     get_args = <function get_args at 0x7fce7ae507c0>                                         │ │
│ │   get_origin = <function get_origin at 0x7fce7ae50720>                                       │ │
│ │  is_bearable = <function is_bearable at 0x7fce79ec7b00>                                      │ │
│ │         Lazy = <class '__main__.Lazy'>                                                       │ │
│ │    LazyAlias = <class '__main__.LazyAlias'>                                                  │ │
│ │     LazyMeta = <class '__main__.LazyMeta'>                                                   │ │
│ │       pprint = <function pprint at 0x7fce7aafd6c0>                                           │ │
│ │      subtype = <multiple-dispatch function subtype (with 5 registered and 0 pending          │ │
│ │                method(s))>                                                                   │ │
│ │            T = ~T                                                                            │ │
│ │      TypeVar = <class 'typing.TypeVar'>                                                      │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                                  │
│ /nix/store/gc7zi20fw16lykavx1pr02g4vxrhv6gj-python3-3.11.8-env/lib/python3.11/site-packages/bear │
│ type/door/_doorcheck.py:299 in is_bearable                                                       │
│                                                                                                  │
│   296 │   func_tester = make_func_tester(hint, conf)                                             │
│   297 │                                                                                          │
│   298 │   # Return true only if the passed object satisfies this hint.                           │
│ ❱ 299 │   return func_tester(obj)  # pyright: ignore                                             │
│   300                                                                                            │
│                                                                                                  │
│ ╭───────────────────────────── locals ─────────────────────────────╮                             │
│ │        conf = BeartypeConf()                                     │                             │
│ │ func_tester = <function __beartype_checker_13 at 0x7fce77d35f80> │                             │
│ │        hint = __main__.Lazy[__main__.<lambda>]                   │                             │
│ │         obj = 0                                                  │                             │
│ ╰──────────────────────────────────────────────────────────────────╯                             │
│ in __beartype_checker_13:9                                                                       │
│                                                                                                  │
│ /mnt/wsl/sylvorg/sylvorg/sylveon/siluam/oreo/./test25.py:114 in __instancecheck__                │
│                                                                                                  │
│   111                                                                                            │
│   112 class LazyMeta(ABCMeta):                                                                   │
│   113 │   def __instancecheck__(cls, value):                                                     │
│ ❱ 114 │   │   return cls.__args__[0](value)                                                      │
│   115                                                                                            │
│   116 class LazyAlias(GenericAlias):                                                             │
│   117 │   def __instancecheck__(self, value):                                                    │
│                                                                                                  │
│ ╭──────────── locals ─────────────╮                                                              │
│ │   cls = <class '__main__.Lazy'> │                                                              │
│ │ value = 0                       │                                                              │
│ ╰─────────────────────────────────╯                                                              │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
AttributeError: type object 'Lazy' has no attribute '__args__'

Would it be possible to massage the above for use with is_bearable, or is your method the only way forward?

[leycec]

WOAH. There are multiple incredibly intriguing insights ⁱ³ you've just made there. If I'm reading this debug output correctly, you've independently invented a novel mechanism for defining third-party type hint factories that play well with first-party typing introspection (e.g., typing.get_args(), typing.get_origin()). Moreover, your approach is considerably more concise than the standard approach detailed above.

For your massive cleverness, you win the Internet. Congrats. Let's open up a new feature request tracking support for this – because @beartype should support this but currently doesn't, probably because you're the first person to invent this. Congrats.

^{dancing bears congratulate @sylvorg}

[sylvorg]

Yay! 🤗

Also, note that the ABCMeta does not seem to be strictly necessary, but seems to be in line with the typing classes.

We can also create a factory using types.new_class, where just providing the name and the __instancecheck__ could generate all three of the necessary classes and add them to the global namespace, though with all the appropriate warnings, of course.

Or perhaps even just avoid returning the Meta and Alias classes, since they seem to be useless after getting them? We could just put them in a class variable in the final class, such as __meta__ and __alias__ or similar.

[leycec]

Feature request #388 is live. Let's blow the lid off this madness! 🥳

Oh – and I simplified your original invention a bit. The LazyMeta metaclass is (...probably) ultimately unnecessary and only complicating matters. Things are already spicy enough. So, I chucked LazyMeta into the dustbin of GitHub. Everything else looks stellar, though. If you wouldn't mind, let's continue confabulating over at #388.

If all goes well and @beartype eventually supports this – which it absolutely should, right @beartype!? – your invention will (...probably) be the central focus of the new FAQ entry resolving this doco issue. Hype strengthens.

[sylvorg]

No worries, and see you there! 😹