[Feature Request] Better Support for Evil Dependent-Type Style Shenanigans

[iamrecursion]

I want to preface all of this by saying that I have no idea if it is possible. To that end, I find myself sat outside the bear's cave with little but an imploring question to ask.

Consider the following piece of mis-worded arcane scribbling.

from beartype import beartype
from beartype.vale import Is
from typing import TypeVar, Annotated, 

@beartype
class MyInterface:
    length: Final[int]
    
@beartype
class MyImpl(MyInterface):
    length = 8
        
T = TypeVar('T', bound=MyInterface)

@beartype
class Example(Generic[T]):
    
    # I use an alias here for simplicity, but I would want the same in a function signature.
    MyAlias: TypeAlias = Annotated[list[int], Is[lambda l: len(l) == T.length]]
    
    @beartype
    def my_function(self, t: T, xs: MyAlias) -> None:
        pass
    
Example().my_function(MyImpl(), [i for i in range(8)])

I have no doubt that the sticking point is instantly visible to the bear's keen eyes. The T in T.length is the type variable itself, which definitely does not have the member length no matter what my IDE wants to think (as it has figured out that T indeed must have such a member).

To that end, I found myself wondering whether it would be possible to extend the validator API to have a magic function that lets us get at the correct interface by which T is bounded, and hence allow the above code to actually work at runtime. Something akin to ValueOf[T], which might also appease MyPy's complaints about T only being "uSaBlE In a tYpE CoNtExT".

Furthering My Descent Into Madness

While the above purely concerns itself with class-level fields of types bound to type variables, my time spent gazing into the dependently-typed abyss still leaves me wanting more, much like a bear after long hibernation.

Have a peek at the following, and please keep your eyes narrowed in case the madness of the abyss stares back. Please assume the same preamble as before.

@beartype
class Example2:
    TheMadness: TypeAlias = Annotated[list[int], Is[lambda l: len(l) == GetSelf.member]]

    member: int

    def __init__(self, member: int) -> None:
        self.member = member

    def do_something(self, list: TheMadness) -> bool:
        return True

Put simply, some way to access a late-bound (after instantiation) version of self would work wonders to quieting such gibbering in the back of my mind.

I do so apologise for bothering the great bear with my near-certainly insane ramblings, but would truly appreciate some thoughts on all of this!

[leycec]

...heh. What a wonderfully striking feature request. The pathos in your words moves me. Allow me to now collect your accumulated wisdom for the edification of future generations, who are data-mining the past ^{...confusingly, our present} even as we speak in this historical timeline:

I find myself sat outside the bear's cave

instantly visible to the bear's keen eyes

MyPy's complaints about T only being "uSaBlE In a tYpE CoNtExT".

Furthering My Descent Into Madness

my time spent gazing into the dependently-typed abyss still leaves me wanting more, much like a bear after long hibernation.

keep your eyes narrowed in case the madness of the abyss stares back.

quieting such gibbering in the back of my mind.

bothering the great bear with my near-certainly insane ramblings

Good times. Good times were had. Future generations have now acquired wisdom through us.

Indeed, this wisdom increasingly resembles the unutterable space abominations of H.P. Lovecraft. Anyone else notice that? Anyone? No? Just me? In our inchoate and kinda incoherent importunement of the truth, we now summon that well-known and much-loved eldritch typing god. Verily, I speak of The Grizzled Beartyper at the Threshold. Awaken, ponderous and greasy bulk of fur! Arise and crush the many-appendaged bugs arrayed in these virgin codebases before you!

...uhh

Ooooops. I kinda went Asperger's there... again. Let's see if we can't steer this derailed hype train back to a semblance of useful yet unpaid productivity. Specifically, let's review these type hint abominations that you probably shouldn't be trying to summon yet are:

MyAlias: TypeAlias = Annotated[list[int], Is[lambda l: len(l) == T.length]]
TheMadness: TypeAlias = Annotated[list[int], Is[lambda l: len(l) == GetSelf.member]]

As you astutely espy from your digital eyrie, Acolyte Dread Priestess @iamrecursion, the references to T and GetSelf are where sanity problematically breaks down. Those things don't exist. But other comparable things do exist – things like:

• Rather than T.length, t.length for the t parameter passed to the Example.my_function() method.
• Rather than GetSelf.member, self.member for the self parameter passed to the Example2.do_something() method.

So we're agreed that comparable things exist. We're also agreed that the beartype.vale API currently offers no means of passing these things to beartype validators. Ideally, some new beartype validator would.

So I dub this shambolic nightmare...

beartype.vale.IsArgumentative! Of course, that name is awful. That's just how @beartype roll. We accept an awful nomenclature and then exhaustedly run many miles under that heavy load, which we later regret but can no longer rename without breaking PyTorch and the basis for American supremacy itself.

beartype.vale.IsArgumentative generalizes beartype.vale.Is. It's probably best to keep these two validators separate. The central dogma here is that:

• Like Is:
  • IsArgumentative is subscripted by a single callable – typically, a lambda function for convenience. Laziness. Laziness is what I'm saying.
  • The first argument accepted by the callable subscripting IsArgumentative is the current object being type-checked (e.g., l for the list[int] type hint above).
• Unlike Is:
  • The callable subscripting IsArgumentative is required to accept two or more arguments.
  • All arguments following the first argument accepted by the callable subscripting IsArgumentative are the names of parameters accepted by the parent callables annotated by this IsArgumentative validator.

That final bullet point is the beating heart of IsArgumentative. In a pre-cracked nutshell, IsArgumentative facilitates communication between calls to external callables and type hints by forwarding parameters of interest passed to those calls onto those type hints.

...uhh. Can We Get Funny Again? I Had More Fun Back in Those Days.

Those days were just a few paragraphs above! And those days were useless. We didn't do anything except sit around trading diabolical puns and cutthroat jibes with keyboards. Things are better now. Our keyboards are doing something useful for once.

Admittedly, that "something" is confusing even me. Let's put a concrete spin on things by reframing your above examples in terms of this bold and disturbing new IsArgumentative reality:

from beartype import beartype
from beartype.vale import IsArgumentative  # <-- *IMPORTED FOR TRUTH*
from typing import TypeAlias, TypeVar, Annotated 

@beartype
class MyInterface:
    length: Final[int]
    
@beartype
class MyImpl(MyInterface):
    length = 8
        
T = TypeVar('T', bound=MyInterface)

@beartype
class Example(Generic[T]):
    
    # I use an alias here for simplicity, but I would want the same in a function signature.
    MyAlias: TypeAlias = Annotated[list[int], IsArgumentative[
        lambda l, t: len(l) == t.length]]  # <-- *OMG* the truth stuns my eyes like a UAP on fire
    
    @beartype
    def my_function(self, t: T, xs: MyAlias) -> None:
        pass
    
Example().my_function(MyImpl(), [i for i in range(8)])

Above, we see that the MyAlias type hint now "reaches around" and into the open kimono of the Example.my_function() method – directly accessing the exact same t parameter as passed to that method. Likewise:

@beartype
class Example2:
    TheMadness: TypeAlias = Annotated[list[int], IsArgumentative[
        lambda l, self: len(l) == self.member]]  # <-- *WOAH* more promiscuous function invasions

    member: int

    def __init__(self, member: int) -> None:
        self.member = member

    def do_something(self, list: TheMadness) -> bool:
        return True

Again, we see that the appropriately named TheMadness type hint slyly insinuates itself into the namespace of the Example2.do_something() method – directly accessing the exact same self parameter as passed to that method.

I Like What I See. I Want to Believe. But Will Anybody Actually Do This?

You've hit the nail on the fluffy head right there, @iamrecursion. Thank you for acknowledging the 750-pound fat lethargic bear in the room. I play video games, watch anime, and read light novels. Those things don't exactly do themselves. What does this mean? What am I saying? Why am I dashing everyone's hopes and dreams on the hard shoals of depressing reality!?!

...because I am very tired. I may do this. I would like to believe that I will do this. But I have promised many things over many years to many nice people. Of those promises, I fulfilled two or three. It's not a prideful track record. We're on the Highway to Project Hell here. The Guinness Book of World Project Management Records may be someone's destination – but it's probably not ours.

Beartype: do something for once! 😮‍💨

[iamrecursion]

Specifically, let's review these type hint abominations that you probably shouldn't be trying to summon yet are

Story of my life. I reach for the deepest of unspeakables and the darkest of magicks. After all, surely we should be able to do all validation in types, so sayeth the great bear.

As you astutely espy from your digital eyrie, Acolyte Dread Priestess @iamrecursion, the references to T and GetSelf are where sanity problematically breaks down.

"Acolyte Dread Priestess", hm? I feel very honored to have been bestowed this title that speaks of such power, and of such madness.

The Gory Details

beartype.vale.IsArgumentative generalizes beartype.vale.Is. It's probably best to keep these two validators separate.

I definitely agree on keeping them separate! Fundamentally the requirement to accept two or more arguments makes sense, as does the decorator (at least in a class context) being able able to pull these things out and marry up the additional arguments with the parameter values from the parent callable.

One worry I have with this is that there are unfortunately restrictions placed on defining type aliases inside classes. Those that fear the typing cabal are always trying to get in the way of the eldritch horrors. In particular, you cannot define generic type aliases inside classes, even with the new syntax for type aliases! I can, at least, think of some horrendous deferred ways in which we could achieve solutions to that as well should we need to.

Alas, though certain types of deep magick may remain unspoken—perhaps rightfully so—your proposed IsArgumentative seems like it would solve many of the use-cases I have run into for such a thing, even the ones today that had me beseeching the bear for support.

Time and Space are Limited Resources

Thank you for acknowledging the 750-pound fat lethargic bear in the room. I play video games, watch anime, and read light novels.

As you so presciently say, the bear only can do so much at once.

I do, however, have a question to posit. After all, bears to build families from time to time, with littler and less-experienced bears to learn the same rituals and hold the types accountable.

If you see a way—from the perspective of this codebase—to allow the accursed unutterable IsArgumentative to actually work without putting too much onus on the great bear (and yourself, dear maintainer), then I am certain that with some guidance and some kindness from my schedule that I could give this a go.

After all, though beartype's codebase is an arcane weave of the likes I have not delved deeply into before, staring for long at the eldritch horrors of more and more powerful type-releated systems truly does bring on the madness. I think I am just about mad enough to try.

If, however, such an uttering of the deep horrors would impose too much burden on the bear's codices, I shall abandon this notion entirely and consign it to an appendix of The Book of Dark Magicks, Volume 7.

[iamrecursion]

I am sorry to be making a continued nuisance of myself outside the bear's cave, but @leycec do you think something like this would be doable inside the codebase without causing too much trouble?

Not by you, to be clear, as I'm totally up for trying my hand at it if it would not result in undue maintenance burden!

[iamrecursion]

Well I knew this would be complicated but not quite how complicated. I've given this a bit of a go today and I'm totally lost in the machinery of beartype!

I definitely plan on bashing away at this over time regardless!