Optional attrs and methods for Protocol classes

[tvdboom]

Alright, here I come with a new issue. I was searching for a way to add optional attributes and methods to a protocol and this is everything I could find: python/typing#601. Example:

from typing import NotRequired, Protocol
from sklearn.preprocessing import StandardScaler

class Transformer(Protocol):
    optional_attr: NotRequired[str]
    def transform(self, **params): ...

isinstance(StandardScaler(), Transformer)  # false, but mypy passes! Would be nice if beartype passes as well...

Using NotRequired is not ideal, but it does the trick for mypy. Unfortunately, beartype raises an error becasue it doesn't acknowledge the NotRequired attribute as being optional (probably because isinstance also returns False).

So, how to solve this? Is there a way to add optional attributes and methods to protocol classes? Would it be noption to implement something like this in beartype?

[leycec]

Excellent question! Does mypy really accept that? That... might be a bug. Maybe? That's certainly non-standard, which isn't great. Even if mypy accepts that, it's doubtful any other type-checkers do. To boringly quote PEP 544 – Protocols: Structural subtyping (static duck typing):

While the idea of optional members looks interesting, it would complicate this proposal and it is not clear how useful it will be. Therefore, it is proposed to postpone this; see rejected ideas.

...

We can come up with examples where it would be handy to be able to say that a method or data attribute does not need to be present in a class implementing a protocol, but if it is present, it must conform to a specific signature or type. One could use a hasattr() check to determine whether they can use the attribute on a particular instance.

Languages such as TypeScript have similar features and apparently they are pretty commonly used. The current realistic potential use cases for protocols in Python don’t require these. In the interest of simplicity, we propose to not support optional methods or attributes. We can always revisit this later if there is an actual need.

Sadly, PEP 544 intentionally chose not to do anything here. </sigh>

Sadly x 2, the completely unrelated PEP 655 – Marking individual TypedDict items as required or potentially-missing that standardized the typing.NotRequired type hint currently only applies to typing.TypedDict subclasses. It doesn't actually apply to anything else — or it shouldn't, anyway.

What Is Even Going on Anymore!?

If mypy has quietly extended typing.NotRequired support to non-typing.TypedDict subclasses (like typing.Protocol subclasses), that would be very non-standard, unexpected, and exciting. Alas, this is probably just a mypy bug.

Would you mind copy-pasting your fascinating Transformer example to either the python/typing#601 thread or a new issue on mypy's tracker? You're likely to receive an answer that actually answers something, instead of more baseless speculation from @leycec. I know so little; yet, I pretend to know something.

Oh — and thanks so much for the continued excitement surrounding @beartype and protocols. By some divine miracle, we will eventually get all of this to work for you. You are posing thought-provoking problems and deserve Python typing that actually works. Two jaundiced thumbs way up! 👍 👍