add_exception_handler have type annotation problems

[pohmelie]

Since starlette 0.31.0 with mypy 0.108.0:

from starlette.applications import Request, Starlette
from starlette.responses import JSONResponse

class MyException(Exception):
    def __init__(self, message: str) -> None:
        self.message = message
        self.extra = "extra"

def my_exception_handler(request: Request, exc: MyException) -> JSONResponse:
    return JSONResponse({"detail": exc.message, "extra": exc.extra}, status_code=400)

app = Starlette(debug=True, routes=[])
app.add_exception_handler(MyException, my_exception_handler)

mypy produce:

yoba.py:13: error: Argument 2 to "add_exception_handler" of "Starlette" has incompatible type "Callable[[Request, MyException], JSONResponse]"; expected
"Callable[[Request, Exception], Response | Awaitable[Response]] | Callable[[WebSocket, Exception], Awaitable[None]]"  [arg-type]
    app.add_exception_handler(MyException, my_exception_handler)
                                           ^~~~~~~~~~~~~~~~~~~~
Found 1 error in 1 file (checked 1 source file)

And it definetly right, if handler must receive Exception type, then MyException is invalid. But, if I change MyException to Exception like this:

def my_exception_handler(request: Request, exc: Exception) -> JSONResponse:

mypy produce:

yoba.py:10: error: "Exception" has no attribute "message"  [attr-defined]
        return JSONResponse({"detail": exc.message, "extra": exc.extra}, status_code=400)
                                       ^~~~~~~~~~~
yoba.py:10: error: "Exception" has no attribute "extra"  [attr-defined]
        return JSONResponse({"detail": exc.message, "extra": exc.extra}, status_code=400)
                                                             ^~~~~~~~~
Found 2 errors in 1 file (checked 1 source file)

And it is reasonable too, because Exception have no such attributes as MyException have.

I saw the problem, because fastapi bumped starlette version above 0.30.0. Anyway, it looks like a starlette issue. Because if I bind handler to handle some kind of exception, then I expect exact this exception or its subclass as a handler argument.

[adriangb]

This part of the API can’t be typed correctly. Maybe something like #2042 (comment) works for you?

[pohmelie]

I don't get, what should works for me? I do not want change starlette sources, just my code outside. If it is not possible, then # typing: ignore is the best solution here.

Anyway, I checked out your links and do not get why you do not want split Starlette.exception_handlers from typing.Mapping[typing.Any, ExceptionHandler] to something like typing.Mapping[tuple[ExceptionType, int | None], ExceptionHandler[ExceptionType]] generic?

[adriangb]

You can just copy ExceptionHandlerBuilders to you code and get type safe behavior

[pohmelie]

I don't get how this should work. Can you provide code for this?

from typing import Callable, TypeVar, overload
from starlette.applications import Request, Starlette
from starlette.exceptions import HTTPException
from starlette.responses import JSONResponse

class MyException(Exception):
    def __init__(self, message: str) -> None:
        self.message = message
        self.extra = "extra"

def my_exception_handler(request: Request, exc: MyException) -> JSONResponse:
    return JSONResponse({"detail": exc.message, "extra": exc.extra}, status_code=400)

ExcType = TypeVar("ExcType", bound=Exception)
Handler = Callable[[Request, ExcType], JSONResponse]

class ExcHandlersBuilder(dict[int | ExcType, Handler[ExcType]]):
    @overload
    def add_handler(self, code: int, handler: Handler[HTTPException]) -> None:
        ...

    @overload
    def add_handler(self, exc: ExcType, handler: Handler[ExcType]) -> None:
        ...

    def add_handler(self, code_or_exc: int | ExcType, handler: Handler[ExcType]) -> None:
        self[code_or_exc] = handler

handlers = ExcHandlersBuilder()
handlers.add_handler(MyException, my_exception_handler)
app = Starlette(debug=True, routes=[])

yoba.py:26: error: Overloaded function implementation does not accept all possible arguments of signature 1  [misc]
        def add_handler(self, code_or_exc: int | ExcType, handler: Handler[ExcType]) -> None:
        ^
yoba.py:26: error: Overloaded function implementation does not accept all possible arguments of signature 2  [misc]
        def add_handler(self, code_or_exc: int | ExcType, handler: Handler[ExcType]) -> None:
        ^
yoba.py:29: error: Need type annotation for "handlers"  [var-annotated]
    handlers = ExcHandlersBuilder()
               ^~~~~~~~~~~~~~~~~~~~
Found 3 errors in 1 file (checked 1 source file)

[pohmelie]

Nevermind, this works:

from starlette.applications import Request, Starlette
from starlette.responses import JSONResponse

class MyException(Exception):
    def __init__(self, message: str) -> None:
        self.message = message
        self.extra = "extra"

def my_exception_handler(request: Request, exc: MyException) -> JSONResponse:
    return JSONResponse({"detail": exc.message, "extra": exc.extra}, status_code=400)

handlers = {MyException: my_exception_handler}
app = Starlette(debug=True, routes=[], exception_handlers=handlers)

[pohmelie]

Should I close this?

[tekumara]

The workaround fails for me in mypy, and I get the same error in pyright, when trying the original snippet and the workaround:

Argument of type "(request: Request, exc: MyException) -> JSONResponse" cannot be assigned to parameter "handler" of type "ExceptionHandler" in function "add_exception_handler"
  Type "(request: Request, exc: MyException) -> JSONResponse" cannot be assigned to type "ExceptionHandler"
    Type "(request: Request, exc: MyException) -> JSONResponse" cannot be assigned to type "HTTPExceptionHandler"
      Parameter 2: type "Exception" cannot be assigned to type "MyException"
        "Exception" is incompatible with "MyException"
    Type "(request: Request, exc: MyException) -> JSONResponse" cannot be assigned to type "WebSocketExceptionHandler"
      Parameter 1: type "WebSocket" cannot be assigned to type "Request"
        "WebSocket" is incompatible with "Request"
      Parameter 2: type "Exception" cannot be assigned to type "MyException"

I think the problem is Callable is contravariant (for good reason) and so whilst MyException is a sub-type of Exception, the Callable is not.

I believe its possible to fix the type exception handler type hints within Starlette by using a bound TypeVar. I can attempt a PR to demonstrate this.

[tekumara]

Here's a proposed PR to address this: #2403

[pohmelie]

Looks good! I hope maintainers will take a look on this PR 🤞🏻