Raising exceptions with registered exception handlers from BackgroundTasks causes starlette exception

[alonme]

First check

• [V ] I added a very descriptive title to this issue.
• [ V] I used the GitHub search to find a similar issue and didn't find it.
• [ V] I searched the FastAPI documentation, with the integrated search.
• [ V] I already searched in Google "How to X in FastAPI" and didn't find any information.
• [ V] I already read and followed all the tutorial in the docs and didn't find an answer.
• [ V] I already checked if it is not related to FastAPI but to Pydantic.
• [ V] I already checked if it is not related to FastAPI but to Swagger UI.
• [ V] I already checked if it is not related to FastAPI but to ReDoc.

Example

from fastapi import BackgroundTasks, FastAPI, status
from starlette.requests import Request
from starlette.responses import JSONResponse


class MyException(Exception):
    ...

app = FastAPI()


@app.exception_handler(MyException)
async def sampler_exception_handler(request: Request, exc: MyException):
    return JSONResponse(
        status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
        content={"error": "i hope someone sees this"},
    )


def write_notification(email: str, message=""):
    raise MyException("this is my exception")
    with open("log.txt", mode="w") as email_file:
        content = f"notification for {email}: {message}"
        email_file.write(content)


@app.post("/send-notification/{email}")
async def send_notification(email: str, background_tasks: BackgroundTasks):
    background_tasks.add_task(write_notification, email, message="some notification")
    return {"message": "Notification sent in the background"}

Description

Raising an exception that an exception_handler is registered for, from a background task - causes an error from starlette, which also hides the original exception.

running the code from the example, and triggering the endpoint, i expected the error to be either handled by the exception handler, or maybe just ignoring the exception handler as this exception does not happen as part of a request context.

the actual behavior is as follows

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/Users/alonmenczer/.pyenv/versions/3.7.9/lib/python3.7/site-packages/uvicorn/protocols/http/h11_impl.py", line 369, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/Users/alonmenczer/.pyenv/versions/3.7.9/lib/python3.7/site-packages/uvicorn/middleware/proxy_headers.py", line 59, in __call__
    return await self.app(scope, receive, send)
  File "/Users/alonmenczer/.pyenv/versions/3.7.9/lib/python3.7/site-packages/fastapi/applications.py", line 199, in __call__
    await super().__call__(scope, receive, send)
  File "/Users/alonmenczer/.pyenv/versions/3.7.9/lib/python3.7/site-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/Users/alonmenczer/.pyenv/versions/3.7.9/lib/python3.7/site-packages/starlette/middleware/errors.py", line 181, in __call__
    raise exc from None
  File "/Users/alonmenczer/.pyenv/versions/3.7.9/lib/python3.7/site-packages/starlette/middleware/errors.py", line 159, in __call__
    await self.app(scope, receive, _send)
  File "/Users/alonmenczer/.pyenv/versions/3.7.9/lib/python3.7/site-packages/starlette/exceptions.py", line 86, in __call__
    raise RuntimeError(msg) from exc
RuntimeError: Caught handled exception, but response already started.

I understand this might be connected to a starlette design decision from previous discussions, but haven't actually seen this exact cause for the problem.

Environment

• OS: macOS
• FastAPI Version: 0.67.0
• Python version: Python 3.7.9

[alonme]

This seems to be resolved by encode/starlette#1158.

Will open a PR to update Starlette.

[alonme]

resolved by #4016
can be closed

[Matthieu-Tinycoaching]

@alonme I have the same problem of HTTP exceptions defined in the background task not been returned even with last version of Starlette.

How did this solve the problem?

[tiangolo]

The thing is that background tasks are run after the response is sent. So exception handlers that would return a different response can't do anything there. Also, just ignoring an exception because the handled couldn't handle it wouldn't be appropriate, that would be a bug. If the handler can't really handle the exception it should raise another error so that you can at least see in the server error logs that something is wrong in the code.

Also, even though the stack trace is not full, I would think it probably also included the original place of the exception, right? That would be the right thing to do, I would think.

Otherwise, given that background tasks are run after the response is already sent, what would you expect/want to happen?

[github-actions]

Assuming the original need was handled, this will be automatically closed now. But feel free to add more comments or create new issues or PRs.