How to make middleware to run on each thread created by FastApi

[karan-kumar-by]

First check

• I added a very descriptive title to this issue.
• I used the GitHub search to find a similar issue and didn't find it.
• I searched the FastAPI documentation, with the integrated search.
• I already searched in Google "How to X in FastAPI" and didn't find any information.
• I already read and followed all the tutorial in the docs and didn't find an answer.
• I already checked if it is not related to FastAPI but to Pydantic.
• I already checked if it is not related to FastAPI but to Swagger UI.
• I already checked if it is not related to FastAPI but to ReDoc.
• After submitting this, I commit to one of:
  • Read open issues with questions until I find 2 issues where I can help someone and add a comment to help there.
  • I already hit the "watch" button in this repository to receive notifications and I commit to help at least 2 people that ask questions in the future.
  • Implement a Pull Request for a confirmed bug.

Example

Here's a self-contained, minimal, reproducible, example with my use case:

from fastapi import FastAPI

app = FastAPI()


@app.get("/")
def read_root():
    return {"Hello": "World"}

Description

• Open the browser and call the endpoint /.
• It returns a JSON with {"Hello": "World"}.
• But I expected it to return {"Hello": "Sara"}.

Environment

• OS: [e.g. Linux / Windows / macOS]:
• FastAPI Version [e.g. 0.3.0]:

To know the FastAPI version use:

python -c "import fastapi; print(fastapi.__version__)"

• Python version:

To know the Python version use:

python --version

Additional context

Main Issue: @tiangolo @dmontagu

I am using a middleware that needs to set a context at each thread pool being created by fast API but as of now only it is done at the main thread. So when the thread pool tries to use the context it is not available. Which is causing errors.

I don't want to set the context on each path in my application.

Let me know if you need more info we can chat or meet if needed.

[yantrikskaran]

This is how I have created middleware
@app.middleware("http")
async def client_context(request: Request, call_next):
LOGGER.info(f"Request Start.")
client_id = request.url.path.split('/')[1]
Context.get().set_client(id=client_id)
try:
_LOGGER.info(f"Client Set.")
response = await call_next(request)
LOGGER.info(f"API executed.")
finally:
Context.get().set_client(id=None)
_LOGGER.info(f"Client un-set.")
_LOGGER.info(f"Request End.")
return response

[karan-kumar-by]

@Kludex Your thumps down means it is not possible or you don't like the question itself?

[Kludex]

I didn't like the fact that you ignored the template and didn't even care to remove it. 😗

[karan-kumar-by]

I didn't like the fact that you ignored the template and didn't even care to remove it. 😗

Oh my bad. I thought I had just a question to be asked. So I have put my question at the last. I am not sure If I can change now. But is my question is valid or not? Is there any workaround to achieve what I am looking for?

[Kludex]

I'm not sure if I completely understand the question. If you can add a code that shows what you're doing and what you expect, it will be clearer.

About the issue, removing the stuff that you ignored or using it would be a good idea 😅

[karan-kumar-by]

@Kludex This is the minimal code.

from fastapi import FastAPI, Request
from core import Context
import time

app = FastAPI()

@app.middleware("http")
async def client_context(request: Request, call_next):
client_id = request.url.path.split('/')[1]
Context.get().set_client(id=client_id)
try:
response = await call_next(request)
finally:
Context.get().set_client(id=None)
return response

@app.get("{client_id}/items/{item_id}")
async def read_item(item_id):
# If I define my endpoint as async def , It is blocking main thread and make next request to wait.
# Entire execution happens in main thread only.
print(Context.get().get_client())
time.sleep(10)
return {"item_id": item_id}

@app.get("{client_id}/items/{item_id}")
def read_item(item_id):
# If I define my endpoint as def, It is doesn't block the main thread as the execution happen in
# external thread pool but my client context is lost which I am setting using middleware.
print(Context.get().get_client())
time.sleep(10)
return {"item_id": item_id}

So what I am looking for a way to execute middleware in an external thread pool that actually handles the execution when I define my endpoint using def.

[karan-kumar-by]

The application context set by middleware is supposed to be used by the downstream thread/process which does the actual execution. We need to set the context for each process/thread per request.

We are trying to migrate from Flask to fast API. Flask by default starts the execution in the thread pool, not in the main thread. Hence the context is set per-thread based on the request it handles.

[karan-kumar-by]

@Kludex @tiangolo Any thought here? I am new to fast API so may be missing something here.

[hanzhenbjm]

Is there any good solution？

[estebanx64]

Hi there.

First of all @karan-kumar-by edit your question to follow the guidelines and to leave everything clear for someone else who could be the same question

Here's an example using contextvars that works for what you are looking for

from fastapi import FastAPI, Request
import time
import contextvars

app = FastAPI()


client_id = contextvars.ContextVar("client_id", default="unknown")

@app.middleware("http")
async def client_context(request: Request, call_next):
    client_id.set(request.url.path.split('/')[1])
    try:
        response = await call_next(request)
    finally:
        client_id.set("unknown")
    return response


@app.get("/items_async/{item_id}")
async def read_item_async(item_id):
    return {"item_id": item_id}


@app.get("/items/{item_id}")
def read_item(item_id: str):
    print(client_id.get())
    time.sleep(10)
    return {"item_id": item_id}