How to use TestClient with requests_mock?

[wadamek65]

Suppose I have a route in my API that creates an http request to an external resource using requests package. When creating a starlette TestClient, it creates four adapters for ws, wss, http and https, so that it can handle all requests to the API. This is problematic since if I want to mock a route with requests_mock.Mocker() for requests to use, all the adapters created by TestClient get overwritten and do not get matched anymore resulting in NoMockAddressError.

I want to create a test with a mocked response for the address I use in my tested API route, so that the route actually sends the request, gets the response and returns it. How do I go about using TestClient along with requests_mock.Mocker? Can I somehow extend TestClient's http matchers or should I do it differently? Below is an example of a test that is not working for me:

queue.py

import fastapi
import requests

router = fastapi.APIRouter()
@router.get('/queue/')
def queue_messages():
    return requests.get('http://test-url/stats').json()

conftest.py

import fastapi
import pytest
import starlette.testclient

from source import queue

@pytest.fixture(scope='function')
def test_server():
    app = fastapi.FastAPI()

    app.include_router(queue.router, prefix='/api')

    return starlette.testclient.TestClient(app)

test_queue.py

def test_queue_get(requests_mock, test_server):
    stats = { ... }
    requests_mock.get('http://test-url/stats', json=stats)

    response = test_server.get('/api/queue/')
    assert response.json() == {'messages': 45}

[tomchristie]

I'm afraid I'm not able to help out with this - you'll need to dig into it yourself.

[em92]

# conftest.py
@fixture
def mock_requests_get(monkeypatch):
    def wrapped(return_value=None, side_effect=None):
        def what_to_return(*args, **kwargs):
            if side_effect:
                raise side_effect  # pragma: nocover
            return return_value

        monkeypatch.setattr("requests.get", what_to_return)

return wrapped

# test_something.py

def test_something_useful(mock_requests_get):
    mock_requests_get(MyFakeResponse())
    # ....

Real examples are here: https://github.com/em92/quakelive-local-ratings/blob/7e1fd1432d36dd4912ce3d8db4efee1168cdb05e/tests/test_balance_api.py#L191-L223

[wadamek65]

Thanks for the response @em92 , but this heavy mocking is exactly what I would like to avoid. Not only this would have less testing value but will require heavy refactors on all tests. Is there really no way to make starlette's TestClient play nice with requests_mock? And if there isn't would it be fine to create a PR and implement this functionality?

[tomchristie]

Presumably requests_mock has some way of specifying which session instance you want to mock.

(So create a global session instance for your endpoints to use and only attach the mocking to that instance.)

[buotex]

Nowadays it's easy to do: requests_mock supports an argument real_http. Setting this to True, all urls that you haven't mocked will still request the original URL, which in this case will be TestClient implementation.

[hmajid2301]

@buotex I was able to getting the following working thanks

with requests_mock.Mocker(real_http=True) as m:
    m.post(
        "http://localhost:3001/services/oauth2/token",
        json={"access_token": "a_random_token"},
    )
    response = client.post("/order/", json=data)

Where the client is a ASGI2App which makes test api requests on your behalf and part of that makes a request to an external client. This all seems to work :).

[cbensimon]

It works fine for me (on FastAPI) even without real_http :

conftest.py

@pytest.fixture(scope='session')
def client():
    with requests_mock.Mocker() as rm:
        rm.get('https://domain.com/path', json={})
        with TestClient(app) as c:
            yield c

test_app.py

def test_route(client):
    res = client.get('/route')
    assert res.status_code == 200

Application code that uses requests goes through the mocker, but TestClient calls don't !

[four43]

It works fine for me (on FastAPI) even without real_http :

This doesn't seem to work anymore due to the TestClient using a base_url of http://testserver? I keep getting

> requests_mock.exceptions.NoMockAddress: No mock address: GET http://testserver/route

../../../.venv/lib/python3.9/site-packages/requests_mock/adapter.py:261: NoMockAddress

[madispp]

I got it working by adding client.base_url to exceptions (real_http=True) for the requests_mock's fixture.
I keep my TestClient in the module scope but have to override requests_mock in its function scope:

@pytest.fixture(scope="module")
def module_client():
    with TestClient(app) as c:
        yield c

@pytest.fixture
def client(module_client, requests_mock):
    test_app_base_url_prefix_regex = re.compile(fr"{re.escape(module_client.base_url)}(/.*)?")
    requests_mock.register_uri(ANY, test_app_base_url_prefix_regex, real_http=True)
    return module_client

[felix-hilden]

For anyone ending up here through upgrading Starlette to 0.21 or FastAPI >= 0.87 after #1376 replacing Requests for HTTPX in the TestClient, and getting mocking failures along the lines of

No response can be found for POST request on http://testserver/...

There's an easy way of telling pytest_httpx to ignore the test server:

@pytest.fixture()
def non_mocked_hosts() -> list:
    return ["testserver"]

I know this thread is 3 years old 😄 sorry for the bump