Connection aborted after HttpParserInvalidMethodError when making consecutive POST requests

[vogre]

Checklist

• The bug is reproducible against the latest release or master.
• There are no similar issues or pull requests to fix it yet.

Describe the bug

After making a few POST requests, one of the requests ends with the connection aborted. The server logs a log about an invalid HTTP method, however only POST method is consistently attempted.

Steps to reproduce the bug

The bug is reproduced when making multiple requests.posts against a service running with latest uvicorn

Unfortunately I was not able to create a reliable reproduction, as running locally I seemed to make successful requests

Expected behavior

All the requests used to pass in sequence without errors

Actual behavior

One of the requests, after an inconsistent number of successful requests, fails with a connection error after sending the request. The server logs

Debugging material

Request aborted

  File "/usr/local/lib/python3.10/site-packages/requests/api.py", line 117, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 529, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 645, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 501, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

Server warning (uvicorn.error).

"Invalid HTTP request received."
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 132, in data_received
    self.parser.feed_data(data)
  File "httptools/parser/parser.pyx", line 212, in httptools.parser.parser.HttpParser.feed_data
httptools.parser.errors.HttpParserInvalidMethodError: Invalid method encountered

Environment

• Running uvicorn 0.17.1 with CPython 3.10.1 on Linux
• Encountered with Kubernetes services, without reverse proxy/ingress

Additional context

Since requests library is using a keepalive for the requests, a suspect change is the recent #1332, and reverting uvicorn version seems to make the problem go away.

[UKnowWhoIm]

I had the same issue with PATCH too and it wasn't consecutive PATCH/POST requests only, there were consecutive GET and OPTIONS requests between these invalid requests which worked.

Since requests library is using a keepalive for the requests, a suspect change is the recent #1332, and reverting uvicorn version seems to make the problem go away.

This seemed to fix it

[smeinecke]

I have the same problem with 0.17.0 and 0.17.1 for PUT/POST calls where the body data exceed 64kb and the request does not have an Except: 100-continue header (which is done by curl, but not by python requests library). Downgrade to 0.16.0 fixed the problem.

Can be reproduced by a simple application with one endpoint and following curl call:
curl -v -H 'Expect:' --data-binary '@test.bin' 'http://127.0.0.1:8000/test'
test.bin was filled with 80kb of random data.

[narolski]

I can also confirm that this issue occurs for uvicorn=0.17.1.

Tried to add the Expect: 100-continue header per @smeinecke's suggestion, but that did not resolve the issue.

Can confirm that the issue occurs for requests with larger body data.

[jamescw19]

I've just encountered this problem. Again, uvicorn==0.17.1 and occurs with >65kb data.
I'm also still seeing this error with the Expect: 100-continue header.

My service is deployed on Kubernetes, and I've found when I allocate >3 CPUs, the issue no longer occurs and I can raise the request body size, finally hitting the 413 threshold before ever seeing this HttpParserInvalidMethodError

[mromagnoli]

I'm having this very same issue for uvicorn==0.17.1, but sadly I can't reproduce it easily, it seems "random" but I haven't detected the special conditions to reproduce it. Downgrading to 0.16.0 solves it.
This happened in production but also in my local env.
In other posts, I've seen people suggesting to check and change the URL from https to http if the server does not manages HTTPS requests, but this is not my case since it's already http

[motybz]

I also faced this issue, I'm using conda package uvicorn-standard=0.17.1.
uvicorn-standard=0.17.0.post1 solved the problem

[tomchristie]

I assume the sensible thing here for now would be to revert #1332 and issue a new release. Does that seem like a reasonable course of action?

[Kludex]

Not really. That PR was solving a memory leak.

What we can do is:

• Revert Fix ensure http connection is closed #1332
• Go with Revert stream interface #1307

I can check if I can solve this issue this afternoon without having to do the above. Although, I think we should go with #1307 anyway... For the reasons I've already stated on that PR.

The way I see it, reverting #1332 and only that is not an option here.

[Kludex]

How to reproduce the bug:

from httpx import Client

with Client() as client:
    client.post("http://localhost:8000/test", content="x" * 100000)

Uvicorn command: python -m uvicorn main:app --http httptools

[Kludex]

The solution I propose is to revert both. PRs were created:

• Revert "Fix ensure http connection is closed" #1354
• Revert stream interface #1355 (branch on top of the above)

Both memory leak and this issue are solved with my proposal.

[tomchristie]

Now resolved in 0.17.2 - #1356