check max_content_length
consistently
#2620
Merged
+324
−364
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
max_content_length
is now checked consistently for anything that accessesrequest.stream
, such asrequest.data
,request.form
, andrequest.json
. The check is done inget_input_stream
.If
request.max_content_length
is set, it is checked immediately when accessingrequest.stream
, and raises a 413 Content Too Large error (RequestEntityTooLarge
). IfContent-Length
is not set (a streaming request),max_content_length
is used inLimitedStream
so that it may raise once the limit is met.LimitedStream
now implementsio.RawIOBase
instead of the even more basicio.IOBase
. Onlyreadinto
needs to be implemented,read
,readline
, andreadlines
have default C implementations based on that. The defaultreadall
required an extra read to check for EOF, so that had to be implemented as well.LimitedStream
has a newis_max
parameter to distinguish whether the limit is a client-reportedContent-Length
or an application-setmax_content_length
. If the limit is a maximum, reading from an exhausted stream raises a 413 error instead of returning empty. Reading empty does not raise aClientDisconnected
error, since there's no way to tell if the client stopped sending because the stream was done or because it really disconnected.The
readlines
implementation handles thesize
parameter correctly now. It reads at least one line, but stops reading more lines if the size was reached while reading the last line. Therefore, more thansize
bytes may be read. It is still safe, since each read goes throughreadinto
, so it will still be limited.fixes #1513
fixes #2620
fixes #690
It is not possible for the WSGI application to safely exhaust the input stream. It's up to the WSGI/HTTP server which handles the socket, or the client application which handles what happens when the server closes the input stream. Therefore it's expected that the client might show a "connection reset" failure rather than a 413 error message.
All "exhaust input" handling is removed from the application side of Werkzeug. It now always raises an error once
Content-Length
ormax_content_length
is reached, and will not read past that. However, the development server will now continue to read up to 10GB, 10MB at a time, up to 1000 reads, for 10 seconds. This allows the client to see the 413 error in all but extreme cases. The development server is already not appropriate for production, so adding this does not seem unreasonable.The server is still not able to handle HTTP/1.1 keep-alive connections. It was too complicated to handle exhausting the input while not accidentally reading the next request line.
I really think we should take all these things we've been adding on top of
http.server
and push them back upstream, if anyone wants to work on that.