Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bugfix: wsgi.file_wrapper should pass-through seek/tell attrs #363

Merged
merged 3 commits into from Jan 17, 2022

Conversation

digitalresistor
Copy link
Member

When the application author uses wsgi.file_wrapper we want to allow downstream WSGI middleware to seek that file if the underlying file supports it, rather than needing to iterate over the full file to get to the part that the WSGI middleware wants to start at.

Closes #359

@mmerickel mmerickel merged commit e828cd5 into master Jan 17, 2022
@mmerickel mmerickel deleted the bugfix/wsgi-file-wrapper branch January 17, 2022 23:26
This was referenced Mar 6, 2022
kraj pushed a commit to YoeDistro/meta-openembedded that referenced this pull request Mar 7, 2022
Changelog:
=========
Python Version Support
----------------------
- Python 3.6 is no longer supported by Waitress
- Python 3.10 is fully supported by Waitress

Bugfix
-------
- "wsgi.file_wrapper" now sets the "seekable", "seek", and "tell"
  attributes from the underlying file if the underlying file is seekable. This
  allows WSGI middleware to implement things like range requests for example

  See Pylons/waitress#359 and
  Pylons/waitress#363

- In Python 3 "OSError" is no longer subscriptable, this caused failures on
  Windows attempting to loop to find an socket that would work for use in the
  trigger.

  See Pylons/waitress#361

- Fixed an issue whereby "BytesIO" objects were not properly closed, and
  thereby would not get cleaned up until garbage collection would get around to
  it.

  This led to potential for random memory spikes/memory issues, see
  Pylons/waitress#358 and
  Pylons/waitress#357 .

Features
--------
- When the WSGI app starts sending data to the output buffer, we now attempt to
  send data directly to the socket. This avoids needing to wake up the main
  thread to start sending data. Allowing faster transmission of the first byte.
  See Pylons/waitress#364

- Add REQUEST_URI to the WSGI environment.

  REQUEST_URI is similar to "request_uri" in nginx. It is a string that
  contains the request path before separating the query string and
  decoding "%"-escaped characters.

Signed-off-by: Wang Mingyu <wangmy@fujitsu.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
kraj pushed a commit to YoeDistro/meta-openembedded that referenced this pull request Mar 7, 2022
Changelog:
=========
Python Version Support
----------------------
- Python 3.6 is no longer supported by Waitress
- Python 3.10 is fully supported by Waitress

Bugfix
-------
- "wsgi.file_wrapper" now sets the "seekable", "seek", and "tell"
  attributes from the underlying file if the underlying file is seekable. This
  allows WSGI middleware to implement things like range requests for example

  See Pylons/waitress#359 and
  Pylons/waitress#363

- In Python 3 "OSError" is no longer subscriptable, this caused failures on
  Windows attempting to loop to find an socket that would work for use in the
  trigger.

  See Pylons/waitress#361

- Fixed an issue whereby "BytesIO" objects were not properly closed, and
  thereby would not get cleaned up until garbage collection would get around to
  it.

  This led to potential for random memory spikes/memory issues, see
  Pylons/waitress#358 and
  Pylons/waitress#357 .

Features
--------
- When the WSGI app starts sending data to the output buffer, we now attempt to
  send data directly to the socket. This avoids needing to wake up the main
  thread to start sending data. Allowing faster transmission of the first byte.
  See Pylons/waitress#364

- Add REQUEST_URI to the WSGI environment.

  REQUEST_URI is similar to "request_uri" in nginx. It is a string that
  contains the request path before separating the query string and
  decoding "%"-escaped characters.

Signed-off-by: Wang Mingyu <wangmy@fujitsu.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
kraj pushed a commit to YoeDistro/meta-openembedded that referenced this pull request Mar 7, 2022
Changelog:
=========
Python Version Support
----------------------
- Python 3.6 is no longer supported by Waitress
- Python 3.10 is fully supported by Waitress

Bugfix
-------
- "wsgi.file_wrapper" now sets the "seekable", "seek", and "tell"
  attributes from the underlying file if the underlying file is seekable. This
  allows WSGI middleware to implement things like range requests for example

  See Pylons/waitress#359 and
  Pylons/waitress#363

- In Python 3 "OSError" is no longer subscriptable, this caused failures on
  Windows attempting to loop to find an socket that would work for use in the
  trigger.

  See Pylons/waitress#361

- Fixed an issue whereby "BytesIO" objects were not properly closed, and
  thereby would not get cleaned up until garbage collection would get around to
  it.

  This led to potential for random memory spikes/memory issues, see
  Pylons/waitress#358 and
  Pylons/waitress#357 .

Features
--------
- When the WSGI app starts sending data to the output buffer, we now attempt to
  send data directly to the socket. This avoids needing to wake up the main
  thread to start sending data. Allowing faster transmission of the first byte.
  See Pylons/waitress#364

- Add REQUEST_URI to the WSGI environment.

  REQUEST_URI is similar to "request_uri" in nginx. It is a string that
  contains the request path before separating the query string and
  decoding "%"-escaped characters.

Signed-off-by: Wang Mingyu <wangmy@fujitsu.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
kraj pushed a commit to YoeDistro/meta-openembedded that referenced this pull request Mar 8, 2022
Changelog:
=========
Python Version Support
----------------------
- Python 3.6 is no longer supported by Waitress
- Python 3.10 is fully supported by Waitress

Bugfix
-------
- "wsgi.file_wrapper" now sets the "seekable", "seek", and "tell"
  attributes from the underlying file if the underlying file is seekable. This
  allows WSGI middleware to implement things like range requests for example

  See Pylons/waitress#359 and
  Pylons/waitress#363

- In Python 3 "OSError" is no longer subscriptable, this caused failures on
  Windows attempting to loop to find an socket that would work for use in the
  trigger.

  See Pylons/waitress#361

- Fixed an issue whereby "BytesIO" objects were not properly closed, and
  thereby would not get cleaned up until garbage collection would get around to
  it.

  This led to potential for random memory spikes/memory issues, see
  Pylons/waitress#358 and
  Pylons/waitress#357 .

Features
--------
- When the WSGI app starts sending data to the output buffer, we now attempt to
  send data directly to the socket. This avoids needing to wake up the main
  thread to start sending data. Allowing faster transmission of the first byte.
  See Pylons/waitress#364

- Add REQUEST_URI to the WSGI environment.

  REQUEST_URI is similar to "request_uri" in nginx. It is a string that
  contains the request path before separating the query string and
  decoding "%"-escaped characters.

Signed-off-by: Wang Mingyu <wangmy@fujitsu.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
kraj pushed a commit to YoeDistro/meta-openembedded that referenced this pull request Mar 8, 2022
Changelog:
=========
Python Version Support
----------------------
- Python 3.6 is no longer supported by Waitress
- Python 3.10 is fully supported by Waitress

Bugfix
-------
- "wsgi.file_wrapper" now sets the "seekable", "seek", and "tell"
  attributes from the underlying file if the underlying file is seekable. This
  allows WSGI middleware to implement things like range requests for example

  See Pylons/waitress#359 and
  Pylons/waitress#363

- In Python 3 "OSError" is no longer subscriptable, this caused failures on
  Windows attempting to loop to find an socket that would work for use in the
  trigger.

  See Pylons/waitress#361

- Fixed an issue whereby "BytesIO" objects were not properly closed, and
  thereby would not get cleaned up until garbage collection would get around to
  it.

  This led to potential for random memory spikes/memory issues, see
  Pylons/waitress#358 and
  Pylons/waitress#357 .

Features
--------
- When the WSGI app starts sending data to the output buffer, we now attempt to
  send data directly to the socket. This avoids needing to wake up the main
  thread to start sending data. Allowing faster transmission of the first byte.
  See Pylons/waitress#364

- Add REQUEST_URI to the WSGI environment.

  REQUEST_URI is similar to "request_uri" in nginx. It is a string that
  contains the request path before separating the query string and
  decoding "%"-escaped characters.

Signed-off-by: Wang Mingyu <wangmy@fujitsu.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
kraj pushed a commit to YoeDistro/meta-openembedded that referenced this pull request Mar 9, 2022
Changelog:
=========
Python Version Support
----------------------
- Python 3.6 is no longer supported by Waitress
- Python 3.10 is fully supported by Waitress

Bugfix
-------
- "wsgi.file_wrapper" now sets the "seekable", "seek", and "tell"
  attributes from the underlying file if the underlying file is seekable. This
  allows WSGI middleware to implement things like range requests for example

  See Pylons/waitress#359 and
  Pylons/waitress#363

- In Python 3 "OSError" is no longer subscriptable, this caused failures on
  Windows attempting to loop to find an socket that would work for use in the
  trigger.

  See Pylons/waitress#361

- Fixed an issue whereby "BytesIO" objects were not properly closed, and
  thereby would not get cleaned up until garbage collection would get around to
  it.

  This led to potential for random memory spikes/memory issues, see
  Pylons/waitress#358 and
  Pylons/waitress#357 .

Features
--------
- When the WSGI app starts sending data to the output buffer, we now attempt to
  send data directly to the socket. This avoids needing to wake up the main
  thread to start sending data. Allowing faster transmission of the first byte.
  See Pylons/waitress#364

- Add REQUEST_URI to the WSGI environment.

  REQUEST_URI is similar to "request_uri" in nginx. It is a string that
  contains the request path before separating the query string and
  decoding "%"-escaped characters.

Signed-off-by: Wang Mingyu <wangmy@fujitsu.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
halstead pushed a commit to openembedded/meta-openembedded that referenced this pull request Mar 9, 2022
Changelog:
=========
Python Version Support
----------------------
- Python 3.6 is no longer supported by Waitress
- Python 3.10 is fully supported by Waitress

Bugfix
-------
- "wsgi.file_wrapper" now sets the "seekable", "seek", and "tell"
  attributes from the underlying file if the underlying file is seekable. This
  allows WSGI middleware to implement things like range requests for example

  See Pylons/waitress#359 and
  Pylons/waitress#363

- In Python 3 "OSError" is no longer subscriptable, this caused failures on
  Windows attempting to loop to find an socket that would work for use in the
  trigger.

  See Pylons/waitress#361

- Fixed an issue whereby "BytesIO" objects were not properly closed, and
  thereby would not get cleaned up until garbage collection would get around to
  it.

  This led to potential for random memory spikes/memory issues, see
  Pylons/waitress#358 and
  Pylons/waitress#357 .

Features
--------
- When the WSGI app starts sending data to the output buffer, we now attempt to
  send data directly to the socket. This avoids needing to wake up the main
  thread to start sending data. Allowing faster transmission of the first byte.
  See Pylons/waitress#364

- Add REQUEST_URI to the WSGI environment.

  REQUEST_URI is similar to "request_uri" in nginx. It is a string that
  contains the request path before separating the query string and
  decoding "%"-escaped characters.

Signed-off-by: Wang Mingyu <wangmy@fujitsu.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com>
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Jan 28, 2024
2.1.2
-----

Bugfix
~~~~~~

- When expose_tracebacks is enabled waitress would fail to properly encode
  unicode thereby causing another error during error handling. See
  Pylons/waitress#378

- Header length checking had a calculation that was done incorrectly when the
  data was received across multple socket reads. This calculation has been
  corrected, and no longer will Waitress send back a 413 Request Entity Too
  Large. See Pylons/waitress#376

Security Bugfix
~~~~~~~~~~~~~~~

- in 2.1.0 a new feature was introduced that allowed the WSGI thread to start
  sending data to the socket. However this introduced a race condition whereby
  a socket may be closed in the sending thread while the main thread is about
  to call select() therey causing the entire application to be taken down.
  Waitress will no longer close the socket in the WSGI thread, instead waking
  up the main thread to cleanup. See Pylons/waitress#377

2.1.1
-----

Security Bugfix
~~~~~~~~~~~~~~~

- Waitress now validates that chunked encoding extensions are valid, and don't
  contain invalid characters that are not allowed. They are still skipped/not
  processed, but if they contain invalid data we no longer continue in and
  return a 400 Bad Request. This stops potential HTTP desync/HTTP request
  smuggling. Thanks to Zhang Zeyu for reporting this issue. See
  GHSA-4f7p-27jc-3c36

- Waitress now validates that the chunk length is only valid hex digits when
  parsing chunked encoding, and values such as ``0x01`` and ``+01`` are no
  longer supported. This stops potential HTTP desync/HTTP request smuggling.
  Thanks to Zhang Zeyu for reporting this issue. See
  GHSA-4f7p-27jc-3c36

- Waitress now validates that the Content-Length sent by a remote contains only
  digits in accordance with RFC7230 and will return a 400 Bad Request when the
  Content-Length header contains invalid data, such as ``+10`` which would
  previously get parsed as ``10`` and accepted. This stops potential HTTP
  desync/HTTP request smuggling Thanks to Zhang Zeyu for reporting this issue. See
  GHSA-4f7p-27jc-3c36

2.1.0
-----

Python Version Support
~~~~~~~~~~~~~~~~~~~~~~

- Python 3.6 is no longer supported by Waitress

- Python 3.10 is fully supported by Waitress

Bugfix
~~~~~~

- ``wsgi.file_wrapper`` now sets the ``seekable``, ``seek``, and ``tell``
  attributes from the underlying file if the underlying file is seekable. This
  allows WSGI middleware to implement things like range requests for example

  See Pylons/waitress#359 and
  Pylons/waitress#363

- In Python 3 ``OSError`` is no longer subscriptable, this caused failures on
  Windows attempting to loop to find an socket that would work for use in the
  trigger.

  See Pylons/waitress#361

- Fixed an issue whereby ``BytesIO`` objects were not properly closed, and
  thereby would not get cleaned up until garbage collection would get around to
  it.

  This led to potential for random memory spikes/memory issues, see
  Pylons/waitress#358 and
  Pylons/waitress#357 .

  With thanks to Florian Schulze for testing/vaidating this fix!

Features
~~~~~~~~

- When the WSGI app starts sending data to the output buffer, we now attempt to
  send data directly to the socket. This avoids needing to wake up the main
  thread to start sending data. Allowing faster transmission of the first byte.
  See Pylons/waitress#364

  With thanks to Michael Merickel for being a great rubber ducky!

- Add REQUEST_URI to the WSGI environment.

  REQUEST_URI is similar to ``request_uri`` in nginx. It is a string that
  contains the request path before separating the query string and
  decoding ``%``-escaped characters.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Super slow 206 range requests
2 participants