New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add streaming multipart/form-data encoder #3361
Draft
alexwlchan
wants to merge
21
commits into
urllib3:main
Choose a base branch
from
alexwlchan:add-multipart-formdata-encoder-2024
base: main
Could not load branches
Branch not found: {{ refName }}
Could not load tags
Nothing to show
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Add streaming multipart/form-data encoder #3361
alexwlchan
wants to merge
21
commits into
urllib3:main
from
alexwlchan:add-multipart-formdata-encoder-2024
Commits on Jan 2, 2022
-
Add multipart/form-data streaming encoder
I initially implemented this for Requests in the requests-toolbelt but this was already pretty generic because Requests just passes file-like objects (which is how the streaming encoder behaves) directly to urllib3. All that needed to change was what we were relying on from the requests namespace and imports and such. This also adds the decoder in ther same breath because it's easier to ensure that's all working together properly in one and it all fits together nicely. One thing we _could_ do is consolidate a bunch of the logic too and make `encode_multipart_formdata` rely on the streaming encoder and call `getall` instead so that we don't have 2 implementations of the same logic.
Configuration menu - View commit details
-
Copy full SHA for c8c153e - Browse repository at this point
Copy the full SHA c8c153eView commit details
Commits on Jun 4, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 6c6117c - Browse repository at this point
Copy the full SHA 6c6117cView commit details -
Type annotations made it clear that a dangling comma was added by mistake, converting a value into a tuple.
Configuration menu - View commit details
-
Copy full SHA for b2f8061 - Browse repository at this point
Copy the full SHA b2f8061View commit details
Commits on Jun 12, 2022
-
Configuration menu - View commit details
-
Copy full SHA for ba64375 - Browse repository at this point
Copy the full SHA ba64375View commit details
Commits on Aug 19, 2022
-
Configuration menu - View commit details
-
Copy full SHA for c996550 - Browse repository at this point
Copy the full SHA c996550View commit details -
Revert back to using Union[str, bytes] for rfc2231
It's incorrect in the context of the streaming encoder changes.
Configuration menu - View commit details
-
Copy full SHA for 2ecc96d - Browse repository at this point
Copy the full SHA 2ecc96dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 06ba311 - Browse repository at this point
Copy the full SHA 06ba311View commit details
Commits on Mar 11, 2024
-
Merge remote-tracking branch 'origin/main' into add-multipart-formdat…
…a-encoder-2024 # Conflicts: # docs/conf.py # src/urllib3/connection.py # src/urllib3/fields.py # src/urllib3/filepost.py # test/with_dummyserver/test_connectionpool.py
Configuration menu - View commit details
-
Copy full SHA for be4633c - Browse repository at this point
Copy the full SHA be4633cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 591b0c3 - Browse repository at this point
Copy the full SHA 591b0c3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4329d0f - Browse repository at this point
Copy the full SHA 4329d0fView commit details -
Ensure we close the open files at the end of the test
Previously this test was failing with a warning that was being raised as an error: _____________________________________________ TestMultipartEncoder.test_regression_1 ______________________________________________ Traceback (most recent call last): File "/Users/alexwlchan/repos/urllib3/.nox/test-3-12/lib/python3.12/site-packages/pluggy/_callers.py", line 155, in _multicall teardown[0].send(outcome) File "/Users/alexwlchan/repos/urllib3/.nox/test-3-12/lib/python3.12/site-packages/_pytest/unraisableexception.py", line 88, in pytest_runtest_call yield from unraisable_exception_runtest_hook() File "/Users/alexwlchan/repos/urllib3/.nox/test-3-12/lib/python3.12/site-packages/_pytest/unraisableexception.py", line 78, in unraisable_exception_runtest_hook warnings.warn(pytest.PytestUnraisableExceptionWarning(msg)) pytest.PytestUnraisableExceptionWarning: Exception ignored in: <_io.FileIO [closed]> Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/unittest/case.py", line 589, in _callTestMethod if method() is not None: ^^^^^^^^ ResourceWarning: unclosed file <_io.BufferedReader name='/Users/alexwlchan/repos/urllib3/test/multipart/test_encoder.py'> During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/alexwlchan/repos/urllib3/.nox/test-3-12/lib/python3.12/site-packages/_pytest/runner.py", line 341, in from_call result: Optional[TResult] = func() ^^^^^^ File "/Users/alexwlchan/repos/urllib3/.nox/test-3-12/lib/python3.12/site-packages/_pytest/runner.py", line 262, in <lambda> lambda: ihook(item=item, **kwds), when=when, reraise=reraise ^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/alexwlchan/repos/urllib3/.nox/test-3-12/lib/python3.12/site-packages/pluggy/_hooks.py", line 501, in __call__ return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/alexwlchan/repos/urllib3/.nox/test-3-12/lib/python3.12/site-packages/pluggy/_manager.py", line 119, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/alexwlchan/repos/urllib3/.nox/test-3-12/lib/python3.12/site-packages/pluggy/_callers.py", line 159, in _multicall _warn_teardown_exception(hook_name, teardown[1], e) File "/Users/alexwlchan/repos/urllib3/.nox/test-3-12/lib/python3.12/site-packages/pluggy/_callers.py", line 49, in _warn_teardown_exception warnings.warn(PluggyTeardownRaisedWarning(msg), stacklevel=5) pluggy.PluggyTeardownRaisedWarning: A plugin raised an exception during an old-style hookwrapper teardown. Plugin: unraisableexception, Hook: pytest_runtest_call PytestUnraisableExceptionWarning: Exception ignored in: <_io.FileIO [closed]> Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/unittest/case.py", line 589, in _callTestMethod if method() is not None: ^^^^^^^^ ResourceWarning: unclosed file <_io.BufferedReader name='/Users/alexwlchan/repos/urllib3/test/multipart/test_encoder.py'>
Configuration menu - View commit details
-
Copy full SHA for 84c1773 - Browse repository at this point
Copy the full SHA 84c1773View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8dc4245 - Browse repository at this point
Copy the full SHA 8dc4245View commit details -
Configuration menu - View commit details
-
Copy full SHA for 786613e - Browse repository at this point
Copy the full SHA 786613eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 40ea533 - Browse repository at this point
Copy the full SHA 40ea533View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5dc22c2 - Browse repository at this point
Copy the full SHA 5dc22c2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0814a9a - Browse repository at this point
Copy the full SHA 0814a9aView commit details
Commits on Mar 16, 2024
-
Configuration menu - View commit details
-
Copy full SHA for d14f34d - Browse repository at this point
Copy the full SHA d14f34dView commit details -
Configuration menu - View commit details
-
Copy full SHA for e0d588f - Browse repository at this point
Copy the full SHA e0d588fView commit details
Commits on Mar 24, 2024
-
Configuration menu - View commit details
-
Copy full SHA for b0be345 - Browse repository at this point
Copy the full SHA b0be345View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6a8a3a6 - Browse repository at this point
Copy the full SHA 6a8a3a6View commit details -
Configuration menu - View commit details
-
Copy full SHA for 581f906 - Browse repository at this point
Copy the full SHA 581f906View commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.