Skip to content

Commit

Permalink
[s3] Fix newlines test for python 3.11 and above (#1400)
Browse files Browse the repository at this point in the history
SpooledTemporaryFile introduced new changes in 3.11 (https://docs.python.org/3/library/tempfile.html)
It now fully implements the io.BufferedIOBase and io.TextIOBase abstract base classes allowing the file
to be readable in the mode that it was specified (without accessing the underlying ._file object).

It seems like wrapping the underlying ._file is forbidden for versions 3.11 and higher (results in the ValueError)
Having two have separate methods for different versions isn't the prettiest fix but should suffice.
  • Loading branch information
skim618 committed May 10, 2024
1 parent 1f9e246 commit 0965403
Showing 1 changed file with 10 additions and 1 deletion.
11 changes: 10 additions & 1 deletion storages/backends/s3.py
Expand Up @@ -185,7 +185,16 @@ def _get_file(self):
if self._storage.gzip and self.obj.content_encoding == "gzip":
self._file = self._decompress_file(mode=self._mode, file=self._file)
elif "b" not in self._mode:
self._file = io.TextIOWrapper(self._file._file, encoding="utf-8")
if hasattr(self._file, "readable"):
# For versions > Python 3.10 compatibility
# See SpooledTemporaryFile changes in 3.11 (https://docs.python.org/3/library/tempfile.html)
# Now fully implements the io.BufferedIOBase and io.TextIOBase abstract base classes allowing the file
# to be readable in the mode that it was specified (without accessing the underlying _file object).
# In this case, we need to wrap the file in a TextIOWrapper to ensure that the file is read as a text file.
self._file = io.TextIOWrapper(self._file, encoding="utf-8")
else:
# For versions <= Python 3.10 compatibility
self._file = io.TextIOWrapper(self._file._file, encoding="utf-8")
self._closed = False
return self._file

Expand Down

0 comments on commit 0965403

Please sign in to comment.