Skip to content

Commit

Permalink
Avoid zip extract racing condition by using read+write instead extract
Browse files Browse the repository at this point in the history
Extract also creates the folder hierarchy, however we do not need that,
the file itself being extracted to a temporary folder is good enough.
Instead we read the content of the zip and then write it. The write is
not locked but it's OK to update the same file multiple times given the
update operation will not alter the content of the file. By not creating
the folder hierarchy (default via extract) we no longer can run into the
problem of two parallel extracts both trying to create the folder
hierarchy without exists ok flag, and one must fail.

Signed-off-by: Bernát Gábor <bgabor8@bloomberg.net>
  • Loading branch information
gaborbernat committed Dec 24, 2020
1 parent c2b307d commit 7a1945f
Showing 1 changed file with 4 additions and 3 deletions.
7 changes: 4 additions & 3 deletions requests/utils.py
Expand Up @@ -256,10 +256,11 @@ def extract_zipped_paths(path):

# we have a valid zip archive and a valid member of that archive
tmp = tempfile.gettempdir()
extracted_path = os.path.join(tmp, *member.split('/'))
extracted_path = os.path.join(tmp, member.split('/')[-1])
if not os.path.exists(extracted_path):
extracted_path = zip_file.extract(member, path=tmp)

# use read + write to avoid the creating nested folders, we only want the file, avoids mkdir racing condition
with open(extracted_path, 'wb') as file_handler:
file_handler.write(zip_file.read(member))
return extracted_path


Expand Down

0 comments on commit 7a1945f

Please sign in to comment.