Google cloud storage not retrying file downloads

[and3rson]

We are randomly getting this sort of errors when trying to download a file:

  File "./utils/tools/fingerprint.py", line 11, in generate_fingerprint
    img_file.seek(0)                                       
  File "/usr/local/lib/python3.7/site-packages/django/core/files/utils.py", line 20, in <lambda>
    seek = property(lambda self: self.file.seek)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/fields/files.py", line 43, in _get_file
    self._file = self.storage.open(self.name, 'rb')
  File "/usr/local/lib/python3.7/site-packages/django/core/files/storage.py", line 33, in open
    return self._open(name, mode)
  File "/usr/local/lib/python3.7/site-packages/storages/backends/gcloud.py", line 164, in _open
    file_object = GoogleCloudFile(name, mode, self)
  File "/usr/local/lib/python3.7/site-packages/storages/backends/gcloud.py", line 32, in __init__
    self.blob = storage.bucket.get_blob(name)
  File "/usr/local/lib/python3.7/site-packages/google/cloud/storage/bucket.py", line 380, in get_blob
    _target_object=blob,                                   
  File "/usr/local/lib/python3.7/site-packages/google/cloud/_http.py", line 290, in api_request
    headers=headers, target_object=_target_object)
  File "/usr/local/lib/python3.7/site-packages/google/cloud/_http.py", line 183, in _make_request
    return self._do_request(method, url, headers, data, target_object)
  File "/usr/local/lib/python3.7/site-packages/google/cloud/_http.py", line 212, in _do_request
    url=url, method=method, headers=headers, data=data)
  File "/usr/local/lib/python3.7/site-packages/google/auth/transport/requests.py", line 208, in request
    method, url, data=data, headers=request_headers, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

It looks like django-storages gcloud adapter does not retry the attempt if it fails, and handling it manually everywhere would lead to a lot of boilerplate code (I was considering using this: googleapis/google-cloud-python#5879)

Any ways to fix this?

[jschneier]

This was added in #1156