requests.utils.super_len() can't cope with tarfile members

[dimbleby]

Expected Result

POSTing a large file that I have extracted from a tarfile. I expect that requests should be able to determine the length of that file, set the Content-Length header, and upload it efficiently.

Actual Result

requests fails to determine the file length, goes with Transfer-Encoding: chunked, and the upload is much slower than desired.

Reproduction Steps

Start with a small tarfile, say

echo foo > file1
tar cf tarfile.tar file1

Then use requests.utils.super_len to try and get the length of file1 in the tarfile:

#!/usr/bin/env python
import tarfile
import requests

tar = tarfile.open("tarfile.tar")
member = tar.extractfile("file1")
print(f"super_length of member is {requests.utils.super_len(member)}")

result:

Traceback (most recent call last):
  File "./demo.py", line 7, in <module>
    print(f"super_length of member is {requests.utils.super_len(member)}")
  File "/data/developer/.virtualenvs/req-tar/lib/python3.6/site-packages/requests/utils.py", line 119, in super_len
    fileno = o.fileno()
AttributeError: '_FileInFile' object has no attribute 'fileno'

The behavior of the member is quite strange here: hasattr(member, fileno) is true, but trying to call member.fileno() results in an AttributeError.

I expect that a sensible fix might be for this exception to catch a broader range of errors.

One can get the length of the member by seeking, so I expect that super_len() should then succeed.

System Information

$ python -m requests.help

{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.8"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.8"
  },
  "platform": {
    "release": "3.10.0-1062.1.1.el7.x86_64",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.22.0"
  },
  "system_ssl": {
    "version": "100020bf"
  },
  "urllib3": {
    "version": "1.25.6"
  },
  "using_pyopenssl": false
}

[tirkarthi]

I feel it should be raising unsupported exception when the underlying attribute doesn't have fileno. I have opened https://bugs.python.org/issue38572

[dimbleby]

Sounds reasonable, thanks - though obviously I hope that meanwhile we can get a workaround/fix here, per #5239.

[nateprewitt]

Resolved with #5239