proxy with authentication

[menta78]

At office we have a proxy server with authentication. The environment variables HTTP_PROXY and HTTPS_PROXY have value http://username:password@serveraddress:port .

Using version 0.15.0 of httplib2 (installed from conda-forge) the following code:

http = httplib2.Http()
http.request('http://www.google.com')

results in an AttributeError: 'bytes' object has no attribute 'encode'.
That's because httplib2 encodes the username and password after loading them from the environment variable, and passes them encoded to sock (line 1170 of file __init__.py that I'm using, at line 1174 of the latest commit a8716b1). And then socks tries to re-encode them.

This code works for me:
proxy_info = httplib2.proxy_info_from_environment()
proxy_info.proxy_user = proxy_info.proxy_user.decode()
proxy_info.proxy_pass = proxy_info.proxy_pass.decode()
http = httplib2.Http(proxy_info=proxy_info)
http.request('http://www.google.com')

So, I would suggest either to not encoding these variables in proxy_info_from_environment(), or to decode them before passing them to socks

[temoto]

@menta78 can't reproduce here. Please, show python -V and traceback.

[menta78]

$ python -V
Python 3.7.6

file bug.py:
"""
import httplib2
http = httplib2.Http()
http.request('http://www.google.com')
"""

traceback:
File "bug.py", line 4, in
http.request('http://www.google.com')
File "/STORAGE/usr/miniconda3/lib/python3.7/site-packages/httplib2/init.py", line 1976, in request
cachekey,
File "/STORAGE/usr/miniconda3/lib/python3.7/site-packages/httplib2/init.py", line 1640, in _request
conn, request_uri, method, body, headers
File "/STORAGE/usr/miniconda3/lib/python3.7/site-packages/httplib2/init.py", line 1546, in _conn_request
conn.connect()
File "/STORAGE/usr/miniconda3/lib/python3.7/site-packages/httplib2/init.py", line 1176, in connect
proxy_pass,
File "/STORAGE/usr/miniconda3/lib/python3.7/site-packages/socks.py", line 324, in setproxy
return self.set_proxy(*args, **kwargs)
File "/STORAGE/usr/miniconda3/lib/python3.7/site-packages/socks.py", line 318, in set_proxy
username.encode() if username else None,
AttributeError: 'bytes' object has no attribute 'encode'

[menta78]

If the proxy server does not require authentication the code works fine

[temoto]

Thanks, it didn't reproduce because failing line is not in httplib2 source. But the problem is very clear now.

[temoto]

Could you also share where this site-packages/socks.py comes from?

Something like pip freeze |fgrep -i socks

[temoto]

Nevermind, I found it's pysocks.

[temoto]

@menta78 fix is merged in master fa9a3bb

Release machinery is awoken too.

[temoto]

Please check if it works and I wasn't too fast to merge it?

[menta78]

now it works for me.

Many thanks!

[temoto]

Released as v0.16.0
Thank you too.