idna.core.InvalidCodepoint and idna.core.IDNAError errors

[fxcoudert]

This URL loads with any browser, with cURL and wget, without error: https://earthlingsoft.net/UnicodeChecker/appcast.xml

But trying to reach it with Python requests leads to an error in urllib3:

>>> import requests
>>> requests.get("https://earthlingsoft.net/UnicodeChecker/appcast.xml")
Traceback (most recent call last):
  File "/Users/fx/anaconda3/lib/python3.6/site-packages/idna/core.py", line 271, in alabel
    ulabel(label)
  File "/Users/fx/anaconda3/lib/python3.6/site-packages/idna/core.py", line 311, in ulabel
    check_label(label)
  File "/Users/fx/anaconda3/lib/python3.6/site-packages/idna/core.py", line 261, in check_label
    raise InvalidCodepoint('Codepoint {0} at position {1} of {2} not allowed'.format(_unot(cp_value), pos+1, repr(label)))
idna.core.InvalidCodepoint: Codepoint U+2603 at position 1 of '☃' not allowed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/fx/anaconda3/lib/python3.6/site-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/Users/fx/anaconda3/lib/python3.6/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/fx/anaconda3/lib/python3.6/site-packages/requests/sessions.py", line 512, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/fx/anaconda3/lib/python3.6/site-packages/requests/sessions.py", line 622, in send
    r = adapter.send(request, **kwargs)
  File "/Users/fx/anaconda3/lib/python3.6/site-packages/requests/adapters.py", line 445, in send
    timeout=timeout
  File "/Users/fx/anaconda3/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/Users/fx/anaconda3/lib/python3.6/site-packages/urllib3/connectionpool.py", line 343, in _make_request
    self._validate_conn(conn)
  File "/Users/fx/anaconda3/lib/python3.6/site-packages/urllib3/connectionpool.py", line 849, in _validate_conn
    conn.connect()
  File "/Users/fx/anaconda3/lib/python3.6/site-packages/urllib3/connection.py", line 367, in connect
    cert = self.sock.getpeercert()
  File "/Users/fx/anaconda3/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py", line 351, in getpeercert
    'subjectAltName': get_subj_alt_name(x509)
  File "/Users/fx/anaconda3/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py", line 228, in get_subj_alt_name
    for name in ext.get_values_for_type(x509.DNSName)
  File "/Users/fx/anaconda3/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py", line 228, in <listcomp>
    for name in ext.get_values_for_type(x509.DNSName)
  File "/Users/fx/anaconda3/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py", line 181, in _dnsname_to_stdlib
    name = idna_encode(name)
  File "/Users/fx/anaconda3/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py", line 179, in idna_encode
    return idna.encode(name)
  File "/Users/fx/anaconda3/lib/python3.6/site-packages/idna/core.py", line 361, in encode
    s = alabel(label)
  File "/Users/fx/anaconda3/lib/python3.6/site-packages/idna/core.py", line 273, in alabel
    raise IDNAError('The label {0} is not a valid A-label'.format(label))
idna.core.IDNAError: The label b'xn--n3h' is not a valid A-label

I originally filed it with pyopenssl: pyca/pyopenssl#771

[sethmlarson]

I've got a patch that fixes the issue in #1406. Pending review from others. :)

[sethmlarson]

That patch landed in #1406. Closing this issue.

[fxcoudert]

Thanks @SethMichaelLarson!