SSL tests fail with uncaught SSL validation #173

jaraco · 2019-02-04T20:55:02Z

❓ I'm submitting a ...

🐞 bug report
🐣 feature request
❓ question about the decisions made in the repository

🐞 Describe the bug. What is the current behavior?
Running tests on macOS, two tests fail:

       2 failed
         - cheroot/test/test_ssl.py:192 test_tls_client_auth[VerifyMode.CERT_OPTIONAL-False-localhost-pyopenssl]
         - cheroot/test/test_ssl.py:192 test_tls_client_auth[VerifyMode.CERT_REQUIRED-False-localhost-pyopenssl]

❓ What is the motivation / use case for changing the behavior?

Tests should pass on master.

💡 To Reproduce

Steps to reproduce the behavior:

Run tox -r on macOS

💡 Expected behavior

Tests should pass.

📋 Details

Failures look like:

――――――――――――――――――――――――――――――――――――――――――――――――――――――― test_tls_client_auth[VerifyMode.CERT_OPTIONAL-False-localhost-pyopenssl] ――――――――――――――――――――――――――――――――――――――――――――――――――――――――

mocker = <pytest_mock.MockFixture object at 0x103995ba8>, tls_http_server = <generator object tls_http_server.<locals>.start_srv at 0x10392c228>, adapter_type = 'pyopenssl'
ca = <trustme.CA object at 0x103995fd0>, tls_certificate = <trustme.LeafCert object at 0x10396bb70>
tls_certificate_chain_pem_path = '/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/tmpg0wjfzji.pem'
tls_certificate_private_key_pem_path = '/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/tmpii4yp07w.pem'
tls_ca_certificate_pem_path = '/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/tmpyv5h7s8l.pem', is_trusted_cert = False, tls_client_identity = 'localhost'
tls_verify_mode = <VerifyMode.CERT_OPTIONAL: 1>

    @pytest.mark.parametrize(
        'adapter_type',
        (
            'builtin',
            'pyopenssl',
        ),
    )
    @pytest.mark.parametrize(
        'is_trusted_cert,tls_client_identity',
        (
            (True, 'localhost'), (True, '127.0.0.1'),
            (True, '*.localhost'), (True, 'not_localhost'),
            (False, 'localhost'),
        ),
    )
    @pytest.mark.parametrize(
        'tls_verify_mode',
        (
            ssl.CERT_NONE,  # server shouldn't validate client cert
            ssl.CERT_OPTIONAL,  # same as CERT_REQUIRED in client mode, don't use
            ssl.CERT_REQUIRED,  # server should validate if client cert CA is OK
        ),
    )
    def test_tls_client_auth(
        # FIXME: remove twisted logic, separate tests
        mocker,
        tls_http_server, adapter_type,
        ca,
        tls_certificate,
        tls_certificate_chain_pem_path,
        tls_certificate_private_key_pem_path,
        tls_ca_certificate_pem_path,
        is_trusted_cert, tls_client_identity,
        tls_verify_mode,
    ):
        """Verify that client TLS certificate auth works correctly."""
        test_cert_rejection = (
            tls_verify_mode != ssl.CERT_NONE
            and not is_trusted_cert
        )
        interface, _host, port = _get_conn_data(ANY_INTERFACE_IPV4)
    
        client_cert_root_ca = ca if is_trusted_cert else trustme.CA()
        with mocker.mock_module.patch(
            'idna.core.ulabel',
            return_value=ntob(tls_client_identity),
        ):
            client_cert = client_cert_root_ca.issue_server_cert(
                # FIXME: change to issue_cert once new trustme is out
                ntou(tls_client_identity),
            )
            del client_cert_root_ca
    
        with client_cert.private_key_and_cert_chain_pem.tempfile() as cl_pem:
            tls_adapter_cls = get_ssl_adapter_class(name=adapter_type)
            tls_adapter = tls_adapter_cls(
                tls_certificate_chain_pem_path,
                tls_certificate_private_key_pem_path,
            )
            if adapter_type == 'pyopenssl':
                tls_adapter.context = tls_adapter.get_context()
                tls_adapter.context.set_verify(
                    _stdlib_to_openssl_verify[tls_verify_mode],
                    lambda conn, cert, errno, depth, preverify_ok: preverify_ok,
                )
            else:
                tls_adapter.context.verify_mode = tls_verify_mode
    
            ca.configure_trust(tls_adapter.context)
            tls_certificate.configure_cert(tls_adapter.context)
    
            tlshttpserver = tls_http_server.send(
                (
                    (interface, port),
                    tls_adapter,
                ),
            )
    
            interface, _host, port = _get_conn_data(tlshttpserver.bind_addr)
    
            make_https_request = functools.partial(
                requests.get,
                'https://' + interface + ':' + str(port) + '/',
    
                # Server TLS certificate verification:
                verify=tls_ca_certificate_pem_path,
    
                # Client TLS certificate verification:
                cert=cl_pem,
            )
    
            if not test_cert_rejection:
                resp = make_https_request()
                assert resp.status_code == 200
                assert resp.text == 'Hello world!'
                return
    
            with pytest.raises(requests.exceptions.SSLError) as ssl_err:
>               make_https_request()

cheroot/test/test_ssl.py:290: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.tox/python/lib/python3.7/site-packages/requests/api.py:75: in get
    return request('get', url, params=params, **kwargs)
.tox/python/lib/python3.7/site-packages/requests/api.py:60: in request
    return session.request(method=method, url=url, **kwargs)
.tox/python/lib/python3.7/site-packages/requests/sessions.py:533: in request
    resp = self.send(prep, **send_kwargs)
.tox/python/lib/python3.7/site-packages/requests/sessions.py:646: in send
    r = adapter.send(request, **kwargs)
.tox/python/lib/python3.7/site-packages/requests/adapters.py:449: in send
    timeout=timeout
.tox/python/lib/python3.7/site-packages/urllib3/connectionpool.py:600: in urlopen
    chunked=chunked)
.tox/python/lib/python3.7/site-packages/urllib3/connectionpool.py:384: in _make_request
    six.raise_from(e, None)
<string>:2: in raise_from
    ???
.tox/python/lib/python3.7/site-packages/urllib3/connectionpool.py:380: in _make_request
    httplib_response = conn.getresponse()
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py:1321: in getresponse
    response.begin()
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py:296: in begin
    version, status, reason = self._read_status()
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py:257: in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/socket.py:589: in readinto
    return self._sock.recv_into(b)
.tox/python/lib/python3.7/site-packages/urllib3/contrib/pyopenssl.py:294: in recv_into
    return self.connection.recv_into(*args, **kwargs)
.tox/python/lib/python3.7/site-packages/OpenSSL/SSL.py:1822: in recv_into
    self._raise_ssl_error(self._ssl, result)
.tox/python/lib/python3.7/site-packages/OpenSSL/SSL.py:1647: in _raise_ssl_error
    _raise_current_error()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

exception_type = <class 'OpenSSL.SSL.Error'>

    def exception_from_error_queue(exception_type):
        """
        Convert an OpenSSL library failure into a Python exception.
    
        When a call to the native OpenSSL library fails, this is usually signalled
        by the return value, and an error code is stored in an error queue
        associated with the current thread. The err library provides functions to
        obtain these error codes and textual error messages.
        """
        errors = []
    
        while True:
            error = lib.ERR_get_error()
            if error == 0:
                break
            errors.append((
                text(lib.ERR_lib_error_string(error)),
                text(lib.ERR_func_error_string(error)),
                text(lib.ERR_reason_error_string(error))))
    
>       raise exception_type(errors)
E       OpenSSL.SSL.Error: [('SSL routines', 'ssl3_read_bytes', 'tlsv1 alert unknown ca')]

.tox/python/lib/python3.7/site-packages/OpenSSL/_util.py:54: Error

 cheroot/test/test_ssl.py::test_tls_client_auth[VerifyMode.CERT_OPTIONAL-False-localhost-pyopenssl] ⨯                                                                      84% ████████▍ 
 cheroot/test/test_ssl.py::test_tls_client_auth[VerifyMode.CERT_REQUIRED-True-localhost-builtin] ✓                                                                         85% ████████▌ 
 cheroot/test/test_ssl.py::test_tls_client_auth[VerifyMode.CERT_REQUIRED-True-localhost-pyopenssl] ✓                                                                       86% ████████▋ 
 cheroot/test/test_ssl.py::test_tls_client_auth[VerifyMode.CERT_REQUIRED-True-127.0.0.1-builtin] ✓                                                                         87% ████████▋ 
 cheroot/test/test_ssl.py::test_tls_client_auth[VerifyMode.CERT_REQUIRED-True-127.0.0.1-pyopenssl] ✓                                                                       88% ████████▊ 
 cheroot/test/test_ssl.py::test_tls_client_auth[VerifyMode.CERT_REQUIRED-True-*.localhost-builtin] ✓                                                                       89% ████████▉ 
 cheroot/test/test_ssl.py::test_tls_client_auth[VerifyMode.CERT_REQUIRED-True-*.localhost-pyopenssl] ✓                                                                     90% ████████▉ 
 cheroot/test/test_ssl.py::test_tls_client_auth[VerifyMode.CERT_REQUIRED-True-not_localhost-builtin] ✓                                                                     90% █████████▏
 cheroot/test/test_ssl.py::test_tls_client_auth[VerifyMode.CERT_REQUIRED-True-not_localhost-pyopenssl] ✓                                                                   91% █████████▎
 cheroot/test/test_ssl.py::test_tls_client_auth[VerifyMode.CERT_REQUIRED-False-localhost-builtin] ✓                                                                        92% █████████▎

――――――――――――――――――――――――――――――――――――――――――――――――――――――― test_tls_client_auth[VerifyMode.CERT_REQUIRED-False-localhost-pyopenssl] ――――――――――――――――――――――――――――――――――――――――――――――――――――――――

mocker = <pytest_mock.MockFixture object at 0x10395c6d8>, tls_http_server = <generator object tls_http_server.<locals>.start_srv at 0x103a1ae58>, adapter_type = 'pyopenssl'
ca = <trustme.CA object at 0x103ee1160>, tls_certificate = <trustme.LeafCert object at 0x103c2f630>
tls_certificate_chain_pem_path = '/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/tmp78479eyf.pem'
tls_certificate_private_key_pem_path = '/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/tmpwdexlrl7.pem'
tls_ca_certificate_pem_path = '/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/tmpf3obdeu1.pem', is_trusted_cert = False, tls_client_identity = 'localhost'
tls_verify_mode = <VerifyMode.CERT_REQUIRED: 2>

    @pytest.mark.parametrize(
        'adapter_type',
        (
            'builtin',
            'pyopenssl',
        ),
    )
    @pytest.mark.parametrize(
        'is_trusted_cert,tls_client_identity',
        (
            (True, 'localhost'), (True, '127.0.0.1'),
            (True, '*.localhost'), (True, 'not_localhost'),
            (False, 'localhost'),
        ),
    )
    @pytest.mark.parametrize(
        'tls_verify_mode',
        (
            ssl.CERT_NONE,  # server shouldn't validate client cert
            ssl.CERT_OPTIONAL,  # same as CERT_REQUIRED in client mode, don't use
            ssl.CERT_REQUIRED,  # server should validate if client cert CA is OK
        ),
    )
    def test_tls_client_auth(
        # FIXME: remove twisted logic, separate tests
        mocker,
        tls_http_server, adapter_type,
        ca,
        tls_certificate,
        tls_certificate_chain_pem_path,
        tls_certificate_private_key_pem_path,
        tls_ca_certificate_pem_path,
        is_trusted_cert, tls_client_identity,
        tls_verify_mode,
    ):
        """Verify that client TLS certificate auth works correctly."""
        test_cert_rejection = (
            tls_verify_mode != ssl.CERT_NONE
            and not is_trusted_cert
        )
        interface, _host, port = _get_conn_data(ANY_INTERFACE_IPV4)
    
        client_cert_root_ca = ca if is_trusted_cert else trustme.CA()
        with mocker.mock_module.patch(
            'idna.core.ulabel',
            return_value=ntob(tls_client_identity),
        ):
            client_cert = client_cert_root_ca.issue_server_cert(
                # FIXME: change to issue_cert once new trustme is out
                ntou(tls_client_identity),
            )
            del client_cert_root_ca
    
        with client_cert.private_key_and_cert_chain_pem.tempfile() as cl_pem:
            tls_adapter_cls = get_ssl_adapter_class(name=adapter_type)
            tls_adapter = tls_adapter_cls(
                tls_certificate_chain_pem_path,
                tls_certificate_private_key_pem_path,
            )
            if adapter_type == 'pyopenssl':
                tls_adapter.context = tls_adapter.get_context()
                tls_adapter.context.set_verify(
                    _stdlib_to_openssl_verify[tls_verify_mode],
                    lambda conn, cert, errno, depth, preverify_ok: preverify_ok,
                )
            else:
                tls_adapter.context.verify_mode = tls_verify_mode
    
            ca.configure_trust(tls_adapter.context)
            tls_certificate.configure_cert(tls_adapter.context)
    
            tlshttpserver = tls_http_server.send(
                (
                    (interface, port),
                    tls_adapter,
                ),
            )
    
            interface, _host, port = _get_conn_data(tlshttpserver.bind_addr)
    
            make_https_request = functools.partial(
                requests.get,
                'https://' + interface + ':' + str(port) + '/',
    
                # Server TLS certificate verification:
                verify=tls_ca_certificate_pem_path,
    
                # Client TLS certificate verification:
                cert=cl_pem,
            )
    
            if not test_cert_rejection:
                resp = make_https_request()
                assert resp.status_code == 200
                assert resp.text == 'Hello world!'
                return
    
            with pytest.raises(requests.exceptions.SSLError) as ssl_err:
>               make_https_request()

cheroot/test/test_ssl.py:290: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.tox/python/lib/python3.7/site-packages/requests/api.py:75: in get
    return request('get', url, params=params, **kwargs)
.tox/python/lib/python3.7/site-packages/requests/api.py:60: in request
    return session.request(method=method, url=url, **kwargs)
.tox/python/lib/python3.7/site-packages/requests/sessions.py:533: in request
    resp = self.send(prep, **send_kwargs)
.tox/python/lib/python3.7/site-packages/requests/sessions.py:646: in send
    r = adapter.send(request, **kwargs)
.tox/python/lib/python3.7/site-packages/requests/adapters.py:449: in send
    timeout=timeout
.tox/python/lib/python3.7/site-packages/urllib3/connectionpool.py:600: in urlopen
    chunked=chunked)
.tox/python/lib/python3.7/site-packages/urllib3/connectionpool.py:384: in _make_request
    six.raise_from(e, None)
<string>:2: in raise_from
    ???
.tox/python/lib/python3.7/site-packages/urllib3/connectionpool.py:380: in _make_request
    httplib_response = conn.getresponse()
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py:1321: in getresponse
    response.begin()
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py:296: in begin
    version, status, reason = self._read_status()
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py:257: in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/socket.py:589: in readinto
    return self._sock.recv_into(b)
.tox/python/lib/python3.7/site-packages/urllib3/contrib/pyopenssl.py:294: in recv_into
    return self.connection.recv_into(*args, **kwargs)
.tox/python/lib/python3.7/site-packages/OpenSSL/SSL.py:1822: in recv_into
    self._raise_ssl_error(self._ssl, result)
.tox/python/lib/python3.7/site-packages/OpenSSL/SSL.py:1647: in _raise_ssl_error
    _raise_current_error()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

exception_type = <class 'OpenSSL.SSL.Error'>

    def exception_from_error_queue(exception_type):
        """
        Convert an OpenSSL library failure into a Python exception.
    
        When a call to the native OpenSSL library fails, this is usually signalled
        by the return value, and an error code is stored in an error queue
        associated with the current thread. The err library provides functions to
        obtain these error codes and textual error messages.
        """
        errors = []
    
        while True:
            error = lib.ERR_get_error()
            if error == 0:
                break
            errors.append((
                text(lib.ERR_lib_error_string(error)),
                text(lib.ERR_func_error_string(error)),
                text(lib.ERR_reason_error_string(error))))
    
>       raise exception_type(errors)
E       OpenSSL.SSL.Error: [('SSL routines', 'ssl3_read_bytes', 'tlsv1 alert unknown ca')]

.tox/python/lib/python3.7/site-packages/OpenSSL/_util.py:54: Error

📋 Environment

Cheroot version: master (e616c25)
CherryPy version: n/a
Python version: 3.7.2
OS: macOS 10.14.3
Browser: n/a

📋 Additional context

The text was updated successfully, but these errors were encountered:

jaraco · 2019-02-04T20:56:19Z

It appears that those tests are attempting to catch the SSL Exception, but requests.exceptions.SSLError doesn't include OpenSSL error. Maybe requests switched to OpenSSL... or will rely on it in certain environments?

jaraco · 2019-02-04T20:57:23Z

I see linux is also affected.

jaraco · 2019-02-04T20:59:03Z

Looks like tests passed 17 days ago. At the time, these were the deps installed:

apipkg==1.5
argh==0.26.2
asn1crypto==0.24.0
atomicwrites==1.2.1
attrs==18.2.0
backports.functools-lru-cache==1.5
certifi==2018.11.29
cffi==1.11.5
chardet==3.0.4
-e git+https://github.com/cherrypy/cheroot.git@d179b71c9a8c43d2595b6a892f41f44ccfd72bb4#egg=cheroot
codecov==2.0.15
colorama==0.4.1
coverage==4.5.2
cryptography==2.4.2
ddt==1.2.0
docopt==0.6.2
execnet==1.5.0
idna==2.8
more-itertools==5.0.0
packaging==18.0
pathtools==0.1.2
pluggy==0.8.1
py==1.7.0
pycparser==2.19
pyOpenSSL==18.0.0
pyparsing==2.3.1
pytest==4.1.1
pytest-cov==2.6.1
pytest-forked==1.0.1
pytest-mock==1.10.0
pytest-sugar==0.9.2
pytest-testmon==0.9.14
pytest-watch==4.2.0
pytest-xdist==1.26.0
PyYAML==3.13
requests==2.21.0
requests-unixsocket==0.1.5
six==1.12.0
termcolor==1.1.0
trustme==0.4.0
urllib3==1.24.1
watchdog==0.9.0

webknjaz · 2019-02-04T21:00:15Z

Yeah... About that. It looks like different combos of python runtime (cpython/pypy) + underlying ssl lib (openssl/libressl) + it's version (0.9/1.0/1.1) produce weirdly different results.

I was trying to identify certain patterns but it's not usually obvious what's going on. Sometime test would behave differently under the seemingly similar env (like real macbook vs Travis CI vs Circle CI with the same version of macOS and openssl lib).

So currently I'm trying to figure this out time to time when possible and then give up periodically...

I also noticed that upgrade of pyOpenSSL affected the behavior.

jaraco · 2019-02-04T21:01:30Z

Here we have one of the early failures where here are the deps installed:

apipkg==1.5
argh==0.26.2
asn1crypto==0.24.0
atomicwrites==1.2.1
attrs==18.2.0
backports.functools-lru-cache==1.5
certifi==2018.11.29
cffi==1.11.5
chardet==3.0.4
-e git+https://github.com/cherrypy/cheroot.git@d179b71c9a8c43d2595b6a892f41f44ccfd72bb4#egg=cheroot
codecov==2.0.15
colorama==0.4.1
coverage==4.5.2
cryptography==2.5
ddt==1.2.0
docopt==0.6.2
execnet==1.5.0
idna==2.8
more-itertools==5.0.0
packaging==19.0
pathtools==0.1.2
pluggy==0.8.1
py==1.7.0
pycparser==2.19
pyOpenSSL==19.0.0
pyparsing==2.3.1
pytest==4.1.1
pytest-cov==2.6.1
pytest-forked==1.0.1
pytest-mock==1.10.0
pytest-sugar==0.9.2
pytest-testmon==0.9.14
pytest-watch==4.2.0
pytest-xdist==1.26.0
PyYAML==3.13
requests==2.21.0
requests-unixsocket==0.1.5
six==1.12.0
termcolor==1.1.0
trustme==0.5.0
urllib3==1.24.1
watchdog==0.9.0

webknjaz · 2019-02-04T21:01:51Z

One of the issues is that currently code catches and ignores "safe" exceptions during the initial TLS handshake (i.e. on connect) but I saw those happenning on the later stage (basically when we're trying to read from the network stream) as well in some cases.

jaraco · 2019-02-04T21:03:03Z

I tried downgrading to pyopenssl 18, but that didn't help.

webknjaz · 2019-02-04T21:04:04Z

maybe some transitive dependency upgrade + it also depends on the actual openssl backend

webknjaz · 2019-02-04T21:04:50Z

Oh and this "combo" issue thing is kinda complicated: the stdlib ssl module can be linked against different version of openssl, also pyopenssl (cryptography really) can be linked against something different

jaraco · 2019-02-04T21:05:26Z

Installing the full set of deps from that successful build does work around the issue:

Installing collected packages: atomicwrites, cryptography, packaging, pytest, pytest-mock, pytest-xdist, trustme
  Found existing installation: atomicwrites 1.3.0
    Uninstalling atomicwrites-1.3.0:
      Successfully uninstalled atomicwrites-1.3.0
  Found existing installation: cryptography 2.5
    Uninstalling cryptography-2.5:
      Successfully uninstalled cryptography-2.5
  Found existing installation: packaging 19.0
    Uninstalling packaging-19.0:
      Successfully uninstalled packaging-19.0
  Found existing installation: pytest 4.2.0
    Uninstalling pytest-4.2.0:
      Successfully uninstalled pytest-4.2.0
  Found existing installation: pytest-mock 1.10.1
    Uninstalling pytest-mock-1.10.1:
      Successfully uninstalled pytest-mock-1.10.1
  Found existing installation: pytest-xdist 1.26.1
    Uninstalling pytest-xdist-1.26.1:
      Successfully uninstalled pytest-xdist-1.26.1
  Found existing installation: trustme 0.5.0
    Uninstalling trustme-0.5.0:
      Successfully uninstalled trustme-0.5.0
Successfully installed atomicwrites-1.2.1 cryptography-2.4.2 packaging-18.0 pytest-4.1.1 pytest-mock-1.10.0 pytest-xdist-1.26.0 trustme-0.4.0

webknjaz · 2019-02-04T21:06:00Z

And PyPy has its own CFFI based wrapper for OpenSSL while CPython has that in pure C and cryptography also does pure C thing I think.

webknjaz · 2019-02-04T21:08:44Z

Okay, from diff it seems that it's cryptography's fault... But I'm curious what it's build against now and whether it matches a shared lib actually present on the machine.

jaraco · 2019-02-04T21:12:55Z

According to the changelog, cryptography 2.5 introduced this change:

Updated Windows, macOS, and manylinux1 wheels to be compiled with OpenSSL 1.1.1a.

Based on your other assertions above, this sounds like a likely proximate cause.

webknjaz · 2019-02-04T21:14:57Z

@jaraco you can try this to check actual versions of stuff:

python -m OpenSSL.debug

webknjaz · 2019-02-04T21:16:43Z

I've actually made CI print it a while back:

jaraco · 2019-02-04T21:20:42Z

Really annoying that requests would raise different exceptions depending on the openssl version present in cryptography.

jaraco · 2019-02-04T21:22:27Z

I wonder if there's a way to test this behavior outside of cheroot - using just requests and cryptography versions against some known SSL endpoint with an unknown CA.

jaraco · 2019-02-04T21:25:51Z

Running with simple requests and cryptography 2.5, I see that 'requests' raises the expected requests.exceptions.SSLError:

cheroot master $ pip-run requests 'cryptography>=2.5' -- -c "import requests; requests.get('https://untrusted-root.badssl.com/')"                                                        
Collecting cryptography>=2.5
  Using cached https://files.pythonhosted.org/packages/d7/9e/12bb10fd009b0146935c169cc0e1e86221eacf8dc207990d54b783c47a7d/cryptography-2.5-cp34-abi3-macosx_10_6_intel.whl
Collecting asn1crypto>=0.21.0 (from cryptography>=2.5)
  Using cached https://files.pythonhosted.org/packages/ea/cd/35485615f45f30a510576f1a56d1e0a7ad7bd8ab5ed7cdc600ef7cd06222/asn1crypto-0.24.0-py2.py3-none-any.whl
Collecting cffi!=1.11.3,>=1.8 (from cryptography>=2.5)
  Using cached https://files.pythonhosted.org/packages/0b/ba/32835c9965d8a0090723e1d0b47373365525c4bd08c807b5efdc9fecbc99/cffi-1.11.5-cp37-cp37m-macosx_10_9_x86_64.whl
Collecting six>=1.4.1 (from cryptography>=2.5)
  Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Collecting pycparser (from cffi!=1.11.3,>=1.8->cryptography>=2.5)
Installing collected packages: asn1crypto, pycparser, cffi, six, cryptography
cheroot 6.5.5.dev7+ge616c254.d20190204 requires backports.functools_lru_cache, which is not installed.
Successfully installed asn1crypto-0.24.0 cffi-1.11.5 cryptography-2.5 pycparser-2.19 six-1.12.0
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/connectionpool.py", line 343, in _make_request
    self._validate_conn(conn)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/connectionpool.py", line 849, in _validate_conn
    conn.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/connection.py", line 356, in connect
    ssl_context=context)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/util/ssl_.py", line 359, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line 412, in wrap_socket
    session=session
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line 853, in _create
    self.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line 1117, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/adapters.py", line 445, in send
    timeout=timeout
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/util/retry.py", line 398, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='untrusted-root.badssl.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/sessions.py", line 512, in request
    resp = self.send(prep, **send_kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/sessions.py", line 622, in send
    r = adapter.send(request, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/adapters.py", line 511, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='untrusted-root.badssl.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)')))

So the problem seems to have something to do with cheroot.

webknjaz · 2019-02-04T21:26:29Z

Well, it's actually not requests itself, but underlying libraries behaving like this. I've integrated trustme to have our own fake CA and it's very easy to use + I'm pretty sure it works correctly.

OTOH because implementation differences and a lot of env factors (like I mentioned above) it's kinda hard to cover all cases. In our tests I saw for one "logical" case of wrong setup the underlying library would sometimes return an alert from TLS or SSL, for example, which have different codes because they are from different namespaces.

jaraco · 2019-02-04T21:29:11Z

Yeah, I'm pretty sure my test above isn't accurately exercising the behavior. I'm not passing verify or cert to requests.

jaraco · 2019-02-04T21:48:32Z

Well, I'm stumped. I can't tell what factors are leading to requests not wrapping the underlying error... to implicate requests, or what factors cheroot could possibly do to mitigate the error.

I did try trapping both requests.exceptions.SSLError and OpenSSL.SSL.Error thus:

diff --git a/cheroot/test/test_ssl.py b/cheroot/test/test_ssl.py
index 4708e23f..b628c36c 100644
--- a/cheroot/test/test_ssl.py
+++ b/cheroot/test/test_ssl.py
@@ -286,7 +286,11 @@ def test_tls_client_auth(
             assert resp.text == 'Hello world!'
             return
 
-        with pytest.raises(requests.exceptions.SSLError) as ssl_err:
+        expected = (
+            requests.exceptions.SSLError,
+            OpenSSL.SSL.Error,
+        )
+        with pytest.raises(expected) as ssl_err:
             make_https_request()
 
         err_text = ssl_err.value.args[0].reason.args[0].args[0]

But that didn't work as the follow-up assertions on err_text fail.

I need to punt on this error as I'm trying to investigate other issues. I suggest we either mark these tests as xfail or proceed with #174 (pin cryptography) until someone has time to investigate further (or the problem is discovered and fixed elsewhere).

@webknjaz Any preference on those two options ^?

webknjaz · 2019-02-04T21:52:43Z

I'd go for xfail. Regarding requests wrapping an error, it actually puts the underlying error inside of its own exception and err_text = ssl_err.value.args[0].reason.args[0].args[0] thing tries to extract it back for comparison.

xfail when encountering #173

stale · 2019-04-05T22:37:46Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

webknjaz · 2019-04-08T10:12:17Z

It looks like urllib3 > 1.24.1 should fix this via urllib3/urllib3#1496. So we have to either wait or point to our test env to their git repo directly...

webknjaz · 2019-04-21T23:00:50Z

urllib==1.24.2 has been released 4 days ago.

jaraco changed the title ~~SSL tests fail with uncaught SSL validation on macOS~~ SSL tests fail with uncaught SSL validation Feb 4, 2019

jaraco added a commit that referenced this issue Feb 4, 2019

Pin cryptography to <2.5. Temporary workaround for #173.

93ffdea

jaraco mentioned this issue Feb 4, 2019

Pin cryptography to <2.5. Temporary workaround for #173. #174

Closed

15 tasks

jaraco mentioned this issue Feb 4, 2019

OpenSSL.SSL.Error raised with invalid client cert and openssl 1.1.1. psf/requests#4961

Closed

jaraco added a commit that referenced this issue Feb 4, 2019

xfail when encountering #173

57d70d4

jaraco mentioned this issue Feb 4, 2019

xfail when encountering #173 #175

Merged

15 tasks

jaraco added a commit that referenced this issue Feb 4, 2019

Merge pull request #175 from cherrypy/bugfix/173-xfail-client-ssl-fails

6d32c40

xfail when encountering #173

jaraco mentioned this issue Feb 5, 2019

OpenSSL.SSL.Error raised with invalid client cert and cryptography 2.5 urllib3/urllib3#1537

Closed

stale bot added the stale This thing has been ignored for too long label Apr 5, 2019

webknjaz added the bug Something is broken label Apr 6, 2019

stale bot removed the stale This thing has been ignored for too long label Apr 6, 2019

webknjaz mentioned this issue Apr 22, 2019

Fix TLS 1.3 client tests with requests #192

Merged

15 tasks

webknjaz closed this as completed in #192 Apr 22, 2019

webknjaz mentioned this issue May 16, 2019

test_tls_client_auth failing (1-False-localhost-builtin, 1-False-localhost-pyopenssl, 2-False-localhost-pyopenssl and 2-False-localhost-builtin) with OpenSSL 1.1.1 #200

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SSL tests fail with uncaught SSL validation #173

SSL tests fail with uncaught SSL validation #173

jaraco commented Feb 4, 2019

jaraco commented Feb 4, 2019

jaraco commented Feb 4, 2019

jaraco commented Feb 4, 2019 •

edited

webknjaz commented Feb 4, 2019

jaraco commented Feb 4, 2019

webknjaz commented Feb 4, 2019

jaraco commented Feb 4, 2019

webknjaz commented Feb 4, 2019

webknjaz commented Feb 4, 2019

jaraco commented Feb 4, 2019

webknjaz commented Feb 4, 2019

webknjaz commented Feb 4, 2019

jaraco commented Feb 4, 2019 •

edited

webknjaz commented Feb 4, 2019

webknjaz commented Feb 4, 2019

jaraco commented Feb 4, 2019

jaraco commented Feb 4, 2019

jaraco commented Feb 4, 2019

webknjaz commented Feb 4, 2019

jaraco commented Feb 4, 2019

jaraco commented Feb 4, 2019

webknjaz commented Feb 4, 2019

stale bot commented Apr 5, 2019

webknjaz commented Apr 8, 2019

webknjaz commented Apr 21, 2019

SSL tests fail with uncaught SSL validation #173

SSL tests fail with uncaught SSL validation #173

Comments

jaraco commented Feb 4, 2019

jaraco commented Feb 4, 2019

jaraco commented Feb 4, 2019

jaraco commented Feb 4, 2019 • edited

webknjaz commented Feb 4, 2019

jaraco commented Feb 4, 2019

webknjaz commented Feb 4, 2019

jaraco commented Feb 4, 2019

webknjaz commented Feb 4, 2019

webknjaz commented Feb 4, 2019

jaraco commented Feb 4, 2019

webknjaz commented Feb 4, 2019

webknjaz commented Feb 4, 2019

jaraco commented Feb 4, 2019 • edited

webknjaz commented Feb 4, 2019

webknjaz commented Feb 4, 2019

jaraco commented Feb 4, 2019

jaraco commented Feb 4, 2019

jaraco commented Feb 4, 2019

webknjaz commented Feb 4, 2019

jaraco commented Feb 4, 2019

jaraco commented Feb 4, 2019

webknjaz commented Feb 4, 2019

stale bot commented Apr 5, 2019

webknjaz commented Apr 8, 2019

webknjaz commented Apr 21, 2019

jaraco commented Feb 4, 2019 •

edited

jaraco commented Feb 4, 2019 •

edited