Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test_tls_client_auth failing (1-False-localhost-builtin, 1-False-localhost-pyopenssl, 2-False-localhost-pyopenssl and 2-False-localhost-builtin) with OpenSSL 1.1.1 #200

Closed
1 of 3 tasks
mcepl opened this issue May 14, 2019 · 2 comments
Closed
1 of 3 tasks

test_tls_client_auth failing (1-False-localhost-builtin, 1-False-localhost-pyopenssl, 2-False-localhost-pyopenssl and 2-False-localhost-builtin) with OpenSSL 1.1.1 #200

mcepl opened this issue May 14, 2019 · 2 comments
Labels
invalid This is irrelevant

Comments

@mcepl
Copy link

mcepl commented May 14, 2019

❓ I'm submitting a ...

  • 🐞 bug report
  • 🐣 feature request
  • ❓ question about the decisions made in the repository

🐞 Describe the bug. What is the current behavior?
Multiple tests are failing on openSUSE/Tumbleweed with new OpenSSL 1.1.1:

[   87s] =================================== FAILURES ===================================
[   87s] _______________ test_tls_client_auth[1-False-localhost-builtin] ________________
[   87s] 
[   87s] mocker = <pytest_mock.MockFixture object at 0x7f8124c1fd50>
[   87s] tls_http_server = <generator object start_srv at 0x7f811f46b9b0>
[   87s] adapter_type = 'builtin', ca = <trustme.CA object at 0x7f8124c12d10>
[   87s] tls_certificate = <trustme.LeafCert object at 0x7f8124c09250>
[   87s] tls_certificate_chain_pem_path = '/tmp/tmpvO4LqR.pem'
[   87s] tls_certificate_private_key_pem_path = '/tmp/tmpV0oZYc.pem'
[   87s] tls_ca_certificate_pem_path = '/tmp/tmpl6z2dm.pem', is_trusted_cert = False
[   87s] tls_client_identity = 'localhost', tls_verify_mode = 1
[   87s] 
[   87s]     @pytest.mark.parametrize(
[   87s]         'adapter_type',
[   87s]         (
[   87s]             'builtin',
[   87s]             'pyopenssl',
[   87s]         ),
[   87s]     )
[   87s]     @pytest.mark.parametrize(
[   87s]         'is_trusted_cert,tls_client_identity',
[   87s]         (
[   87s]             (True, 'localhost'), (True, '127.0.0.1'),
[   87s]             (True, '*.localhost'), (True, 'not_localhost'),
[   87s]             (False, 'localhost'),
[   87s]         ),
[   87s]     )
[   87s]     @pytest.mark.parametrize(
[   87s]         'tls_verify_mode',
[   87s]         (
[   87s]             ssl.CERT_NONE,  # server shouldn't validate client cert
[   87s]             ssl.CERT_OPTIONAL,  # same as CERT_REQUIRED in client mode, don't use
[   87s]             ssl.CERT_REQUIRED,  # server should validate if client cert CA is OK
[   87s]         ),
[   87s]     )
[   87s]     def test_tls_client_auth(
[   87s]         # FIXME: remove twisted logic, separate tests
[   87s]         mocker,
[   87s]         tls_http_server, adapter_type,
[   87s]         ca,
[   87s]         tls_certificate,
[   87s]         tls_certificate_chain_pem_path,
[   87s]         tls_certificate_private_key_pem_path,
[   87s]         tls_ca_certificate_pem_path,
[   87s]         is_trusted_cert, tls_client_identity,
[   87s]         tls_verify_mode,
[   87s]     ):
[   87s]         """Verify that client TLS certificate auth works correctly."""
[   87s]         test_cert_rejection = (
[   87s]             tls_verify_mode != ssl.CERT_NONE
[   87s]             and not is_trusted_cert
[   87s]         )
[   87s]         interface, _host, port = _get_conn_data(ANY_INTERFACE_IPV4)
[   87s]     
[   87s]         client_cert_root_ca = ca if is_trusted_cert else trustme.CA()
[   87s]         with mocker.mock_module.patch(
[   87s]             'idna.core.ulabel',
[   87s]             return_value=ntob(tls_client_identity),
[   87s]         ):
[   87s]             client_cert = client_cert_root_ca.issue_server_cert(
[   87s]                 # FIXME: change to issue_cert once new trustme is out
[   87s]                 ntou(tls_client_identity),
[   87s]             )
[   87s]             del client_cert_root_ca
[   87s]     
[   87s]         with client_cert.private_key_and_cert_chain_pem.tempfile() as cl_pem:
[   87s]             tls_adapter_cls = get_ssl_adapter_class(name=adapter_type)
[   87s]             tls_adapter = tls_adapter_cls(
[   87s]                 tls_certificate_chain_pem_path,
[   87s]                 tls_certificate_private_key_pem_path,
[   87s]             )
[   87s]             if adapter_type == 'pyopenssl':
[   87s]                 tls_adapter.context = tls_adapter.get_context()
[   87s]                 tls_adapter.context.set_verify(
[   87s]                     _stdlib_to_openssl_verify[tls_verify_mode],
[   87s]                     lambda conn, cert, errno, depth, preverify_ok: preverify_ok,
[   87s]                 )
[   87s]             else:
[   87s]                 tls_adapter.context.verify_mode = tls_verify_mode
[   87s]     
[   87s]             ca.configure_trust(tls_adapter.context)
[   87s]             tls_certificate.configure_cert(tls_adapter.context)
[   87s]     
[   87s]             tlshttpserver = tls_http_server.send(
[   87s]                 (
[   87s]                     (interface, port),
[   87s]                     tls_adapter,
[   87s]                 ),
[   87s]             )
[   87s]     
[   87s]             interface, _host, port = _get_conn_data(tlshttpserver.bind_addr)
[   87s]     
[   87s]             make_https_request = functools.partial(
[   87s]                 requests.get,
[   87s]                 'https://' + interface + ':' + str(port) + '/',
[   87s]     
[   87s]                 # Server TLS certificate verification:
[   87s]                 verify=tls_ca_certificate_pem_path,
[   87s]     
[   87s]                 # Client TLS certificate verification:
[   87s]                 cert=cl_pem,
[   87s]             )
[   87s]     
[   87s]             if not test_cert_rejection:
[   87s]                 resp = make_https_request()
[   87s]                 is_req_successful = resp.status_code == 200
[   87s]                 if (
[   87s]                         not is_req_successful
[   87s]                         and IS_PYOPENSSL_SSL_VERSION_1_0
[   87s]                         and adapter_type == 'builtin'
[   87s]                         and tls_verify_mode == ssl.CERT_REQUIRED
[   87s]                         and tls_client_identity == 'localhost'
[   87s]                         and is_trusted_cert
[   87s]                 ):
[   87s]                     pytest.xfail(
[   87s]                         'OpenSSL 1.0 has problems with verifying client certs',
[   87s]                     )
[   87s]                 assert is_req_successful
[   87s]                 assert resp.text == 'Hello world!'
[   87s]                 return
[   87s]     
[   87s]             with pytest.raises(requests.exceptions.SSLError) as ssl_err:
[   87s] >               make_https_request()
[   87s] 
[   87s] cheroot/test/test_ssl.py:316: 
[   87s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   87s] /usr/lib/python2.7/site-packages/requests/api.py:75: in get
[   87s]     return request('get', url, params=params, **kwargs)
[   87s] /usr/lib/python2.7/site-packages/requests/api.py:60: in request
[   87s]     return session.request(method=method, url=url, **kwargs)
[   87s] /usr/lib/python2.7/site-packages/requests/sessions.py:533: in request
[   87s]     resp = self.send(prep, **send_kwargs)
[   87s] /usr/lib/python2.7/site-packages/requests/sessions.py:646: in send
[   87s]     r = adapter.send(request, **kwargs)
[   87s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   87s] 
[   87s] self = <requests.adapters.HTTPAdapter object at 0x7f8124b90a90>
[   87s] request = <PreparedRequest [GET]>, stream = False
[   87s] timeout = <urllib3.util.timeout.Timeout object at 0x7f8124bbfd10>
[   87s] verify = '/tmp/tmpl6z2dm.pem', cert = '/tmp/tmpF4asyA.pem'
[   87s] proxies = OrderedDict()
[   87s] 
[   87s]     def send(self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None):
[   87s]         """Sends PreparedRequest object. Returns Response object.
[   87s]     
[   87s]         :param request: The :class:`PreparedRequest <PreparedRequest>` being sent.
[   87s]         :param stream: (optional) Whether to stream the request content.
[   87s]         :param timeout: (optional) How long to wait for the server to send
[   87s]             data before giving up, as a float, or a :ref:`(connect timeout,
[   87s]             read timeout) <timeouts>` tuple.
[   87s]         :type timeout: float or tuple or urllib3 Timeout object
[   87s]         :param verify: (optional) Either a boolean, in which case it controls whether
[   87s]             we verify the server's TLS certificate, or a string, in which case it
[   87s]             must be a path to a CA bundle to use
[   87s]         :param cert: (optional) Any user-provided SSL certificate to be trusted.
[   87s]         :param proxies: (optional) The proxies dictionary to apply to the request.
[   87s]         :rtype: requests.Response
[   87s]         """
[   87s]     
[   87s]         try:
[   87s]             conn = self.get_connection(request.url, proxies)
[   87s]         except LocationValueError as e:
[   87s]             raise InvalidURL(e, request=request)
[   87s]     
[   87s]         self.cert_verify(conn, request.url, verify, cert)
[   87s]         url = self.request_url(request, proxies)
[   87s]         self.add_headers(request, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies)
[   87s]     
[   87s]         chunked = not (request.body is None or 'Content-Length' in request.headers)
[   87s]     
[   87s]         if isinstance(timeout, tuple):
[   87s]             try:
[   87s]                 connect, read = timeout
[   87s]                 timeout = TimeoutSauce(connect=connect, read=read)
[   87s]             except ValueError as e:
[   87s]                 # this may raise a string formatting error.
[   87s]                 err = ("Invalid timeout {}. Pass a (connect, read) "
[   87s]                        "timeout tuple, or a single float to set "
[   87s]                        "both timeouts to the same value".format(timeout))
[   87s]                 raise ValueError(err)
[   87s]         elif isinstance(timeout, TimeoutSauce):
[   87s]             pass
[   87s]         else:
[   87s]             timeout = TimeoutSauce(connect=timeout, read=timeout)
[   87s]     
[   87s]         try:
[   87s]             if not chunked:
[   87s]                 resp = conn.urlopen(
[   87s]                     method=request.method,
[   87s]                     url=url,
[   87s]                     body=request.body,
[   87s]                     headers=request.headers,
[   87s]                     redirect=False,
[   87s]                     assert_same_host=False,
[   87s]                     preload_content=False,
[   87s]                     decode_content=False,
[   87s]                     retries=self.max_retries,
[   87s]                     timeout=timeout
[   87s]                 )
[   87s]     
[   87s]             # Send the request.
[   87s]             else:
[   87s]                 if hasattr(conn, 'proxy_pool'):
[   87s]                     conn = conn.proxy_pool
[   87s]     
[   87s]                 low_conn = conn._get_conn(timeout=DEFAULT_POOL_TIMEOUT)
[   87s]     
[   87s]                 try:
[   87s]                     low_conn.putrequest(request.method,
[   87s]                                         url,
[   87s]                                         skip_accept_encoding=True)
[   87s]     
[   87s]                     for header, value in request.headers.items():
[   87s]                         low_conn.putheader(header, value)
[   87s]     
[   87s]                     low_conn.endheaders()
[   87s]     
[   87s]                     for i in request.body:
[   87s]                         low_conn.send(hex(len(i))[2:].encode('utf-8'))
[   87s]                         low_conn.send(b'\r\n')
[   87s]                         low_conn.send(i)
[   87s]                         low_conn.send(b'\r\n')
[   87s]                     low_conn.send(b'0\r\n\r\n')
[   87s]     
[   87s]                     # Receive the response from the server
[   87s]                     try:
[   87s]                         # For Python 2.7, use buffering of HTTP responses
[   87s]                         r = low_conn.getresponse(buffering=True)
[   87s]                     except TypeError:
[   87s]                         # For compatibility with Python 3.3+
[   87s]                         r = low_conn.getresponse()
[   87s]     
[   87s]                     resp = HTTPResponse.from_httplib(
[   87s]                         r,
[   87s]                         pool=conn,
[   87s]                         connection=low_conn,
[   87s]                         preload_content=False,
[   87s]                         decode_content=False
[   87s]                     )
[   87s]                 except:
[   87s]                     # If we hit any problems here, clean up the connection.
[   87s]                     # Then, reraise so that we can handle the actual exception.
[   87s]                     low_conn.close()
[   87s]                     raise
[   87s]     
[   87s]         except (ProtocolError, socket.error) as err:
[   87s] >           raise ConnectionError(err, request=request)
[   87s] E           ConnectionError: ('Connection aborted.', error("(104, 'ECONNRESET')",))
[   87s] 
[   87s] /usr/lib/python2.7/site-packages/requests/adapters.py:498: ConnectionError
[   87s] ______________ test_tls_client_auth[1-False-localhost-pyopenssl] _______________
[   87s] 
[   87s] mocker = <pytest_mock.MockFixture object at 0x7f8124cf6790>
[   87s] tls_http_server = <generator object start_srv at 0x7f811f46b230>
[   87s] adapter_type = 'pyopenssl', ca = <trustme.CA object at 0x7f811f34b790>
[   87s] tls_certificate = <trustme.LeafCert object at 0x7f8124b70a10>
[   87s] tls_certificate_chain_pem_path = '/tmp/tmpzlMkUm.pem'
[   87s] tls_certificate_private_key_pem_path = '/tmp/tmpHIQaCY.pem'
[   87s] tls_ca_certificate_pem_path = '/tmp/tmpt3y3JK.pem', is_trusted_cert = False
[   87s] tls_client_identity = 'localhost', tls_verify_mode = 1
[   87s] 
[   87s]     @pytest.mark.parametrize(
[   87s]         'adapter_type',
[   87s]         (
[   87s]             'builtin',
[   87s]             'pyopenssl',
[   87s]         ),
[   87s]     )
[   87s]     @pytest.mark.parametrize(
[   87s]         'is_trusted_cert,tls_client_identity',
[   87s]         (
[   87s]             (True, 'localhost'), (True, '127.0.0.1'),
[   87s]             (True, '*.localhost'), (True, 'not_localhost'),
[   87s]             (False, 'localhost'),
[   87s]         ),
[   87s]     )
[   87s]     @pytest.mark.parametrize(
[   87s]         'tls_verify_mode',
[   87s]         (
[   87s]             ssl.CERT_NONE,  # server shouldn't validate client cert
[   87s]             ssl.CERT_OPTIONAL,  # same as CERT_REQUIRED in client mode, don't use
[   87s]             ssl.CERT_REQUIRED,  # server should validate if client cert CA is OK
[   87s]         ),
[   87s]     )
[   87s]     def test_tls_client_auth(
[   87s]         # FIXME: remove twisted logic, separate tests
[   87s]         mocker,
[   87s]         tls_http_server, adapter_type,
[   87s]         ca,
[   87s]         tls_certificate,
[   87s]         tls_certificate_chain_pem_path,
[   87s]         tls_certificate_private_key_pem_path,
[   87s]         tls_ca_certificate_pem_path,
[   87s]         is_trusted_cert, tls_client_identity,
[   87s]         tls_verify_mode,
[   87s]     ):
[   87s]         """Verify that client TLS certificate auth works correctly."""
[   87s]         test_cert_rejection = (
[   87s]             tls_verify_mode != ssl.CERT_NONE
[   87s]             and not is_trusted_cert
[   87s]         )
[   87s]         interface, _host, port = _get_conn_data(ANY_INTERFACE_IPV4)
[   87s]     
[   87s]         client_cert_root_ca = ca if is_trusted_cert else trustme.CA()
[   87s]         with mocker.mock_module.patch(
[   87s]             'idna.core.ulabel',
[   87s]             return_value=ntob(tls_client_identity),
[   87s]         ):
[   87s]             client_cert = client_cert_root_ca.issue_server_cert(
[   87s]                 # FIXME: change to issue_cert once new trustme is out
[   87s]                 ntou(tls_client_identity),
[   87s]             )
[   87s]             del client_cert_root_ca
[   87s]     
[   87s]         with client_cert.private_key_and_cert_chain_pem.tempfile() as cl_pem:
[   87s]             tls_adapter_cls = get_ssl_adapter_class(name=adapter_type)
[   87s]             tls_adapter = tls_adapter_cls(
[   87s]                 tls_certificate_chain_pem_path,
[   87s]                 tls_certificate_private_key_pem_path,
[   87s]             )
[   87s]             if adapter_type == 'pyopenssl':
[   87s]                 tls_adapter.context = tls_adapter.get_context()
[   87s]                 tls_adapter.context.set_verify(
[   87s]                     _stdlib_to_openssl_verify[tls_verify_mode],
[   87s]                     lambda conn, cert, errno, depth, preverify_ok: preverify_ok,
[   87s]                 )
[   87s]             else:
[   87s]                 tls_adapter.context.verify_mode = tls_verify_mode
[   87s]     
[   87s]             ca.configure_trust(tls_adapter.context)
[   87s]             tls_certificate.configure_cert(tls_adapter.context)
[   87s]     
[   87s]             tlshttpserver = tls_http_server.send(
[   87s]                 (
[   87s]                     (interface, port),
[   87s]                     tls_adapter,
[   87s]                 ),
[   87s]             )
[   87s]     
[   87s]             interface, _host, port = _get_conn_data(tlshttpserver.bind_addr)
[   87s]     
[   87s]             make_https_request = functools.partial(
[   87s]                 requests.get,
[   87s]                 'https://' + interface + ':' + str(port) + '/',
[   87s]     
[   87s]                 # Server TLS certificate verification:
[   87s]                 verify=tls_ca_certificate_pem_path,
[   87s]     
[   87s]                 # Client TLS certificate verification:
[   87s]                 cert=cl_pem,
[   87s]             )
[   87s]     
[   87s]             if not test_cert_rejection:
[   87s]                 resp = make_https_request()
[   87s]                 is_req_successful = resp.status_code == 200
[   87s]                 if (
[   87s]                         not is_req_successful
[   87s]                         and IS_PYOPENSSL_SSL_VERSION_1_0
[   87s]                         and adapter_type == 'builtin'
[   87s]                         and tls_verify_mode == ssl.CERT_REQUIRED
[   87s]                         and tls_client_identity == 'localhost'
[   87s]                         and is_trusted_cert
[   87s]                 ):
[   87s]                     pytest.xfail(
[   87s]                         'OpenSSL 1.0 has problems with verifying client certs',
[   87s]                     )
[   87s]                 assert is_req_successful
[   87s]                 assert resp.text == 'Hello world!'
[   87s]                 return
[   87s]     
[   87s]             with pytest.raises(requests.exceptions.SSLError) as ssl_err:
[   87s] >               make_https_request()
[   87s] 
[   87s] cheroot/test/test_ssl.py:316: 
[   87s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   87s] /usr/lib/python2.7/site-packages/requests/api.py:75: in get
[   87s]     return request('get', url, params=params, **kwargs)
[   87s] /usr/lib/python2.7/site-packages/requests/api.py:60: in request
[   87s]     return session.request(method=method, url=url, **kwargs)
[   87s] /usr/lib/python2.7/site-packages/requests/sessions.py:533: in request
[   87s]     resp = self.send(prep, **send_kwargs)
[   87s] /usr/lib/python2.7/site-packages/requests/sessions.py:646: in send
[   87s]     r = adapter.send(request, **kwargs)
[   87s] /usr/lib/python2.7/site-packages/requests/adapters.py:449: in send
[   87s]     timeout=timeout
[   87s] /usr/lib/python2.7/site-packages/urllib3/connectionpool.py:600: in urlopen
[   87s]     chunked=chunked)
[   87s] /usr/lib/python2.7/site-packages/urllib3/connectionpool.py:377: in _make_request
[   87s]     httplib_response = conn.getresponse(buffering=True)
[   87s] /usr/lib64/python2.7/httplib.py:1121: in getresponse
[   87s]     response.begin()
[   87s] /usr/lib64/python2.7/httplib.py:438: in begin
[   87s]     version, status, reason = self._read_status()
[   87s] /usr/lib64/python2.7/httplib.py:394: in _read_status
[   87s]     line = self.fp.readline(_MAXLINE + 1)
[   87s] /usr/lib64/python2.7/socket.py:480: in readline
[   87s]     data = self._sock.recv(self._rbufsize)
[   87s] /usr/lib/python2.7/site-packages/urllib3/contrib/pyopenssl.py:276: in recv
[   87s]     data = self.connection.recv(*args, **kwargs)
[   87s] /usr/lib/python2.7/site-packages/OpenSSL/SSL.py:1791: in recv
[   87s]     self._raise_ssl_error(self._ssl, result)
[   87s] /usr/lib/python2.7/site-packages/OpenSSL/SSL.py:1647: in _raise_ssl_error
[   87s]     _raise_current_error()
[   87s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   87s] 
[   87s] exception_type = <class 'OpenSSL.SSL.Error'>
[   87s] 
[   87s]     def exception_from_error_queue(exception_type):
[   87s]         """
[   87s]         Convert an OpenSSL library failure into a Python exception.
[   87s]     
[   87s]         When a call to the native OpenSSL library fails, this is usually signalled
[   87s]         by the return value, and an error code is stored in an error queue
[   87s]         associated with the current thread. The err library provides functions to
[   87s]         obtain these error codes and textual error messages.
[   87s]         """
[   87s]         errors = []
[   87s]     
[   87s]         while True:
[   87s]             error = lib.ERR_get_error()
[   87s]             if error == 0:
[   87s]                 break
[   87s]             errors.append((
[   87s]                 text(lib.ERR_lib_error_string(error)),
[   87s]                 text(lib.ERR_func_error_string(error)),
[   87s]                 text(lib.ERR_reason_error_string(error))))
[   87s]     
[   87s] >       raise exception_type(errors)
[   87s] E       Error: [('SSL routines', 'ssl3_read_bytes', 'tlsv1 alert unknown ca')]
[   87s] 
[   87s] /usr/lib/python2.7/site-packages/OpenSSL/_util.py:54: Error
[   87s] _______________ test_tls_client_auth[2-False-localhost-builtin] ________________
[   87s] 
[   87s] mocker = <pytest_mock.MockFixture object at 0x7f8124c12a50>
[   87s] tls_http_server = <generator object start_srv at 0x7f8120b5c230>
[   87s] adapter_type = 'builtin', ca = <trustme.CA object at 0x7f8120768650>
[   87s] tls_certificate = <trustme.LeafCert object at 0x7f8124baa6d0>
[   87s] tls_certificate_chain_pem_path = '/tmp/tmppvmQk5.pem'
[   87s] tls_certificate_private_key_pem_path = '/tmp/tmp0C1rNu.pem'
[   87s] tls_ca_certificate_pem_path = '/tmp/tmphCe4LF.pem', is_trusted_cert = False
[   87s] tls_client_identity = 'localhost', tls_verify_mode = 2
[   87s] 
[   87s]     @pytest.mark.parametrize(
[   87s]         'adapter_type',
[   87s]         (
[   87s]             'builtin',
[   87s]             'pyopenssl',
[   87s]         ),
[   87s]     )
[   87s]     @pytest.mark.parametrize(
[   87s]         'is_trusted_cert,tls_client_identity',
[   87s]         (
[   87s]             (True, 'localhost'), (True, '127.0.0.1'),
[   87s]             (True, '*.localhost'), (True, 'not_localhost'),
[   87s]             (False, 'localhost'),
[   87s]         ),
[   87s]     )
[   87s]     @pytest.mark.parametrize(
[   87s]         'tls_verify_mode',
[   87s]         (
[   87s]             ssl.CERT_NONE,  # server shouldn't validate client cert
[   87s]             ssl.CERT_OPTIONAL,  # same as CERT_REQUIRED in client mode, don't use
[   87s]             ssl.CERT_REQUIRED,  # server should validate if client cert CA is OK
[   87s]         ),
[   87s]     )
[   87s]     def test_tls_client_auth(
[   87s]         # FIXME: remove twisted logic, separate tests
[   87s]         mocker,
[   87s]         tls_http_server, adapter_type,
[   87s]         ca,
[   87s]         tls_certificate,
[   87s]         tls_certificate_chain_pem_path,
[   87s]         tls_certificate_private_key_pem_path,
[   87s]         tls_ca_certificate_pem_path,
[   87s]         is_trusted_cert, tls_client_identity,
[   87s]         tls_verify_mode,
[   87s]     ):
[   87s]         """Verify that client TLS certificate auth works correctly."""
[   87s]         test_cert_rejection = (
[   87s]             tls_verify_mode != ssl.CERT_NONE
[   87s]             and not is_trusted_cert
[   87s]         )
[   87s]         interface, _host, port = _get_conn_data(ANY_INTERFACE_IPV4)
[   87s]     
[   87s]         client_cert_root_ca = ca if is_trusted_cert else trustme.CA()
[   87s]         with mocker.mock_module.patch(
[   87s]             'idna.core.ulabel',
[   87s]             return_value=ntob(tls_client_identity),
[   87s]         ):
[   87s]             client_cert = client_cert_root_ca.issue_server_cert(
[   87s]                 # FIXME: change to issue_cert once new trustme is out
[   87s]                 ntou(tls_client_identity),
[   87s]             )
[   87s]             del client_cert_root_ca
[   87s]     
[   87s]         with client_cert.private_key_and_cert_chain_pem.tempfile() as cl_pem:
[   87s]             tls_adapter_cls = get_ssl_adapter_class(name=adapter_type)
[   87s]             tls_adapter = tls_adapter_cls(
[   87s]                 tls_certificate_chain_pem_path,
[   87s]                 tls_certificate_private_key_pem_path,
[   87s]             )
[   87s]             if adapter_type == 'pyopenssl':
[   87s]                 tls_adapter.context = tls_adapter.get_context()
[   87s]                 tls_adapter.context.set_verify(
[   87s]                     _stdlib_to_openssl_verify[tls_verify_mode],
[   87s]                     lambda conn, cert, errno, depth, preverify_ok: preverify_ok,
[   87s]                 )
[   87s]             else:
[   87s]                 tls_adapter.context.verify_mode = tls_verify_mode
[   87s]     
[   87s]             ca.configure_trust(tls_adapter.context)
[   87s]             tls_certificate.configure_cert(tls_adapter.context)
[   87s]     
[   87s]             tlshttpserver = tls_http_server.send(
[   87s]                 (
[   87s]                     (interface, port),
[   87s]                     tls_adapter,
[   87s]                 ),
[   87s]             )
[   87s]     
[   87s]             interface, _host, port = _get_conn_data(tlshttpserver.bind_addr)
[   87s]     
[   87s]             make_https_request = functools.partial(
[   87s]                 requests.get,
[   87s]                 'https://' + interface + ':' + str(port) + '/',
[   87s]     
[   87s]                 # Server TLS certificate verification:
[   87s]                 verify=tls_ca_certificate_pem_path,
[   87s]     
[   87s]                 # Client TLS certificate verification:
[   87s]                 cert=cl_pem,
[   87s]             )
[   87s]     
[   87s]             if not test_cert_rejection:
[   87s]                 resp = make_https_request()
[   87s]                 is_req_successful = resp.status_code == 200
[   87s]                 if (
[   87s]                         not is_req_successful
[   87s]                         and IS_PYOPENSSL_SSL_VERSION_1_0
[   87s]                         and adapter_type == 'builtin'
[   87s]                         and tls_verify_mode == ssl.CERT_REQUIRED
[   87s]                         and tls_client_identity == 'localhost'
[   87s]                         and is_trusted_cert
[   87s]                 ):
[   87s]                     pytest.xfail(
[   87s]                         'OpenSSL 1.0 has problems with verifying client certs',
[   87s]                     )
[   87s]                 assert is_req_successful
[   87s]                 assert resp.text == 'Hello world!'
[   87s]                 return
[   87s]     
[   87s]             with pytest.raises(requests.exceptions.SSLError) as ssl_err:
[   87s] >               make_https_request()
[   87s] 
[   87s] cheroot/test/test_ssl.py:316: 
[   87s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   87s] /usr/lib/python2.7/site-packages/requests/api.py:75: in get
[   87s]     return request('get', url, params=params, **kwargs)
[   87s] /usr/lib/python2.7/site-packages/requests/api.py:60: in request
[   87s]     return session.request(method=method, url=url, **kwargs)
[   87s] /usr/lib/python2.7/site-packages/requests/sessions.py:533: in request
[   87s]     resp = self.send(prep, **send_kwargs)
[   87s] /usr/lib/python2.7/site-packages/requests/sessions.py:646: in send
[   87s]     r = adapter.send(request, **kwargs)
[   87s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   87s] 
[   87s] self = <requests.adapters.HTTPAdapter object at 0x7f8120a94a90>
[   87s] request = <PreparedRequest [GET]>, stream = False
[   87s] timeout = <urllib3.util.timeout.Timeout object at 0x7f8120a703d0>
[   87s] verify = '/tmp/tmphCe4LF.pem', cert = '/tmp/tmp1UR6JX.pem'
[   87s] proxies = OrderedDict()
[   87s] 
[   87s]     def send(self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None):
[   87s]         """Sends PreparedRequest object. Returns Response object.
[   87s]     
[   87s]         :param request: The :class:`PreparedRequest <PreparedRequest>` being sent.
[   87s]         :param stream: (optional) Whether to stream the request content.
[   87s]         :param timeout: (optional) How long to wait for the server to send
[   87s]             data before giving up, as a float, or a :ref:`(connect timeout,
[   87s]             read timeout) <timeouts>` tuple.
[   87s]         :type timeout: float or tuple or urllib3 Timeout object
[   87s]         :param verify: (optional) Either a boolean, in which case it controls whether
[   87s]             we verify the server's TLS certificate, or a string, in which case it
[   87s]             must be a path to a CA bundle to use
[   87s]         :param cert: (optional) Any user-provided SSL certificate to be trusted.
[   87s]         :param proxies: (optional) The proxies dictionary to apply to the request.
[   87s]         :rtype: requests.Response
[   87s]         """
[   87s]     
[   87s]         try:
[   87s]             conn = self.get_connection(request.url, proxies)
[   87s]         except LocationValueError as e:
[   87s]             raise InvalidURL(e, request=request)
[   87s]     
[   87s]         self.cert_verify(conn, request.url, verify, cert)
[   87s]         url = self.request_url(request, proxies)
[   87s]         self.add_headers(request, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies)
[   87s]     
[   87s]         chunked = not (request.body is None or 'Content-Length' in request.headers)
[   87s]     
[   87s]         if isinstance(timeout, tuple):
[   87s]             try:
[   87s]                 connect, read = timeout
[   87s]                 timeout = TimeoutSauce(connect=connect, read=read)
[   87s]             except ValueError as e:
[   87s]                 # this may raise a string formatting error.
[   87s]                 err = ("Invalid timeout {}. Pass a (connect, read) "
[   87s]                        "timeout tuple, or a single float to set "
[   87s]                        "both timeouts to the same value".format(timeout))
[   87s]                 raise ValueError(err)
[   87s]         elif isinstance(timeout, TimeoutSauce):
[   87s]             pass
[   87s]         else:
[   87s]             timeout = TimeoutSauce(connect=timeout, read=timeout)
[   87s]     
[   87s]         try:
[   87s]             if not chunked:
[   87s]                 resp = conn.urlopen(
[   87s]                     method=request.method,
[   87s]                     url=url,
[   87s]                     body=request.body,
[   87s]                     headers=request.headers,
[   87s]                     redirect=False,
[   87s]                     assert_same_host=False,
[   87s]                     preload_content=False,
[   87s]                     decode_content=False,
[   87s]                     retries=self.max_retries,
[   87s]                     timeout=timeout
[   87s]                 )
[   87s]     
[   87s]             # Send the request.
[   87s]             else:
[   87s]                 if hasattr(conn, 'proxy_pool'):
[   87s]                     conn = conn.proxy_pool
[   87s]     
[   87s]                 low_conn = conn._get_conn(timeout=DEFAULT_POOL_TIMEOUT)
[   87s]     
[   87s]                 try:
[   87s]                     low_conn.putrequest(request.method,
[   87s]                                         url,
[   87s]                                         skip_accept_encoding=True)
[   87s]     
[   87s]                     for header, value in request.headers.items():
[   87s]                         low_conn.putheader(header, value)
[   87s]     
[   87s]                     low_conn.endheaders()
[   87s]     
[   87s]                     for i in request.body:
[   87s]                         low_conn.send(hex(len(i))[2:].encode('utf-8'))
[   87s]                         low_conn.send(b'\r\n')
[   87s]                         low_conn.send(i)
[   87s]                         low_conn.send(b'\r\n')
[   87s]                     low_conn.send(b'0\r\n\r\n')
[   87s]     
[   87s]                     # Receive the response from the server
[   87s]                     try:
[   87s]                         # For Python 2.7, use buffering of HTTP responses
[   87s]                         r = low_conn.getresponse(buffering=True)
[   87s]                     except TypeError:
[   87s]                         # For compatibility with Python 3.3+
[   87s]                         r = low_conn.getresponse()
[   87s]     
[   87s]                     resp = HTTPResponse.from_httplib(
[   87s]                         r,
[   87s]                         pool=conn,
[   87s]                         connection=low_conn,
[   87s]                         preload_content=False,
[   87s]                         decode_content=False
[   87s]                     )
[   87s]                 except:
[   87s]                     # If we hit any problems here, clean up the connection.
[   87s]                     # Then, reraise so that we can handle the actual exception.
[   87s]                     low_conn.close()
[   87s]                     raise
[   87s]     
[   87s]         except (ProtocolError, socket.error) as err:
[   87s] >           raise ConnectionError(err, request=request)
[   87s] E           ConnectionError: ('Connection aborted.', error("(104, 'ECONNRESET')",))
[   87s] 
[   87s] /usr/lib/python2.7/site-packages/requests/adapters.py:498: ConnectionError
[   87s] ______________ test_tls_client_auth[2-False-localhost-pyopenssl] _______________
[   87s] 
[   87s] mocker = <pytest_mock.MockFixture object at 0x7f8120b22f90>
[   87s] tls_http_server = <generator object start_srv at 0x7f8120b51280>
[   87s] adapter_type = 'pyopenssl', ca = <trustme.CA object at 0x7f8120a94fd0>
[   87s] tls_certificate = <trustme.LeafCert object at 0x7f8120b3f950>
[   87s] tls_certificate_chain_pem_path = '/tmp/tmp6OVLwI.pem'
[   87s] tls_certificate_private_key_pem_path = '/tmp/tmpv6sN3S.pem'
[   87s] tls_ca_certificate_pem_path = '/tmp/tmpfuqxWc.pem', is_trusted_cert = False
[   87s] tls_client_identity = 'localhost', tls_verify_mode = 2
[   87s] 
[   87s]     @pytest.mark.parametrize(
[   87s]         'adapter_type',
[   87s]         (
[   87s]             'builtin',
[   87s]             'pyopenssl',
[   87s]         ),
[   87s]     )
[   87s]     @pytest.mark.parametrize(
[   87s]         'is_trusted_cert,tls_client_identity',
[   87s]         (
[   87s]             (True, 'localhost'), (True, '127.0.0.1'),
[   87s]             (True, '*.localhost'), (True, 'not_localhost'),
[   87s]             (False, 'localhost'),
[   87s]         ),
[   87s]     )
[   87s]     @pytest.mark.parametrize(
[   87s]         'tls_verify_mode',
[   87s]         (
[   87s]             ssl.CERT_NONE,  # server shouldn't validate client cert
[   87s]             ssl.CERT_OPTIONAL,  # same as CERT_REQUIRED in client mode, don't use
[   87s]             ssl.CERT_REQUIRED,  # server should validate if client cert CA is OK
[   87s]         ),
[   87s]     )
[   87s]     def test_tls_client_auth(
[   87s]         # FIXME: remove twisted logic, separate tests
[   87s]         mocker,
[   87s]         tls_http_server, adapter_type,
[   87s]         ca,
[   87s]         tls_certificate,
[   87s]         tls_certificate_chain_pem_path,
[   87s]         tls_certificate_private_key_pem_path,
[   87s]         tls_ca_certificate_pem_path,
[   87s]         is_trusted_cert, tls_client_identity,
[   87s]         tls_verify_mode,
[   87s]     ):
[   87s]         """Verify that client TLS certificate auth works correctly."""
[   87s]         test_cert_rejection = (
[   87s]             tls_verify_mode != ssl.CERT_NONE
[   87s]             and not is_trusted_cert
[   87s]         )
[   87s]         interface, _host, port = _get_conn_data(ANY_INTERFACE_IPV4)
[   87s]     
[   87s]         client_cert_root_ca = ca if is_trusted_cert else trustme.CA()
[   87s]         with mocker.mock_module.patch(
[   87s]             'idna.core.ulabel',
[   87s]             return_value=ntob(tls_client_identity),
[   87s]         ):
[   87s]             client_cert = client_cert_root_ca.issue_server_cert(
[   87s]                 # FIXME: change to issue_cert once new trustme is out
[   87s]                 ntou(tls_client_identity),
[   87s]             )
[   87s]             del client_cert_root_ca
[   87s]     
[   87s]         with client_cert.private_key_and_cert_chain_pem.tempfile() as cl_pem:
[   87s]             tls_adapter_cls = get_ssl_adapter_class(name=adapter_type)
[   87s]             tls_adapter = tls_adapter_cls(
[   87s]                 tls_certificate_chain_pem_path,
[   87s]                 tls_certificate_private_key_pem_path,
[   87s]             )
[   87s]             if adapter_type == 'pyopenssl':
[   87s]                 tls_adapter.context = tls_adapter.get_context()
[   87s]                 tls_adapter.context.set_verify(
[   87s]                     _stdlib_to_openssl_verify[tls_verify_mode],
[   87s]                     lambda conn, cert, errno, depth, preverify_ok: preverify_ok,
[   87s]                 )
[   87s]             else:
[   87s]                 tls_adapter.context.verify_mode = tls_verify_mode
[   87s]     
[   87s]             ca.configure_trust(tls_adapter.context)
[   87s]             tls_certificate.configure_cert(tls_adapter.context)
[   87s]     
[   87s]             tlshttpserver = tls_http_server.send(
[   87s]                 (
[   87s]                     (interface, port),
[   87s]                     tls_adapter,
[   87s]                 ),
[   87s]             )
[   87s]     
[   87s]             interface, _host, port = _get_conn_data(tlshttpserver.bind_addr)
[   87s]     
[   87s]             make_https_request = functools.partial(
[   87s]                 requests.get,
[   87s]                 'https://' + interface + ':' + str(port) + '/',
[   87s]     
[   87s]                 # Server TLS certificate verification:
[   87s]                 verify=tls_ca_certificate_pem_path,
[   87s]     
[   87s]                 # Client TLS certificate verification:
[   87s]                 cert=cl_pem,
[   87s]             )
[   87s]     
[   87s]             if not test_cert_rejection:
[   87s]                 resp = make_https_request()
[   87s]                 is_req_successful = resp.status_code == 200
[   87s]                 if (
[   87s]                         not is_req_successful
[   87s]                         and IS_PYOPENSSL_SSL_VERSION_1_0
[   87s]                         and adapter_type == 'builtin'
[   87s]                         and tls_verify_mode == ssl.CERT_REQUIRED
[   87s]                         and tls_client_identity == 'localhost'
[   87s]                         and is_trusted_cert
[   87s]                 ):
[   87s]                     pytest.xfail(
[   87s]                         'OpenSSL 1.0 has problems with verifying client certs',
[   87s]                     )
[   87s]                 assert is_req_successful
[   87s]                 assert resp.text == 'Hello world!'
[   87s]                 return
[   87s]     
[   87s]             with pytest.raises(requests.exceptions.SSLError) as ssl_err:
[   87s] >               make_https_request()
[   87s] 
[   87s] cheroot/test/test_ssl.py:316: 
[   87s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   87s] /usr/lib/python2.7/site-packages/requests/api.py:75: in get
[   87s]     return request('get', url, params=params, **kwargs)
[   87s] /usr/lib/python2.7/site-packages/requests/api.py:60: in request
[   87s]     return session.request(method=method, url=url, **kwargs)
[   87s] /usr/lib/python2.7/site-packages/requests/sessions.py:533: in request
[   87s]     resp = self.send(prep, **send_kwargs)
[   87s] /usr/lib/python2.7/site-packages/requests/sessions.py:646: in send
[   87s]     r = adapter.send(request, **kwargs)
[   87s] /usr/lib/python2.7/site-packages/requests/adapters.py:449: in send
[   87s]     timeout=timeout
[   87s] /usr/lib/python2.7/site-packages/urllib3/connectionpool.py:600: in urlopen
[   87s]     chunked=chunked)
[   87s] /usr/lib/python2.7/site-packages/urllib3/connectionpool.py:377: in _make_request
[   87s]     httplib_response = conn.getresponse(buffering=True)
[   87s] /usr/lib64/python2.7/httplib.py:1121: in getresponse
[   87s]     response.begin()
[   87s] /usr/lib64/python2.7/httplib.py:438: in begin
[   87s]     version, status, reason = self._read_status()
[   87s] /usr/lib64/python2.7/httplib.py:394: in _read_status
[   87s]     line = self.fp.readline(_MAXLINE + 1)
[   87s] /usr/lib64/python2.7/socket.py:480: in readline
[   87s]     data = self._sock.recv(self._rbufsize)
[   87s] /usr/lib/python2.7/site-packages/urllib3/contrib/pyopenssl.py:276: in recv
[   87s]     data = self.connection.recv(*args, **kwargs)
[   87s] /usr/lib/python2.7/site-packages/OpenSSL/SSL.py:1791: in recv
[   87s]     self._raise_ssl_error(self._ssl, result)
[   87s] /usr/lib/python2.7/site-packages/OpenSSL/SSL.py:1647: in _raise_ssl_error
[   87s]     _raise_current_error()
[   87s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   87s] 
[   87s] exception_type = <class 'OpenSSL.SSL.Error'>
[   87s] 
[   87s]     def exception_from_error_queue(exception_type):
[   87s]         """
[   87s]         Convert an OpenSSL library failure into a Python exception.
[   87s]     
[   87s]         When a call to the native OpenSSL library fails, this is usually signalled
[   87s]         by the return value, and an error code is stored in an error queue
[   87s]         associated with the current thread. The err library provides functions to
[   87s]         obtain these error codes and textual error messages.
[   87s]         """
[   87s]         errors = []
[   87s]     
[   87s]         while True:
[   87s]             error = lib.ERR_get_error()
[   87s]             if error == 0:
[   87s]                 break
[   87s]             errors.append((
[   87s]                 text(lib.ERR_lib_error_string(error)),
[   87s]                 text(lib.ERR_func_error_string(error)),
[   87s]                 text(lib.ERR_reason_error_string(error))))
[   87s]     
[   87s] >       raise exception_type(errors)
[   87s] E       Error: [('SSL routines', 'ssl3_read_bytes', 'tlsv1 alert unknown ca')]
[   87s] 
[   87s] /usr/lib/python2.7/site-packages/OpenSSL/_util.py:54: Error
[   87s] ========== 4 failed, 98 passed, 1 skipped, 3 xfailed in 69.23 seconds ==========
[   87s] error: Bad exit status from /var/tmp/rpm-tmp.A2bJoL (%check)
[   87s]

❓ What is the motivation / use case for changing the behavior?
The testsuite is failing when building openSUSE packages with OpenSSL
1.1.1 installed.

💡 To Reproduce
Steps to reproduce the behavior:

  1. Run the full testsuite
  2. Observe the error

💡 Expected behavior
The testsuite should pass without a failure.

📋 Details
Full build log

📋 Environment

  • Cheroot version: 6.5.5
  • CherryPy version: not available in the build environment
  • Python version: 2.7.16 and 3.7.2
  • OS: Linux openSUSE/Tumbleweed
  • Browser: NA

📋 Additional context
@webknjaz
Copy link
Member

@mcepl do you have a docker container where it's reproducible?
Also, could you please post the output of your python -m OpenSSL.debug?

@webknjaz webknjaz added bug Something is broken invalid This is irrelevant and removed bug Something is broken labels May 16, 2019
@webknjaz
Copy link
Member

You're hitting #173 which has been already fixed by using an appropriate urllib3 version.

You use outdated urllib3:

[    8s] python2-urllib3-1.24.2-2.3            ########################################

Use >=1.25 instead, it has TLS 1.3 handshake fixed: https://github.com/cherrypy/cheroot/blob/db55f85/setup.cfg#L105-L107 / https://travis-ci.org/cherrypy/cheroot/jobs/533146932#L219

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
invalid This is irrelevant
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mcepl @webknjaz