Skip to content

certificate is not valid for IP #5110

New issue
New issue
Closed
#5118
Closed
certificate is not valid for IP#5110
#5118
Labels
bug
@lilydjwg

Description

@lilydjwg
lilydjwg
opened on Oct 24, 2020 · edited by lilydjwg

🐞 Describe the bug

When requesting with IPv6 enabled, I get this weird exception.

💡 To Reproduce

  1. python3 -m venv venv and . venv/bin/activate
  2. pip install -U aiohttp
  3. Run code:
import asyncio
import aiohttp

async def main():
  async with aiohttp.ClientSession() as session:
    r = await session.get('https://pypi.org/')
    print(await r.text())

if __name__ == '__main__':
  asyncio.run(main())
  1. See error.

💡 Expected behavior

📋 Logs/tracebacks

Traceback (most recent call last): 
  File "/home/lilydjwg/tmpfs/venv/lib/python3.8/site-packages/aiohttp/connector.py", line 946, in _wrap_create_connection  
    return await self._loop.create_connection(*args, **kwargs)  # type: ignore  # noqa
  File "/usr/lib/python3.8/asyncio/base_events.py", line 1050, in create_connection                                      
    transport, protocol = await self._create_connection_transport(
  File "/usr/lib/python3.8/asyncio/base_events.py", line 1080, in _create_connection_transport                                              await waiter                                                                                                                          File "/usr/lib/python3.8/asyncio/sslproto.py", line 529, in data_received
    ssldata, appdata = self._sslpipe.feed_ssldata(data)
  File "/usr/lib/python3.8/asyncio/sslproto.py", line 189, in feed_ssldata                                                             
    self._sslobj.do_handshake()
  File "/usr/lib/python3.8/ssl.py", line 944, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: IP address mismatch, certificate is not valid 
for '151.101.128.223'. (_ssl.c:1124)

The above exception was the direct cause of the following exception: 

Traceback (most recent call last):
  File "t.py", line 12, in <module>
    asyncio.run(main())
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "t.py", line 8, in main
    r = await session.get('https://pypi.org/')
  File "/home/lilydjwg/tmpfs/venv/lib/python3.8/site-packages/aiohttp/client.py", line 490, in _request
    conn = await self._connector.connect(
  File "/home/lilydjwg/tmpfs/venv/lib/python3.8/site-packages/aiohttp/connector.py", line 528, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "/home/lilydjwg/tmpfs/venv/lib/python3.8/site-packages/aiohttp/connector.py", line 868, in _create_connection
    _, proto = await self._create_direct_connection(
  File "/home/lilydjwg/tmpfs/venv/lib/python3.8/site-packages/aiohttp/connector.py", line 1023, in _create_direct_connection
    raise last_exc
  File "/home/lilydjwg/tmpfs/venv/lib/python3.8/site-packages/aiohttp/connector.py", line 999, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
  File "/home/lilydjwg/tmpfs/venv/lib/python3.8/site-packages/aiohttp/connector.py", line 948, in _wrap_create_connection
    raise ClientConnectorCertificateError(
aiohttp.client_exceptions.ClientConnectorCertificateError: Cannot connect to host pypi.org:443 ssl:True [SSLCertVerificationError: (1, "[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: IP address mismatch, certificate is not valid for '151.101.128.223'. (_ssl.c:1124)")]

📋 Your version of the Python

$ python --version
Python 3.8.6

📋 Your version of the aiohttp/yarl/multidict distributions

$ python -m pip show aiohttp
Name: aiohttp
Version: 3.7.0
Summary: Async http client/server framework (asyncio)
Home-page: https://github.com/aio-libs/aiohttp
Author: Nikolay Kim
Author-email: fafhrd91@gmail.com
License: Apache 2
Location: /home/lilydjwg/tmpfs/venv/lib/python3.8/site-packages
Requires: yarl, attrs, async-timeout, chardet, multidict
Required-by: 
$ python -m pip show multidict
Name: multidict
Version: 5.0.0
Summary: multidict implementation
Home-page: https://github.com/aio-libs/multidict
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
Location: /home/lilydjwg/tmpfs/venv/lib/python3.8/site-packages
Requires: 
Required-by: yarl, aiohttp
$ python -m pip show yarl
Name: yarl
Version: 1.6.2
Summary: Yet another URL library
Home-page: https://github.com/aio-libs/yarl/
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
Location: /home/lilydjwg/tmpfs/venv/lib/python3.8/site-packages
Requires: idna, multidict
Required-by: aiohttp

📋 Additional context

I find this when I'm trying to reproduce another error.

Activity

lilydjwg
added
bug
on Oct 24, 2020
lilydjwg
mentioned this on Oct 24, 2020
djmitche

djmitche commented on Oct 24, 2020

@djmitche
djmitche
on Oct 24, 2020
Contributor

Seeing the same with python 3.6 and 3.7.

Logs are here and here.

It looks like aiohttp is passing an IP to the SSL verification.

djmitche

djmitche commented on Oct 24, 2020

@djmitche
djmitche
on Oct 24, 2020
Contributor

Repro for python 3.6:

import aiohttp

loop = asyncio.get_event_loop()

async def main():
  async with aiohttp.ClientSession(loop=loop) as session:                                                                                                                                                                                                                                                                                                                                      
    r = await session.get('https://pypi.org/')
    print(await r.text())

if __name__ == '__main__':
  loop.run_until_complete(main())
djmitche

djmitche commented on Oct 24, 2020

@djmitche
djmitche
on Oct 24, 2020
Contributor

In aiohttp/connector.py it looks like hinfo['hostname'] is a name in aiohttp==3.6.3 and an IP in aiohttp==3.7.0.

djmitche

djmitche commented on Oct 24, 2020

@djmitche
djmitche
on Oct 24, 2020
Contributor

In the good case, self._resolver.resolve (a ThreadedResolver) returns

resolver -> [{'hostname': 'pypi.org', 'host': '151.101.192.223', 'port': 443, 'family': <AddressFamily.AF_INET: 2>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICHOST: 4>}, {'hostname': 'pypi.org', 'host': '151.101.128.223', 'port': 443, 'family': <AddressFamily.AF_INET: 2>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICHOST: 4>}, {'hostname': 'pypi.org', 'host': '151.101.64.223', 'po rt': 443, 'family': <AddressFamily.AF_INET: 2>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICHOST: 4>}, {'hostname': 'pypi.org', 'host': '151.101.0.223', 'port': 443, 'family': <AddressFamily.AF_INET: 2>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICHOST: 4>}, {'hostname': 'pypi.org', 'host': '2a04:4e42::223', 'port': 443, 'family': <AddressFamily.AF_INET6: 10>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICHOST: 4>}, {'hostname': 'pypi.org', 'host': '2a04:4e42:600::223', 'port': 443, 'family': <AddressFamily.AF_INET6: 10>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICHOST: 4>}, {'hostname': 'pypi.org', 'host': '2a04:4e42:400::223', 'port': 443, 'family': <AddressFamily.AF_INET6: 10>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICHOST: 4>}, {'hostname': 'pypi.org ', 'host': '2a04:4e42:200::223', 'port': 443, 'family': <AddressFamily.AF_INET6: 10>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICHOST: 4>}]

In the bad case,

[{'hostname': '151.101.192.223', 'host': '151.101.192.223', 'port': 443, 'family': <AddressFamily.AF_INET: 2>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICSERV|AI_NUMERICHOST: 1028>}, {'hostname': '151.101.128.223', 'host': '151.101.128.223', 'port': 443, 'family': <AddressFamily.AF_INET: 2>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICSERV|AI_NUMERICHOST: 1028>}, {'hostname': '15 1.101.64.223', 'host': '151.101.64.223', 'port': 443, 'family': <AddressFamily.AF_INET: 2>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICSERV|AI_NUMERICHOST: 1028>}, {'hostname': '151.101.0.223', 'host': '151.101.0.223', 'port': 443, 'family': <AddressFamily.AF_INET: 2>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICSERV|AI_NUMERICHOST: 1028>}, {'hostname': '2a04:4e42::223', 'host': '2a04:4e42::223', 'port': 443, 'family': <AddressFamily.AF_INET6: 10>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICSERV|AI_NUMERICHOST: 1028>}, {'hostname': '2a04:4e42:600::223', 'host': '2a04:4e42:600::223', 'port': 443, 'family': <AddressFamily.AF_INET6: 10>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICSERV|AI_NUMERICHOST: 1028>}, {'hostname': '2a04:4e42:400::223', 'host': '2a04 :4e42:400::223', 'port': 443, 'family': <AddressFamily.AF_INET6: 10>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICSERV|AI_NUMERICHOST: 1028>}, {'hostname': '2a04:4e42:200::223', 'host': '2a04:4e42:200::223', 'port': 443, 'family': <AddressFamily.AF_INET6: 10>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICSERV|AI_NUMERICHOST: 1028>}]

djmitche

djmitche commented on Oct 24, 2020

@djmitche
djmitche
on Oct 24, 2020
Contributor

Old resolve:

aiohttp/aiohttp/resolver.py

Lines 27 to 40 in 1940503

async def resolve(self, host: str, port: int=0,
family: int=socket.AF_INET) -> List[Dict[str, Any]]:
infos = await self._loop.getaddrinfo(
host, port, type=socket.SOCK_STREAM, family=family)
hosts = []
for family, _, proto, _, address in infos:
hosts.append(
{'hostname': host,
'host': address[0], 'port': address[1],
'family': family, 'proto': proto,
'flags': socket.AI_NUMERICHOST})
return hosts

New resolve:

aiohttp/aiohttp/resolver.py

Lines 28 to 53 in cac3cd2

async def resolve(self, host: str, port: int=0,
family: int=socket.AF_INET) -> List[Dict[str, Any]]:
infos = await self._loop.getaddrinfo(
host, port, type=socket.SOCK_STREAM, family=family)
hosts = []
for family, _, proto, _, address in infos:
if family == socket.AF_INET6 and address[3]: # type: ignore
# This is essential for link-local IPv6 addresses.
# LL IPv6 is a VERY rare case. Strictly speaking, we should use
# getnameinfo() unconditionally, but performance makes sense.
host, _port = socket.getnameinfo(
address, socket.NI_NUMERICHOST | socket.NI_NUMERICSERV)
port = int(_port)
else:
host, port = address[:2]
hosts.append({
'hostname': host,
'host': host,
'port': port,
'family': family,
'proto': proto,
'flags': socket.AI_NUMERICHOST | socket.AI_NUMERICSERV,
})
return hosts

so it looks like this is a simple matter of variable shadowing. However, it's a very serious error!

djmitche
mentioned this on Oct 24, 2020
asvetlov
closed this as completedin #5118on Oct 24, 2020
bdraco
mentioned this on Oct 24, 2020
djmitche
added a commit that references this issue on Oct 25, 2020

Don't use aiohttp==3.7.0

035960f
markusressel
mentioned this on Oct 25, 2020
jiri-janousek
added a commit that references this issue on Oct 25, 2020

Pin aiohttp[speedups] ==3.6.*

2d1dadb
daviddavis
mentioned this in 2 pull requests on Oct 25, 2020
Harmon758
mentioned this on Nov 22, 2020
gozdal
mentioned this on Jan 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bug

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      Participants

      @djmitche@lilydjwg

      Issue actions
        certificate is not valid for IP · Issue #5110 · aio-libs/aiohttp