How to supress asyncio.exceptions.CancelledError ?

[HiFiPhile]

• Windows 10 21H2
• Python 3.9.7
• aiohttp 3.8.1

I wrote a simple test server for my embedded device, which receive POST data and serve update firmware with GET.

However I always get following exception, I think it's because my device only read the firmware header for version comparison then close the connection. If I use curl GET to read the whole file then no exception is raised.

Is there anyway to suppress this exception ?

Unhandled exception
Traceback (most recent call last):
  File "C:\Program Files\Python39\lib\asyncio\base_events.py", line 1186, in _sendfile_fallback
    await proto.drain()
  File "C:\Program Files\Python39\lib\asyncio\base_events.py", line 226, in drain
    await fut
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files\Python39\lib\site-packages\aiohttp\web_protocol.py", line 514, in start
    resp, reset = await task
  File "C:\Program Files\Python39\lib\site-packages\aiohttp\web_protocol.py", line 460, in _handle_request
    reset = await self.finish_response(request, resp, start_time)
  File "C:\Program Files\Python39\lib\site-packages\aiohttp\web_protocol.py", line 613, in finish_response
    await prepare_meth(request)
  File "C:\Program Files\Python39\lib\site-packages\aiohttp\web_fileresponse.py", line 286, in prepare
    return await self._sendfile(request, fobj, offset, count)
  File "C:\Program Files\Python39\lib\site-packages\aiohttp\web_fileresponse.py", line 99, in _sendfile
    await loop.sendfile(transport, fobj, offset, count)
  File "C:\Program Files\Python39\lib\asyncio\base_events.py", line 1162, in sendfile
    return await self._sendfile_fallback(transport, file,
  File "C:\Program Files\Python39\lib\asyncio\base_events.py", line 1192, in _sendfile_fallback
    await proto.restore()
  File "C:\Program Files\Python39\lib\asyncio\base_events.py", line 263, in restore
    self._transport.resume_reading()
  File "C:\Program Files\Python39\lib\asyncio\sslproto.py", line 343, in resume_reading
    self._ssl_protocol._transport.resume_reading()
AttributeError: 'NoneType' object has no attribute 'resume_reading'

from aiohttp import web
import ssl
import os
from datetime import datetime

async def handle_get(request):
    str = request.rel_url.query_string
    if str == 'firmware':
        return  web.FileResponse('../build/device.bin')
    else:
        return web.HTTPBadRequest()
   

async def handle_post(request):
    str = request.rel_url.query_string
    if str == 'data':
        data = await request.json()
        dateTimeObj = datetime.now()
        print(dateTimeObj)
        print(data)
        res = {"interval": 15, "fw": 1000}
        return web.json_response(res)
    else:
        return web.HTTPBadRequest()


def main():
    os.environ['SSLKEYLOGFILE'] = 'ssl.log'
    app = web.Application()
    app.add_routes([web.get('/device', handle_get),
                    web.post('/device', handle_post)])

    ssl_context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
    ssl_context.load_cert_chain('server.pem', 'server.key')

    try:
        web.run_app(app, ssl_context=ssl_context, port=8200)
    except:
        pass
if __name__ == "__main__":
    # call to main function
    main()

[Dreamsorcerer]

Looks like it could be a tricky one. If you can write a complete test as a PR, that would be really helpful, then we can merge the test and figure out a way to fix the problem.

[HiFiPhile]

I did more tests, seems it happens when client close the connection ungracefully:

With curl I can trigger the error with curl --cacert client.pem https://127.0.0.1:8200/device?firmware --max-filesize 100, but with aiohttp_client of the test how to close the connection on partial read ?

[Dreamsorcerer]

I'm assuming it would work if you just read 1 byte or similar and then close the connection.

i.e. Roughly:

async with client.get() as resp:
    await resp.content.read(1)

You may need to make sure the content is above a certain size, as it might preload a chunk of data when reading the headers or something, if I remember correctly.

[HiFiPhile]

I messed with pytest without success, even with existing tests I've some error like DeprecationWarning: The object should be created within an async function

However I wrote a full script:

from aiohttp import web
import aiohttp
import ssl
import asyncio
import os
import signal
import sys

async def client():
    # wait server
    await asyncio.sleep(3)
    
    ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
    ssl_context.load_verify_locations('client.pem')
    async with aiohttp.ClientSession() as session:
        async with session.get('https://127.0.0.1:8200', ssl_context=ssl_context) as resp:
            await resp.content.read(1)

async def server():
    async def handle_get(request):
        return web.FileResponse('newfile')

    # dummy file
    f = open('newfile',"wb")
    f.seek(64*1024*1024)
    f.write(b"\0")
    f.close()

    app = web.Application()
    app.add_routes([web.get('', handle_get)])

    ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
    ssl_context.load_cert_chain('server.pem', 'server.key')
    
    loop = asyncio.get_event_loop()
    loop.create_task(client())

    # set up the web server
    runner = web.AppRunner(app)
    await runner.setup()
    await web.TCPSite(runner, ssl_context=ssl_context, port=8200).start()

    # wait forever, running both the web server and the tasks
    await asyncio.Event().wait()

def signal_handler(signal, frame):
    os.remove("newfile")
    sys.exit(0)

signal.signal(signal.SIGINT, signal_handler)
asyncio.get_event_loop().run_until_complete(server())

test script with ssl crts