Agent server fails to handle asyncio.CancelledError from create_kernels()

[rapsealk]

What Operating System(s) are you seeing this problem on?

macOS (Apple Silicon)

Backend.AI version

b2c4a7a

Describe the bug

The agent server fails to handle asyncio.CancelledError in some cases:

2023-03-06 11:07:07.884 ERROR callosum.rpc.channel.Peer [16442] RPC user error
Traceback (most recent call last):
  File "/Users/rapsealk/Desktop/git/backend.ai-dev/dist/export/python/virtualenvs/python-default/3.11.2/lib/python3.11/site-packages/callosum/rpc/channel.py", line 281, in _func_task
    result = await self._func_scheduler.get_fut(server_request_id)
    ^^^^^^^^^^^^^^^^^
  File "/Users/rapsealk/Desktop/git/backend.ai-dev/dist/export/python/virtualenvs/python-default/3.11.2/lib/python3.11/site-packages/callosum/ordering.py", line 224, in get_fut
    return await task
  File "/Users/rapsealk/Desktop/git/backend.ai-dev/src/ai/backend/agent/server.py", line 158, in _inner
    return await meth(
    ^^^^^^^^^^^^^^^^^
  File "/Users/rapsealk/Desktop/git/backend.ai-dev/src/ai/backend/agent/server.py", line 133, in _inner
    return await meth(self, *args, **kwargs)
    ^^^^^^^^^^^^^^^^^
  File "/Users/rapsealk/Desktop/git/backend.ai-dev/src/ai/backend/agent/server.py", line 386, in create_kernels
    raw_results = [
  File "/Users/rapsealk/Desktop/git/backend.ai-dev/src/ai/backend/agent/server.py", line 388, in <listcomp>
    "id": str(result["id"]),
    ^^^^^^^^^^^^^^^^^
TypeError: 'CancelledError' object is not subscriptable

asyncio.CancelledError has been a subclass of BaseException rather than Exception since Python 3.8. Therefore, it cannot be filtered by condition filter(lambda item: isinstance(item, Exception)).

backend.ai/src/ai/backend/agent/server.py

Line 380 in b2c4a7a

errors = [*filter(lambda item: isinstance(item, Exception), results)]

To Reproduce

No response

Expected Behavior

Raises an exception and does not attempt to read values by index.

Anything else?

No response

[achimnol]

That part of code has been refactored in #1771.

backend.ai/src/ai/backend/agent/server.py

Lines 454 to 457 in 0176e13

	match result:
	case BaseException():
	errors.append(result)
	case _:

It should not have this issue for 23.09 and 24.03 releases.