Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test under py3.10 #1070

Merged
merged 27 commits into from Jul 30, 2021
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
a51908a
avoid deprecated asyncio.get_event_loop
graingert Jun 4, 2021
d2e8f67
test on py3.10
graingert Jun 7, 2021
16ee7de
get mkdocs from git for py3.10 support
graingert Jun 7, 2021
9f94120
a loop must now be running to construct a WebSocketProtocol
graingert Jun 8, 2021
d03333d
fix py3.10 ssl deprecation warnings
graingert Jun 8, 2021
224f25b
work around websockets issues
graingert Jun 8, 2021
15152fc
work around httpx issues
graingert Jun 8, 2021
8fa30ec
add a test that the Config(ssl_certfile) can provide the key, cert an…
graingert Jun 8, 2021
9a9ce8f
regenerate usage
graingert Jun 8, 2021
d87c971
ssl.TLSVersion is only available on 3.7+
graingert Jun 8, 2021
0e687d8
remove trustme git pin
graingert Jun 8, 2021
b0c171d
use released httpx version
graingert Jun 17, 2021
1e2f2dc
remove workaround for fixed https://github.com/aaugustin/websockets/i…
graingert Jun 17, 2021
871f01d
Update .github/workflows/test-suite.yml
graingert Jun 18, 2021
1c97ac8
Merge branch 'master' into avoid-deprecated-get-event-loop
graingert Jun 30, 2021
81f9d28
fix missing colon
graingert Jul 5, 2021
5678b1a
bump websockets
graingert Jul 5, 2021
cf84eea
Merge branch 'master' of git://github.com/encode/uvicorn into avoid-d…
graingert Jul 25, 2021
69ccd10
mkdocs is on pypi now
graingert Jul 25, 2021
dd9621c
use the new 3.10 beta4
graingert Jul 25, 2021
e881ab0
only use unreleased websockets on py3.10
graingert Jul 25, 2021
a3a1f45
fix websockets logger kwarg
graingert Jul 25, 2021
9c7de99
fix test_loop_auto
graingert Jul 25, 2021
b35ac26
avoid more deprecated get_event_loop calls
graingert Jul 25, 2021
9d6db97
bump websockets git commit pin
graingert Jul 26, 2021
d5a3d25
remove get_running_loop() test
graingert Jul 30, 2021
a47f259
remove unused import
graingert Jul 30, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/test-suite.yml
Expand Up @@ -13,7 +13,7 @@ jobs:
runs-on: "${{ matrix.os }}"
strategy:
matrix:
python-version: ["3.6", "3.7", "3.8", "3.9"]
python-version: ["3.6", "3.7", "3.8", "3.9", "3.10.0-beta.2"]
graingert marked this conversation as resolved.
Show resolved Hide resolved
os: [windows-latest, ubuntu-latest]
steps:
- uses: "actions/checkout@v2"
Expand Down
2 changes: 1 addition & 1 deletion docs/deployment.md
Expand Up @@ -86,7 +86,7 @@ Options:
--ssl-certfile TEXT SSL certificate file
--ssl-keyfile-password TEXT SSL keyfile password
--ssl-version INTEGER SSL version to use (see stdlib ssl module's)
[default: 2]
[default: 17]
euri10 marked this conversation as resolved.
Show resolved Hide resolved
--ssl-cert-reqs INTEGER Whether client certificate is required (see
stdlib ssl module's) [default: 0]
--ssl-ca-certs TEXT CA certificates file
Expand Down
2 changes: 1 addition & 1 deletion docs/index.md
Expand Up @@ -156,7 +156,7 @@ Options:
--ssl-certfile TEXT SSL certificate file
--ssl-keyfile-password TEXT SSL keyfile password
--ssl-version INTEGER SSL version to use (see stdlib ssl module's)
[default: 2]
[default: 17]
--ssl-cert-reqs INTEGER Whether client certificate is required (see
stdlib ssl module's) [default: 0]
--ssl-ca-certs TEXT CA certificates file
Expand Down
8 changes: 6 additions & 2 deletions requirements.txt
Expand Up @@ -18,12 +18,16 @@ mypy
trustme
cryptography
coverage
httpx==0.16.*
httpx>=0.16.0
graingert marked this conversation as resolved.
Show resolved Hide resolved
pytest-asyncio==0.14.*
async_generator; python_version < '3.7'


# Documentation
mkdocs
# from git due to https://github.com/mkdocs/mkdocs/pull/2209#issuecomment-855703485
https://github.com/mkdocs/mkdocs/archive/b221df94fa7f29a2e63edfaeda56e3285b183f91.tar.gz#egg=mkdocs
graingert marked this conversation as resolved.
Show resolved Hide resolved
mkdocs-material

# py3.10 workarounds
https://github.com/aaugustin/websockets/archive/cb11516e0ed4fe2b67ab6c1511650bd42115d0b6.tar.gz#egg=websockets; python_version > '3.6'
graingert marked this conversation as resolved.
Show resolved Hide resolved
https://github.com/encode/httpx/archive/7eb610e606f4b95260c4835de4e9147477dcc6d7.tar.gz#egg=httpx
graingert marked this conversation as resolved.
Show resolved Hide resolved
26 changes: 19 additions & 7 deletions tests/conftest.py
Expand Up @@ -13,7 +13,7 @@ def tls_certificate_authority() -> trustme.CA:

@pytest.fixture
def tls_certificate(tls_certificate_authority: trustme.CA) -> trustme.LeafCert:
return tls_certificate_authority.issue_server_cert(
euri10 marked this conversation as resolved.
Show resolved Hide resolved
return tls_certificate_authority.issue_cert(
"localhost",
"127.0.0.1",
"::1",
Expand All @@ -33,9 +33,9 @@ def tls_ca_certificate_private_key_path(tls_certificate_authority: trustme.CA):


@pytest.fixture
def tls_ca_certificate_private_key_encrypted_path(tls_certificate_authority):
def tls_certificate_private_key_encrypted_path(tls_certificate):
private_key = serialization.load_pem_private_key(
tls_certificate_authority.private_key_pem.bytes(),
tls_certificate.private_key_pem.bytes(),
password=None,
backend=default_backend(),
)
Expand All @@ -49,13 +49,25 @@ def tls_ca_certificate_private_key_encrypted_path(tls_certificate_authority):


@pytest.fixture
def tls_certificate_pem_path(tls_certificate: trustme.LeafCert):
def tls_certificate_private_key_path(tls_certificate: trustme.CA):
with tls_certificate.private_key_pem.tempfile() as private_key:
yield private_key


@pytest.fixture
def tls_certificate_key_and_chain_path(tls_certificate: trustme.LeafCert):
with tls_certificate.private_key_and_cert_chain_pem.tempfile() as cert_pem:
yield cert_pem


@pytest.fixture
def tls_ca_ssl_context(tls_certificate: trustme.LeafCert) -> ssl.SSLContext:
ssl_ctx = ssl.SSLContext()
tls_certificate.configure_cert(ssl_ctx)
def tls_certificate_server_cert_path(tls_certificate: trustme.LeafCert):
with tls_certificate.cert_chain_pems[0].tempfile() as cert_pem:
yield cert_pem
euri10 marked this conversation as resolved.
Show resolved Hide resolved


@pytest.fixture
def tls_ca_ssl_context(tls_certificate_authority: trustme.CA) -> ssl.SSLContext:
ssl_ctx = ssl.create_default_context(ssl.Purpose.SERVER_AUTH)
tls_certificate_authority.configure_trust(ssl_ctx)
return ssl_ctx
euri10 marked this conversation as resolved.
Show resolved Hide resolved
23 changes: 19 additions & 4 deletions tests/protocols/test_http.py
@@ -1,3 +1,4 @@
import asyncio
import contextlib
import logging

Expand Down Expand Up @@ -123,12 +124,15 @@ def set_protocol(self, protocol):
pass


class MockLoop:
class MockLoop(asyncio.AbstractEventLoop):
def __init__(self, event_loop):
self.tasks = []
self.later = []
self.loop = event_loop

def is_running(self):
return True

def create_task(self, coroutine):
self.tasks.insert(0, coroutine)
return MockTask()
Expand All @@ -138,7 +142,14 @@ def call_later(self, delay, callback, *args):

def run_one(self):
coroutine = self.tasks.pop()
self.loop.run_until_complete(coroutine)
self.run_until_complete(coroutine)

def run_until_complete(self, coroutine):
asyncio._set_running_loop(None)
try:
return self.loop.run_until_complete(coroutine)
finally:
asyncio._set_running_loop(self)

def close(self):
self.loop.close()
Expand All @@ -161,13 +172,17 @@ def add_done_callback(self, callback):
@contextlib.contextmanager
def get_connected_protocol(app, protocol_cls, event_loop, **kwargs):
loop = MockLoop(event_loop)
asyncio._set_running_loop(loop)
transport = MockTransport()
config = Config(app=app, **kwargs)
server_state = ServerState()
protocol = protocol_cls(config=config, server_state=server_state, _loop=loop)
protocol.connection_made(transport)
yield protocol
protocol.loop.close()
try:
yield protocol
finally:
protocol.loop.close()
asyncio._set_running_loop(None)
euri10 marked this conversation as resolved.
Show resolved Hide resolved


@pytest.mark.parametrize("protocol_cls", HTTP_PROTOCOLS)
Expand Down
8 changes: 6 additions & 2 deletions tests/test_auto_detection.py
@@ -1,5 +1,7 @@
import asyncio

import pytest

from uvicorn.config import Config
from uvicorn.loops.auto import auto_loop_setup
from uvicorn.main import ServerState
Expand Down Expand Up @@ -39,15 +41,17 @@ def test_loop_auto():
assert type(policy).__module__.startswith(expected_loop)


def test_http_auto():
@pytest.mark.asyncio
euri10 marked this conversation as resolved.
Show resolved Hide resolved
async def test_http_auto():
config = Config(app=app)
server_state = ServerState()
protocol = AutoHTTPProtocol(config=config, server_state=server_state)
expected_http = "H11Protocol" if httptools is None else "HttpToolsProtocol"
assert type(protocol).__name__ == expected_http


def test_websocket_auto():
@pytest.mark.asyncio
async def test_websocket_auto():
config = Config(app=app)
server_state = ServerState()
protocol = AutoWebSocketsProtocol(config=config, server_state=server_state)
Expand Down
4 changes: 2 additions & 2 deletions tests/test_config.py
Expand Up @@ -156,10 +156,10 @@ def test_ssl_config(tls_ca_certificate_pem_path, tls_ca_certificate_private_key_
assert config.is_ssl is True


def test_ssl_config_combined(tls_certificate_pem_path):
def test_ssl_config_combined(tls_certificate_key_and_chain_path):
config = Config(
app=asgi_app,
ssl_certfile=tls_certificate_pem_path,
ssl_certfile=tls_certificate_key_and_chain_path,
)
config.load()

Expand Down
34 changes: 26 additions & 8 deletions tests/test_ssl.py
Expand Up @@ -13,14 +13,17 @@ async def app(scope, receive, send):

@pytest.mark.asyncio
async def test_run(
tls_ca_ssl_context, tls_ca_certificate_pem_path, tls_ca_certificate_private_key_path
tls_ca_ssl_context,
tls_certificate_server_cert_path,
tls_certificate_private_key_path,
tls_ca_certificate_pem_path,
):
config = Config(
app=app,
loop="asyncio",
limit_max_requests=1,
ssl_keyfile=tls_ca_certificate_private_key_path,
ssl_certfile=tls_ca_certificate_pem_path,
ssl_keyfile=tls_certificate_private_key_path,
ssl_certfile=tls_certificate_server_cert_path,
ssl_ca_certs=tls_ca_certificate_pem_path,
)
async with run_server(config):
Expand All @@ -31,13 +34,13 @@ async def test_run(

@pytest.mark.asyncio
async def test_run_chain(
tls_ca_ssl_context, tls_certificate_pem_path, tls_ca_certificate_pem_path
tls_ca_ssl_context, tls_certificate_key_and_chain_path, tls_ca_certificate_pem_path
):
config = Config(
app=app,
loop="asyncio",
limit_max_requests=1,
ssl_certfile=tls_certificate_pem_path,
ssl_certfile=tls_certificate_key_and_chain_path,
ssl_ca_certs=tls_ca_certificate_pem_path,
)
async with run_server(config):
Expand All @@ -46,18 +49,33 @@ async def test_run_chain(
assert response.status_code == 204


@pytest.mark.asyncio
async def test_run_chain_only(tls_ca_ssl_context, tls_certificate_key_and_chain_path):
config = Config(
app=app,
loop="asyncio",
limit_max_requests=1,
ssl_certfile=tls_certificate_key_and_chain_path,
)
async with run_server(config):
async with httpx.AsyncClient(verify=tls_ca_ssl_context) as client:
response = await client.get("https://127.0.0.1:8000")
assert response.status_code == 204


@pytest.mark.asyncio
async def test_run_password(
tls_ca_ssl_context,
tls_certificate_server_cert_path,
tls_ca_certificate_pem_path,
tls_ca_certificate_private_key_encrypted_path,
tls_certificate_private_key_encrypted_path,
):
config = Config(
app=app,
loop="asyncio",
limit_max_requests=1,
ssl_keyfile=tls_ca_certificate_private_key_encrypted_path,
ssl_certfile=tls_ca_certificate_pem_path,
ssl_keyfile=tls_certificate_private_key_encrypted_path,
ssl_certfile=tls_certificate_server_cert_path,
ssl_keyfile_password="uvicorn password for the win",
ssl_ca_certs=tls_ca_certificate_pem_path,
)
Expand Down
3 changes: 1 addition & 2 deletions uvicorn/config.py
Expand Up @@ -66,8 +66,7 @@
INTERFACES = ["auto", "asgi3", "asgi2", "wsgi"]


# Fallback to 'ssl.PROTOCOL_SSLv23' in order to support Python < 3.5.3.
SSL_PROTOCOL_VERSION = getattr(ssl, "PROTOCOL_TLS", ssl.PROTOCOL_SSLv23)
SSL_PROTOCOL_VERSION = ssl.PROTOCOL_TLS_SERVER


LOGGING_CONFIG = {
Expand Down
16 changes: 2 additions & 14 deletions uvicorn/loops/asyncio.py
@@ -1,19 +1,7 @@
import asyncio
import platform
import selectors
import sys


def asyncio_setup() -> None: # pragma: no cover
loop: asyncio.AbstractEventLoop
if (
sys.version_info.major >= 3
and sys.version_info.minor >= 8
and platform.system() == "Windows"
):
selector = selectors.SelectSelector()
loop = asyncio.SelectorEventLoop(selector)
asyncio.set_event_loop(loop)
else:
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
Comment on lines -17 to -19
Copy link
Sponsor Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the cause of the httpx tests failing. @tomchristie

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Kludex - No idea how you tracked that down, but... 👍

if sys.version_info >= (3, 8) and sys.platform == "win32":
euri10 marked this conversation as resolved.
Show resolved Hide resolved
asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
euri10 marked this conversation as resolved.
Show resolved Hide resolved
3 changes: 2 additions & 1 deletion uvicorn/protocols/websockets/websockets_impl.py
Expand Up @@ -4,7 +4,8 @@
from typing import Callable
from urllib.parse import unquote

import websockets
# work around https://github.com/aaugustin/websockets/issues/989
import websockets.exceptions
graingert marked this conversation as resolved.
Show resolved Hide resolved
from websockets.extensions.permessage_deflate import ServerPerMessageDeflateFactory

from uvicorn.protocols.utils import get_local_addr, get_remote_addr, is_ssl
Expand Down
5 changes: 3 additions & 2 deletions uvicorn/server.py
Expand Up @@ -46,8 +46,9 @@ def __init__(self, config):

def run(self, sockets=None):
self.config.setup_event_loop()
loop = asyncio.get_event_loop()
loop.run_until_complete(self.serve(sockets=sockets))
if sys.version_info >= (3, 7):
return asyncio.run(self.serve(sockets=sockets))
return asyncio.get_event_loop().run_until_complete(self.serve(sockets=sockets))

async def serve(self, sockets=None):
process_id = os.getpid()
Expand Down