Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optional uvloop use #2264

Merged
merged 72 commits into from Dec 23, 2021
Merged
Show file tree
Hide file tree
Changes from 59 commits
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
283911d
Make use of uvloop optional
prryplatypus Oct 3, 2021
1a034fc
Replace `USE_UVLOOP` with `NO_UVLOOP` for consistency
prryplatypus Oct 3, 2021
6f776c1
Fix linting
prryplatypus Oct 3, 2021
e6e0cd6
Rename `NO_UVLOOP` to `USE_UVLOOP` in config
prryplatypus Oct 4, 2021
b3ff392
Added warning messages
prryplatypus Oct 9, 2021
93c814a
Tweaked warning message
prryplatypus Oct 9, 2021
d00b868
Fix linting
prryplatypus Oct 9, 2021
48a54f8
Better message
prryplatypus Oct 9, 2021
9576055
Added tests
prryplatypus Oct 9, 2021
33eed11
Remove previous tests
prryplatypus Oct 10, 2021
7f69e43
Run event loop configuration on startup
prryplatypus Oct 10, 2021
257b239
Configure event loop using ASGI mode too
prryplatypus Oct 10, 2021
4bfc09f
Default `USE_UVLOOP` to `True`
prryplatypus Oct 11, 2021
cab4d59
First attempt tests (#1)
prryplatypus Oct 11, 2021
575837d
Merge branch 'feat/optional-uvloop-use' of github.com:prryplatypus/sa…
prryplatypus Oct 11, 2021
0ab7f5f
Attempt #2 at tests
prryplatypus Oct 11, 2021
cb791aa
Fix error message
prryplatypus Oct 11, 2021
eb2a264
Remove failed tests
prryplatypus Oct 11, 2021
9a9a1ea
Reattempt testing number 1
prryplatypus Oct 18, 2021
2e492f9
Create `UVLOOP_INSTALLED` constant
prryplatypus Oct 23, 2021
48b0158
Merge changes into tests branch
prryplatypus Oct 23, 2021
2ff45cc
Fix typing
prryplatypus Oct 23, 2021
3e14dee
Merge branch 'feat/optional-uvloop-use' of github.com:prryplatypus/sa…
prryplatypus Oct 23, 2021
1908eb5
Fix linting... *sigh*
prryplatypus Oct 23, 2021
250c5cf
Merge branch 'feat/optional-uvloop-use' of github.com:prryplatypus/sa…
prryplatypus Oct 23, 2021
6563ee2
Fix tests (hopefully?)
prryplatypus Oct 23, 2021
a1f3660
Fix logging
prryplatypus Oct 23, 2021
0d1e591
Fix logo tests?
prryplatypus Oct 23, 2021
c008c34
Rename loop param
prryplatypus Oct 24, 2021
6351753
Fixes for gunicorn
prryplatypus Oct 24, 2021
41454b2
Fix linting errors
prryplatypus Oct 24, 2021
76492a7
Avoid unnecessary error log
prryplatypus Oct 24, 2021
348c358
Merge branch 'main' of github.com:sanic-org/sanic into feat/optional-…
prryplatypus Oct 24, 2021
471a42b
Merge branch 'main' into feat/optional-uvloop-use
prryplatypus Oct 31, 2021
0dbf1b2
Added first tests
prryplatypus Nov 1, 2021
9de0e34
Split create_server tests so loop isn't closed prematurely
prryplatypus Nov 1, 2021
e6a195b
Move loop setup to after logo
prryplatypus Nov 1, 2021
5ca22d7
Fix some tests
prryplatypus Nov 1, 2021
934d737
Move loop setup to after startup debug messages
prryplatypus Nov 2, 2021
a7a8624
Merge branch 'main' into feat/optional-uvloop-use
prryplatypus Nov 7, 2021
355a9e2
Fix broken test from merge
prryplatypus Nov 7, 2021
12ecf52
Merge branch 'main' of github.com:sanic-org/sanic into feat/optional-…
prryplatypus Nov 18, 2021
2a35578
Fixed some test errors
prryplatypus Nov 18, 2021
796892d
Merge branch 'main' of github.com:sanic-org/sanic into feat/optional-…
prryplatypus Dec 6, 2021
877f5e6
Only change loop policy in non-async methods
prryplatypus Dec 6, 2021
62b9d61
Move uvloop setup method to its own file
prryplatypus Dec 7, 2021
b1b9db3
Distinguish between explicit and default config
prryplatypus Dec 7, 2021
f4acfd1
Fix linting
prryplatypus Dec 7, 2021
97bb0cc
Merge branch 'main' of github.com:prryplatypus/sanic into feat/option…
prryplatypus Dec 7, 2021
710896b
Fix tests
prryplatypus Dec 7, 2021
72152ce
Added some tests
prryplatypus Dec 7, 2021
a3906fc
Use `getenv` for easier mocking
prryplatypus Dec 8, 2021
3c58c09
Added missing tests
prryplatypus Dec 8, 2021
d4d6c61
Fix assertions
prryplatypus Dec 8, 2021
08e1ce5
Fix tests
prryplatypus Dec 8, 2021
d694525
Fix tests for real this time
prryplatypus Dec 8, 2021
a7d5207
Fix tests for python 3.7
prryplatypus Dec 8, 2021
868a2da
Add warning when configuring loop with `create_server`
prryplatypus Dec 8, 2021
155f5fc
Merge branch 'main' into feat/optional-uvloop-use
prryplatypus Dec 8, 2021
8a79dc0
Merge branch 'main' into feat/optional-uvloop-use
prryplatypus Dec 9, 2021
c2e28a8
Merge branch 'main' of https://github.com/sanic-org/sanic into feat/o…
prryplatypus Dec 17, 2021
19069d6
Ensure consistent uvloop config across apps
prryplatypus Dec 18, 2021
2a85824
Add warning when changing loop in ASGI mode
prryplatypus Dec 18, 2021
e608ae7
Reorder imports
prryplatypus Dec 18, 2021
aa1639e
Reword message
prryplatypus Dec 18, 2021
fbe0b71
Fix the other half of the test
prryplatypus Dec 18, 2021
51c7278
Added app logging tests and fixed classvar not being set
prryplatypus Dec 18, 2021
3d1e98f
Merge branch 'main' of github.com:sanic-org/sanic into feat/optional-…
prryplatypus Dec 18, 2021
578f354
Add test for ASGI
ahopkins Dec 19, 2021
373ab7f
Remove registry deprecation
prryplatypus Dec 20, 2021
b290d0f
Update TODO version in comments
prryplatypus Dec 20, 2021
833f1a5
Merge branch 'main' into feat/optional-uvloop-use
ahopkins Dec 23, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
28 changes: 21 additions & 7 deletions sanic/app.py
Expand Up @@ -67,6 +67,7 @@
URLBuildError,
)
from sanic.handlers import ErrorHandler
from sanic.helpers import _default
from sanic.log import LOGGING_CONFIG_DEFAULTS, Colors, error_logger, logger
from sanic.mixins.listeners import ListenerEvent
from sanic.models.futures import (
Expand All @@ -85,7 +86,7 @@
from sanic.router import Router
from sanic.server import AsyncioServer, HttpProtocol
from sanic.server import Signal as ServerSignal
from sanic.server import serve, serve_multiple, serve_single
from sanic.server import serve, serve_multiple, serve_single, try_use_uvloop
from sanic.server.protocols.websocket_protocol import WebSocketProtocol
from sanic.server.websockets.impl import ConnectionClosed
from sanic.signals import Signal, SignalRouter
Expand Down Expand Up @@ -1103,6 +1104,11 @@ def run(
register_sys_signals=register_sys_signals,
)

if self.config.USE_UVLOOP is True or (
self.config.USE_UVLOOP is _default and not OS_IS_WINDOWS
):
try_use_uvloop()

try:
self.is_running = True
self.is_stopping = False
Expand Down Expand Up @@ -1199,12 +1205,13 @@ async def create_server(
WebSocketProtocol if self.websocket_enabled else HttpProtocol
)

# if access_log is passed explicitly change config.ACCESS_LOG
if access_log is not None:
self.config.ACCESS_LOG = access_log

if noisy_exceptions is not None:
self.config.NOISY_EXCEPTIONS = noisy_exceptions
# Set explicitly passed configuration values
for attribute, value in {
"ACCESS_LOG": access_log,
"NOISY_EXCEPTIONS": noisy_exceptions,
}.items():
if value is not None:
setattr(self.config, attribute, value)

server_settings = self._helper(
host=host,
Expand All @@ -1219,6 +1226,13 @@ async def create_server(
run_async=return_asyncio_server,
)

if self.config.USE_UVLOOP is not _default:
error_logger.warning(
"You are trying to configure uvloop, but this is only "
"supported when using the run(...) method. Sanic will now "
"continue to run using the existing event loop."
)
prryplatypus marked this conversation as resolved.
Show resolved Hide resolved

main_start = server_settings.pop("main_start", None)
main_stop = server_settings.pop("main_stop", None)
if main_start or main_stop:
Expand Down
8 changes: 8 additions & 0 deletions sanic/compat.py
Expand Up @@ -8,6 +8,14 @@


OS_IS_WINDOWS = os.name == "nt"
UVLOOP_INSTALLED = False

try:
import uvloop # type: ignore # noqa

UVLOOP_INSTALLED = True
except ImportError:
pass


def enable_windows_color_support():
Expand Down
3 changes: 3 additions & 0 deletions sanic/config.py
Expand Up @@ -7,6 +7,7 @@
from warnings import warn

from sanic.errorpages import check_error_format
from sanic.helpers import Default, _default
from sanic.http import Http
from sanic.utils import load_module_from_file_location, str_to_bool

Expand Down Expand Up @@ -40,6 +41,7 @@
"REQUEST_MAX_SIZE": 100000000, # 100 megabytes
"REQUEST_TIMEOUT": 60, # 60 seconds
"RESPONSE_TIMEOUT": 60, # 60 seconds
"USE_UVLOOP": _default,
"WEBSOCKET_MAX_SIZE": 2 ** 20, # 1 megabyte
"WEBSOCKET_PING_INTERVAL": 20,
"WEBSOCKET_PING_TIMEOUT": 20,
Expand Down Expand Up @@ -69,6 +71,7 @@ class Config(dict):
REQUEST_TIMEOUT: int
RESPONSE_TIMEOUT: int
SERVER_NAME: str
USE_UVLOOP: Union[Default, bool]
WEBSOCKET_MAX_SIZE: int
WEBSOCKET_PING_INTERVAL: int
WEBSOCKET_PING_TIMEOUT: int
Expand Down
13 changes: 2 additions & 11 deletions sanic/server/__init__.py
@@ -1,20 +1,10 @@
import asyncio

from sanic.models.server_types import ConnInfo, Signal
from sanic.server.async_server import AsyncioServer
from sanic.server.loop import try_use_uvloop
from sanic.server.protocols.http_protocol import HttpProtocol
from sanic.server.runners import serve, serve_multiple, serve_single


try:
import uvloop # type: ignore

if not isinstance(asyncio.get_event_loop_policy(), uvloop.EventLoopPolicy):
asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
except ImportError:
pass


__all__ = (
"AsyncioServer",
"ConnInfo",
Expand All @@ -23,4 +13,5 @@
"serve",
"serve_multiple",
"serve_single",
"try_use_uvloop",
)
49 changes: 49 additions & 0 deletions sanic/server/loop.py
@@ -0,0 +1,49 @@
import asyncio

from distutils.util import strtobool
from os import getenv

from sanic.compat import OS_IS_WINDOWS
from sanic.log import error_logger


def try_use_uvloop() -> None:
"""
Use uvloop instead of the default asyncio loop.
"""
if OS_IS_WINDOWS:
error_logger.warning(
"You are trying to use uvloop, but uvloop is not compatible "
"with your system. You can disable uvloop completely by setting "
"the 'USE_UVLOOP' configuration value to false, or simply not "
"defining it and letting Sanic handle it for you. Sanic will now "
"continue to run using the default event loop."
)
return

try:
import uvloop # type: ignore
except ImportError:
error_logger.warning(
"You are trying to use uvloop, but uvloop is not "
"installed in your system. In order to use uvloop "
"you must first install it. Otherwise, you can disable "
"uvloop completely by setting the 'USE_UVLOOP' "
"configuration value to false. Sanic will now continue "
"to run with the default event loop."
)
return

uvloop_install_removed = strtobool(getenv("SANIC_NO_UVLOOP", "no"))
if uvloop_install_removed:
error_logger.info(
"You are requesting to run Sanic using uvloop, but the "
"install-time 'SANIC_NO_UVLOOP' environment variable (used to "
"opt-out of installing uvloop with Sanic) is set to true. If "
"you want to prevent Sanic from overriding the event loop policy "
"during runtime, set the 'USE_UVLOOP' configuration value to "
"false."
)

if not isinstance(asyncio.get_event_loop_policy(), uvloop.EventLoopPolicy):
asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
11 changes: 4 additions & 7 deletions sanic/worker.py
Expand Up @@ -7,8 +7,9 @@

from gunicorn.workers import base # type: ignore

from sanic.compat import UVLOOP_INSTALLED
from sanic.log import logger
from sanic.server import HttpProtocol, Signal, serve
from sanic.server import HttpProtocol, Signal, serve, try_use_uvloop
from sanic.server.protocols.websocket_protocol import WebSocketProtocol


Expand All @@ -17,12 +18,8 @@
except ImportError:
ssl = None # type: ignore

try:
import uvloop # type: ignore

asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
except ImportError:
pass
if UVLOOP_INSTALLED:
try_use_uvloop()


class GunicornWorker(base.Worker):
Expand Down
103 changes: 91 additions & 12 deletions tests/test_app.py
Expand Up @@ -9,8 +9,10 @@

import py
import pytest
import sanic

from sanic import Sanic
from sanic.compat import OS_IS_WINDOWS, UVLOOP_INSTALLED
from sanic.config import Config
from sanic.exceptions import SanicException
from sanic.response import text
Expand All @@ -21,15 +23,6 @@ def clear_app_registry():
Sanic._app_registry = {}


def uvloop_installed():
try:
import uvloop # noqa

return True
except ImportError:
return False


def test_app_loop_running(app):
@app.get("/test")
async def handler(request):
Expand All @@ -41,7 +34,7 @@ async def handler(request):


def test_create_asyncio_server(app):
if not uvloop_installed():
if not UVLOOP_INSTALLED:
loop = asyncio.get_event_loop()
asyncio_srv_coro = app.create_server(return_asyncio_server=True)
assert isawaitable(asyncio_srv_coro)
Expand All @@ -50,7 +43,7 @@ def test_create_asyncio_server(app):


def test_asyncio_server_no_start_serving(app):
if not uvloop_installed():
if not UVLOOP_INSTALLED:
loop = asyncio.get_event_loop()
asyncio_srv_coro = app.create_server(
port=43123,
Expand All @@ -62,7 +55,7 @@ def test_asyncio_server_no_start_serving(app):


def test_asyncio_server_start_serving(app):
if not uvloop_installed():
if not UVLOOP_INSTALLED:
loop = asyncio.get_event_loop()
asyncio_srv_coro = app.create_server(
port=43124,
Expand Down Expand Up @@ -448,6 +441,92 @@ class CustomContext:
assert app.ctx == ctx


def test_uvloop_config(app, monkeypatch):
@app.get("/test")
def handler(request):
return text("ok")

try_use_uvloop = Mock()
monkeypatch.setattr(sanic.app, "try_use_uvloop", try_use_uvloop)

# Default config
app.test_client.get("/test")
if OS_IS_WINDOWS:
try_use_uvloop.assert_not_called()
else:
try_use_uvloop.assert_called_once()

try_use_uvloop.reset_mock()
app.config["USE_UVLOOP"] = False
app.test_client.get("/test")
try_use_uvloop.assert_not_called()

try_use_uvloop.reset_mock()
app.config["USE_UVLOOP"] = True
app.test_client.get("/test")
try_use_uvloop.assert_called_once()


def test_uvloop_is_never_called_with_create_server(app, caplog, monkeypatch):
@app.get("/test")
def handler(request):
return text("ok")

try_use_uvloop = Mock()
monkeypatch.setattr(sanic.app, "try_use_uvloop", try_use_uvloop)

loop = asyncio.get_event_loop()

# Default config
asyncio_srv_coro = app.create_server(
return_asyncio_server=True,
asyncio_server_kwargs=dict(start_serving=False)
)
loop.run_until_complete(asyncio_srv_coro)
try_use_uvloop.assert_not_called()

app.config["USE_UVLOOP"] = False
asyncio_srv_coro = app.create_server(
return_asyncio_server=True,
asyncio_server_kwargs=dict(start_serving=False)
)

with caplog.at_level(logging.WARNING):
loop.run_until_complete(asyncio_srv_coro)

for record in caplog.records:
if record.message.startswith("You are trying to configure"):
break

assert record.message == (
"You are trying to configure uvloop, but this is only "
"supported when using the run(...) method. Sanic will now "
"continue to run using the existing event loop."
)

try_use_uvloop.assert_not_called()

app.config["USE_UVLOOP"] = True
asyncio_srv_coro = app.create_server(
return_asyncio_server=True,
asyncio_server_kwargs=dict(start_serving=False)
)

with caplog.at_level(logging.WARNING):
loop.run_until_complete(asyncio_srv_coro)
try_use_uvloop.assert_not_called()

for record in caplog.records:
if record.message.startswith("You are trying to configure"):
break

assert record.message == (
"You are trying to configure uvloop, but this is only "
"supported when using the run(...) method. Sanic will now "
"continue to run using the existing event loop."
)


def test_cannot_run_fast_and_workers(app):
message = "You cannot use both fast=True and workers=X"
with pytest.raises(RuntimeError, match=message):
Expand Down
10 changes: 7 additions & 3 deletions tests/test_exceptions.py
Expand Up @@ -271,9 +271,13 @@ async def feed(request, ws):
with caplog.at_level(logging.INFO):
app.test_client.websocket("/feed")

error_logs = [r for r in caplog.record_tuples if r[0] == "sanic.error"]
assert error_logs[1][1] == logging.ERROR
assert "Exception occurred while handling uri:" in error_logs[1][2]
for record in caplog.record_tuples:
if record[2].startswith("Exception occurred"):
break

assert record[0] == "sanic.error"
assert record[1] == logging.ERROR
assert "Exception occurred while handling uri:" in record[2]


@pytest.mark.parametrize("debug", (True, False))
Expand Down