New multiprocess manager #2183

abersheeran · 2023-12-15T03:17:25Z

Summary

The multiprocess manager introduced by this PR includes process keep-alive and process hung detection. It also imitates gunicorn and uses hup, ttin, and ttou signals to control child processes.

Checklist

I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)
I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
I've updated the documentation accordingly.

uvicorn/supervisors/multiprocess.py

Kludex · 2024-01-20T09:45:56Z

pyproject.toml

@@ -100,7 +100,7 @@ omit = [

 [tool.coverage.report]
 precision = 2
-fail_under = 98.35
+fail_under = 96.82


I'm not willing to introduce this feature without testing it.

Yes, I just want to see your attitude towards this PR before adding the test. Because I remember you have implemented something similar before.

Ah, then let me review and get back to you.

Kludex

Checklist

Increase coverage for the new multiprocess manager to 100% (if tests are slow, run on a separate job in the CI, and only if the multiprocess.py is changed).
Docstrings.
Documentation (I can help here).
Answer the question: how this should interact with --reload?
Move UvicornWorker out from Uvicorn's source code (I'll do it).

uvicorn/supervisors/multiprocess.py

Kludex · 2024-03-02T13:34:27Z

@abersheeran I'm fully available to move this forward. 👍

Kludex · 2024-03-03T08:12:19Z

Pinging @ahopkins to review the initial steps since he offered helped on #2164. 🙏

…process-manager

Kludex · 2024-03-04T08:37:13Z

Ok. 👍

We can move forward with the checklist above 🙏

uvicorn/supervisors/multiprocess.py

ahopkins · 2024-03-04T09:10:43Z

uvicorn/supervisors/multiprocess.py

+            process.terminate()
+            process.join()
+            del self.processes[idx]
+            process = Process(self.config, self.target, self.sockets)
+            process.start()


Are you confirmed this is the behavior that you want? Shutdown first, then start the new processes with a potential gap in coverage? This was the initial pattern we went for with Sanic, but we received requests and ran into use cases where it was not desired and therefore offered it as configurable: https://github.com/sanic-org/sanic/blob/main/sanic/worker/process.py#L109

What problems might this gap cause?

ahopkins · 2024-03-04T09:19:25Z

uvicorn/supervisors/multiprocess.py

+        process = Process(self.config, self.target, self.sockets)
+        process.start()
+        self.processes.append(process)


This pattern of Process instantiation, start, and append is scattered in several locations. This will IMO become an issue over time as features are built on top of this. I'd suggest centralizing that somehow in a nicer pattern that allows more flexibility.

Excessive abstraction can cause more problems. I don't think a unified approach is needed until many of the new features you talk about are added. At least in my opinion, multi-process managers only have these functions at most (see gunicorn)

ahopkins

I see what the PR is doing, but I somewhat fail to understand the why and the goals. Is the pinging the ultimate feature add here?

Kludex

I think we need to change the logs to print the PID, otherwise it gets confusing:

❯ uvicorn main:app --workers 2
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started parent process [9954]
INFO:     Started child process [9956]
INFO:     Started child process [9957]
INFO:     Started server process [9956]
INFO:     Started server process [9957]
INFO:     Waiting for application startup.
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Application startup complete.

docs/deployment.md

uvicorn/supervisors/multiprocess.py

Kludex · 2024-04-13T10:10:02Z

pyproject.toml

@@ -101,7 +101,7 @@ omit = [

 [tool.coverage.report]
 precision = 2
-fail_under = 98.35
+fail_under = 98.13


I don't want to decrease this, can we add pragmas where needed?

Kludex · 2024-04-13T10:12:04Z

Also, if I press CTRL + C on uvicorn main:app --workers 2, it doesn't work as expected.

Co-authored-by: Marcelo Trylesinski <marcelotryle@gmail.com>

abersheeran · 2024-04-14T08:17:18Z

Also, if I press CTRL + C on uvicorn main:app --workers 2, it doesn't work as expected.

Can you try the latest commit version? This should solve the problem.

abersheeran added 10 commits December 15, 2023 11:17

New multiprocess manager

bb4efdd

lint it

fde0284

Fixed test

efd22a3

Fixed Process.__init__

c602782

Fix signal handling in Multiprocess class

c41e48d

Update coverage fail_under value

cf71b61

Remove redundant log message

3928b3d

Update coverage fail_under value

31c300b

Update coverage fail_under value

fb13e27

Update fail_under value in coverage report

993e695

abersheeran requested a review from Kludex December 15, 2023 04:52

Remove unused threading event

29b3ad6

rbtz-openai mentioned this pull request Dec 22, 2023

Improve shutdown process in Multiprocess mode #2010

Closed

3 tasks

baseplate-admin reviewed Jan 13, 2024

View reviewed changes

uvicorn/supervisors/multiprocess.py Outdated Show resolved Hide resolved

Kludex requested changes Jan 20, 2024

View reviewed changes

Kludex added the waiting author Waiting for author's reply label Jan 20, 2024

Kludex requested changes Mar 2, 2024

View reviewed changes

uvicorn/supervisors/multiprocess.py Outdated Show resolved Hide resolved

uvicorn/supervisors/multiprocess.py Outdated Show resolved Hide resolved

uvicorn/supervisors/multiprocess.py Outdated Show resolved Hide resolved

uvicorn/supervisors/multiprocess.py Show resolved Hide resolved

Kludex mentioned this pull request Mar 2, 2024

Simplified worker monitoring and restart, without gunicorn. #1942

Closed

Kludex requested a review from ahopkins March 3, 2024 08:11

abersheeran added 2 commits March 4, 2024 09:42

Merge branch 'master' of https://github.com/encode/uvicorn into multi…

637a372

…process-manager

lint

f767d9c

ahopkins reviewed Mar 4, 2024

View reviewed changes

uvicorn/supervisors/multiprocess.py Show resolved Hide resolved

ahopkins reviewed Mar 4, 2024

View reviewed changes

ahopkins approved these changes Mar 4, 2024

View reviewed changes

ahopkins self-requested a review March 4, 2024 09:21

Kludex added the hold Don't merge yet label Mar 4, 2024

abersheeran added 11 commits April 2, 2024 21:18

Merge branch 'master' into multiprocess-manager

9397e8e

Tests in windows

8efcad3

lint

7c683ca

lint

4e3ac6e

ignore mypy check in linux

404b864

Add __init__.py

d3a3aab

fix warning

736510a

coverage ignore

42c7187

Update coverage

6fce985

coverage

6e916f3

Add documents

b8ee655

abersheeran requested a review from Kludex April 5, 2024 14:56

abersheeran removed waiting author Waiting for author's reply hold Don't merge yet labels Apr 5, 2024

Kludex requested changes Apr 13, 2024

View reviewed changes

Kludex added the waiting author Waiting for author's reply label Apr 13, 2024

abersheeran and others added 9 commits April 14, 2024 16:14

Update docs/deployment.md

66395d0

Co-authored-by: Marcelo Trylesinski <marcelotryle@gmail.com>

Update uvicorn/supervisors/multiprocess.py

a19f737

Co-authored-by: Marcelo Trylesinski <marcelotryle@gmail.com>

Update uvicorn/supervisors/multiprocess.py

658a4fa

Co-authored-by: Marcelo Trylesinski <marcelotryle@gmail.com>

Update uvicorn/supervisors/multiprocess.py

43e0190

Co-authored-by: Marcelo Trylesinski <marcelotryle@gmail.com>

Update uvicorn/supervisors/multiprocess.py

cc25743

Co-authored-by: Marcelo Trylesinski <marcelotryle@gmail.com>

Update uvicorn/supervisors/multiprocess.py

5f23d2f

Co-authored-by: Marcelo Trylesinski <marcelotryle@gmail.com>

Update uvicorn/supervisors/multiprocess.py

a5d19cb

Co-authored-by: Marcelo Trylesinski <marcelotryle@gmail.com>

Update uvicorn/supervisors/multiprocess.py

1b3ddc7

Co-authored-by: Marcelo Trylesinski <marcelotryle@gmail.com>

Update uvicorn/supervisors/multiprocess.py

1e8db13

Co-authored-by: Marcelo Trylesinski <marcelotryle@gmail.com>

abersheeran added 2 commits April 14, 2024 16:21

Do not output the PID information repeatedly.

a4209cf

Fix occasional abnormal exits.

dbc34c8

baseplate-admin mentioned this pull request May 6, 2024

Replace % string formatting with f-string #2322

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New multiprocess manager #2183

New multiprocess manager #2183

abersheeran commented Dec 15, 2023 •

edited

Kludex Jan 20, 2024

abersheeran Jan 22, 2024

Kludex Jan 22, 2024

Kludex left a comment •

edited

Kludex commented Mar 2, 2024

Kludex commented Mar 3, 2024 •

edited

Kludex commented Mar 4, 2024

ahopkins Mar 4, 2024

abersheeran Mar 5, 2024

ahopkins Mar 4, 2024

abersheeran Mar 5, 2024

ahopkins left a comment

Kludex left a comment

Kludex Apr 13, 2024

Kludex commented Apr 13, 2024

abersheeran commented Apr 14, 2024 •

edited

New multiprocess manager #2183

Are you sure you want to change the base?

New multiprocess manager #2183

Conversation

abersheeran commented Dec 15, 2023 • edited

Summary

Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Kludex left a comment • edited

Choose a reason for hiding this comment

Checklist

Kludex commented Mar 2, 2024

Kludex commented Mar 3, 2024 • edited

Kludex commented Mar 4, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ahopkins left a comment

Choose a reason for hiding this comment

Kludex left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Kludex commented Apr 13, 2024

abersheeran commented Apr 14, 2024 • edited

abersheeran commented Dec 15, 2023 •

edited

Kludex left a comment •

edited

Kludex commented Mar 3, 2024 •

edited

abersheeran commented Apr 14, 2024 •

edited