feat(server): add runner metrics; refactoring batch size metrics #2977
Conversation
Codecov Report
@@ Coverage Diff @@
## main #2977 +/- ##
=======================================
Coverage ? 70.61%
=======================================
Files ? 104
Lines ? 9570
Branches ? 0
=======================================
Hits ? 6758
Misses ? 2812
Partials ? 0 |
|
@bojiang can you also add a check for the prefix in our tracking code here? |
|
@aarnphm There's no breaking change in this PR. Those metrics are still there. |
| START_TIME_VAR.set(default_timer()) | ||
|
|
||
| async def wrapped_send(message: "ext.ASGIMessage") -> None: | ||
| if message["type"] == "http.response.start": |
There was a problem hiding this comment.
Should we register time after the last byte of body is sent like in access.py?
There was a problem hiding this comment.
never thought about that
| "--prometheus-dir", | ||
| prometheus_dir, |
There was a problem hiding this comment.
Is this needed to make runner metrics work?
There was a problem hiding this comment.
yes. Runners might be multi-processing, too
| @@ -96,25 +134,158 @@ async def wrapped_send(message: "ext.ASGIMessage") -> None: | |||
| status_code = message["status"] | |||
There was a problem hiding this comment.
Let's change the behavior for api_server, too.
bojiang
force-pushed
the
runner-metrics
branch
2 times, most recently
from
ff75194
to
cbde20b
Compare
bojiang
force-pushed
the
runner-metrics
branch
from
cbde20b
to
b4ab719
Compare
What does this PR address?
Fixes https://github.com/bentoml/Kitchen/issues/40
Before submitting:
guide on how to create a pull request.
make formatandmake lintscript have passed (instructions)?those accordingly? Here are documentation guidelines and tips on writting docs.
Who can help review?
Feel free to tag members/contributors who can help review your PR.