From 7a8ee5fa31cf929085da794a571c1439d9a867b2 Mon Sep 17 00:00:00 2001 From: Sean Sheng Date: Wed, 7 Dec 2022 00:58:18 -0800 Subject: [PATCH] docs: Update monitoring docs format (#3324) --- docs/source/guides/index.rst | 2 +- docs/source/guides/monitoring.rst | 22 +++++++++++++--------- docs/source/integrations/index.rst | 2 +- 3 files changed, 15 insertions(+), 11 deletions(-) diff --git a/docs/source/guides/index.rst b/docs/source/guides/index.rst index 66b09e24b51..76e48e4fb87 100644 --- a/docs/source/guides/index.rst +++ b/docs/source/guides/index.rst @@ -18,9 +18,9 @@ into this part of the documentation. server configuration graph + monitoring logging metrics - monitoring performance grpc gpu diff --git a/docs/source/guides/monitoring.rst b/docs/source/guides/monitoring.rst index f29b7ad849e..5c8f4634ae9 100644 --- a/docs/source/guides/monitoring.rst +++ b/docs/source/guides/monitoring.rst @@ -16,6 +16,7 @@ BentoML embraces this new paradigm by providing APIs that make a data-centric wo In this guide, we will focus on the online data collection and model monitoring. BentoML provides a unified interface for that. The benefits of having a data collection and model monitoring workflow includes: + - Monitor key statistical business metrics. - Identify early data drift events to determine whether retraining is required. - Enable QA for the previous untracked metrics, such as model performance, accuracy, degradation, etc. @@ -109,7 +110,7 @@ With a complete service definition, we can proceed to build the bento. Deploy the service and collect monitoring data ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -With BentoML, once we have the bento, it's easy to deploy the ML application to any target. https://docs.bentoml.org/en/latest/concepts/deploy.html +With BentoML, once we have the bento, it's easy to :ref:`deploy ` the ML application to any target. Use ``serve --production`` to start the bento in production mode as a standalone server: @@ -160,13 +161,14 @@ To achieve this, we just neet to provide a deployment configuration to bentoml. Built-in Monitoring Data Collectors ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -1. Through log files +Through log files +~~~~~~~~~~~~~~~~~ -The most common way to collect monitoring data is to write it to log files. Many utils like fluentbit, filebeat, logstash, etc. can be used to collect log files and ship them to a data warehouse or a monitoring system. +The most common way to collect monitoring data is to write it to log files. Many utils like `fluentbit `_, `filebeat `_, `logstash `_, etc. can be used to collect log files and ship them to a data warehouse or a monitoring system. This is also the default way BentoML exports monitoring data: .. code-block:: yaml - :caption: `deployment_configuration.yaml` + :caption: ⚙️ `configuration.yml` monitoring: enabled: true @@ -178,10 +180,11 @@ For Docker deployments, user can mount the log directory to a volume to persist For K8s deployments, user can mount the log directory, and deploy a fluentbit daemonset or sidecar container to collect the log files to target destinations. -2. Through a OTLP endpoint +Through a OTLP endpoint +~~~~~~~~~~~~~~~~~~~~~~~ .. code-block:: yaml - :caption: `deployment_configuration.yaml` + :caption: ⚙️ `configuration.yml` monitoring: enable: true @@ -211,9 +214,10 @@ plugins could be more platform specific. For example, it is required to add `bentoml-plugins-arize` to the `python:packages` to use the Arize plugin. See :ref:`the build command` for more details. -1. Arize AI +Arize AI +~~~~~~~~ -For end-to-end solutions for data/model monitoring, BentoML colaborates with Arize AI to provide a plugin for Arize. +For end-to-end solutions for data/model monitoring, BentoML colaborates with `Arize AI `_ to provide a plugin for Arize. If you don't want to deploy a pipeline by yourself but still need data and model monitoring for the bussiness, Arize AI is a good choice. Arize AI provides a unified platform for data scientists, data engineers, and ML engineers to monitor, analyze, and debug ML models in production. @@ -221,7 +225,7 @@ And the `bentoml-plugins-arize` makes it easy to work with BentoML. .. code-block:: yaml - :caption: `deployment_configuration.yaml` + :caption: ⚙️ `configuration.yml` monitoring: enable: true diff --git a/docs/source/integrations/index.rst b/docs/source/integrations/index.rst index 5a05e28ca4c..b0ce2de9c76 100644 --- a/docs/source/integrations/index.rst +++ b/docs/source/integrations/index.rst @@ -9,5 +9,5 @@ Integrations airflow flink - mlflow arize + mlflow