An experimental plugin that integrates Ray Serve with the MLflow pipeline.
mlflow-ray-serve
enables MLflow users to deploy MLflow models at scale on Ray Serve.
This plugin implements the Python API and command-line interface for MLflow deployment plugins.
pip install mlflow-ray-serve
The following packages are required and will be installed along with the plugin:
"ray[serve]"
"mlflow>=1.12.0"
This plugin requires Ray version 1.7.0 or greater.
This plugin must be used with a detached Ray Serve instance running on a Ray cluster. An easy way to set this up is by running the following two commands:
ray start --head # Start a single-node Ray cluster locally.
serve start # Start a detached Ray Serve instance.
The API is summarized below. For full details see the MLflow deployment plugin Python API and command-line interface documentation.
See https://github.com/mlflow/mlflow/tree/master/examples/ray_serve for a full example.
Deploy a model built with MLflow using Ray Serve with the desired configuration parameters; for example, num_replicas
. Currently this plugin only supports the python_function
flavor of MLflow models, and this is the default flavor.
mlflow deployments create -t ray-serve -m <model uri> --name <deployment name> -C num_replicas=<number of replicas>
from mlflow.deployments import get_deploy_client
target_uri = 'ray-serve'
plugin = get_deploy_client(target_uri)
plugin.create_deployment(
name=<deployment name>,
model_uri=<model uri>,
config={"num_replicas": 4})
Modify the configuration of a deployed model and/or replace the deployment with a new model URI.
mlflow deployments update -t ray-serve --name <deployment name> -C num_replicas=<new number of replicas>
plugin.update_deployment(name=<deployment name>, config={"num_replicas": <new number of replicas>})
Delete an existing deployment.
mlflow deployments delete -t ray-serve --name <deployment name>
plugin.delete_deployment(name=<deployment name>)
List the names of all the models deployed on Ray Serve. Includes models not deployed via this plugin.
mlflow deployments list -t ray-serve
plugin.list_deployments()
mlflow deployments get -t ray-serve --name <deployment name>
plugin.get_deployment(name=<deployment name>)
For the prediction inputs, DataFrame, Tensor and JSON formats are supported by the Python API. To invoke via the command line, pass in the path to a JSON file containing the input.
mlflow deployments predict -t ray-serve --name <deployment name> --input-path <input file path> --output-path <output file path>
output-path
is an optional parameter. Without it, the result will be printed in the terminal.
plugin.predict(name=<deployment name>, df=<prediction input>)
Prints the plugin help string.
mlflow deployments help -t ray-serve