Skip to content

Latest commit

 

History

History
17 lines (9 loc) · 1.26 KB

README.md

File metadata and controls

17 lines (9 loc) · 1.26 KB

Build

ModelMesh

The ModelMesh framework is a mature, general-purpose model serving management/routing layer designed for high-scale, high-density and frequently-changing model use cases. It works with existing or custom-built model servers and acts as a distributed LRU cache for serving runtime models.

For full Kubernetes-based deployment and management of ModelMesh clusters and models, see the ModelMesh Serving repo. This includes a separate controller and provides K8s custom resource based management of ServingRuntimes and InferenceServices along with common, abstracted handling of model repository storage and ready-to-use integrations with some existing OSS model servers.

For more information on supported features and design details, see these charts.

Get Started

To learn more about and get started with the ModelMesh framework, check out the documentation.

Developer guide

Use the developer guide to learn about development practices for the project.