A fast, easy-to-use, production-ready inference server for computer vision supporting deployment of many popular model architectures and fine-tuned models.
-
Updated
May 31, 2024 - Python
A fast, easy-to-use, production-ready inference server for computer vision supporting deployment of many popular model architectures and fine-tuned models.
OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
🤗 Optimum Intel: Accelerate inference with Intel optimization tools
TensorRT C++ API Tutorial
Making large AI models cheaper, faster and more accessible
High-efficiency floating-point neural network inference operators for mobile, server, and Web
ncnn is a high-performance neural network inference framework optimized for the mobile platform
📚 Jupyter notebook tutorials for OpenVINO™
A scalable inference server for models optimized with OpenVINO™
Utilities to use the Hugging Face Hub API
Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.
A high-throughput and memory-efficient inference and serving engine for LLMs
Python library for YOLOv8 and YOLOv9 small object detection and instance segmentation
NVIDIA-accelerated, deep learned semantic image segmentation
NVIDIA-accelerated, deep learned model support for image space object detection
Deep learned, NVIDIA-accelerated 3D object pose estimation
Large Language Model Text Generation Inference
TypeDB: the polymorphic database powered by types
Add a description, image, and links to the inference topic page so that developers can more easily learn about it.
To associate your repository with the inference topic, visit your repo's landing page and select "manage topics."