wasmCloud-wasi-nn provides wasmCloud with wasi-nn compatibility.
The implementation of the wasi-nn capability provider is very flexible, and currently two types of providers have been implemented
- Native OpenVINO: This implementation is basically the same as the wasmtime implementation of wasi-nn/openvino..
A simple demo can be seen here, and a more complete video
- Candle based implementation: Candle provides support for loading many types of models, e.g., LLMs, Text to image Models, Image to text Models, etc.
You can see a demo of Candle-based LLMs here
The code for providers hasn't been pushed yet, feel free to discuss with me in Slack or Issue!