Skip to content

Latest commit

 

History

History
23 lines (14 loc) · 580 Bytes

README.md

File metadata and controls

23 lines (14 loc) · 580 Bytes
Raw

Text Generation Inference

Service

The following command deploys Llama 7B Instruct as a service:

dstack run . -f examples/deployment/tgi/serve.dstack.yml

See the configuration at serve.dstack.yml.

Task

The following command runs Llama 7B Instruct as a task:

dstack run . -f examples/deployment/tgi/serve-task.dstack.yml

See the configuration at serve.dstack.yml.

For more details, refer to services or tasks.