Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CLI for deploying built-in models #694

Open
maxjakob opened this issue May 14, 2024 · 3 comments
Open

Add CLI for deploying built-in models #694

maxjakob opened this issue May 14, 2024 · 3 comments
Labels
enhancement New feature or request

Comments

@maxjakob
Copy link

In the context of integration tests and generally in cluster setup automation, the eland_import_hub_model CLI is a great tool to get HuggingFace models deployed in Elasticsearch. There is no a comparable tool for built-in models (most notably ELSER and E5). Users have to do this themselves using with 4 different API calls and a wait condition (example). Adding a CLI tool eland_deploy_builtin_model that works in a similar way (e.g. --model-id .elser_model_v2 --url http://elasticsearch:9200 --start) would reduce the room for error, reduce friction when using built-in models and in turn increase their adoption.

@maxjakob maxjakob added the enhancement New feature or request label May 14, 2024
@joshdevins
Copy link
Member

Should this be in eland or in the main client libs? It's just some API calls.

@pquentin
Copy link
Member

pquentin commented May 15, 2024

The API calls should definitely be in the vectorstore helpers in the client. That will directly improve snippets like https://www.elastic.co/search-labs/tutorials/search-tutorial/semantic-search/elser-model#deploying-the-elser-model which can easily timeout and leave you in a bad state that is difficult to recover from (eg. model deployed but no ingest pipeline).

But it seems to me that there's also value in exposing this through Eland? It would be nice to have a single command to upload both E5 and ELSER in my opinion.

@maxjakob
Copy link
Author

Implementing this in the client's helpers module works. Then we also have a simple programmatic way to do this in Python.

Additionally, I think it's also important to have a CLI for this. This could of course also be through the client package. But since users are already used to deploying HF models with Eland I think it's a good fit to add more model deployment tooling here (also).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants