Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhancement: Dynamically add endpoints to Service #4460

Open
holzweber opened this issue Jan 30, 2024 · 0 comments · May be fixed by #4498
Open

enhancement: Dynamically add endpoints to Service #4460

holzweber opened this issue Jan 30, 2024 · 0 comments · May be fixed by #4498

Comments

@holzweber
Copy link
Contributor

Feature request

Depending on available models, support the possibility to add endpoints to a service.

Main Idea (only a draft, does not work like this in python)

import numpy as np
import bentoml
from bentoml.io import NumpyNdarray

# assuming modeltags holds all available models on device
runner_list = []
for model_tag in modeltags:
    runner_list.add[bentoml.sklearn.get("model_tag").to_runner()]

svc = bentoml.Service("detection", runners=runner_list)


for idx in range(len(runner_list)):
	@svc.api(input=NumpyNdarray(), output=NumpyNdarray(), route=f"runner_{idx}")
	def classify(input_series: np.ndarray) -> np.ndarray:
		result = iris_clf_runner[i].predict.run(input_series)
		return result

Motivation

This feature would solve the problem of not having the possibility to dynamically add endpoints to a service

Other

No response

@holzweber holzweber changed the title feature: Dynamically add endpoints to Service enhancement: Dynamically add endpoints to Service Feb 10, 2024
@holzweber holzweber linked a pull request Feb 10, 2024 that will close this issue
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant