Is the gRPC in BentoML stable and actively maintained?

[ryan-3m]

As far as I know, gRPC is a new feature added after version 1.x of BentoML. Is it still considered experimental?

I have some high-traffic tasks, such as processing videos, for which REST would incur too much overhead. Therefore, I have to choose gRPC.

[parano]

Hi @ryan-3m, yes the gRPC support is actively being developed and maintained, however still considered a beta feature. It would be great to hear your feedback on this.

[MrBeeMovie]

Hi @ryan-3m, yes the gRPC support is actively being developed and maintained, however still considered a beta feature. It would be great to hear your feedback on this.

I have been trying to run the sentence-transformers example repo you guys have but serve-grpc always fails with:

bentoml serve-grpc
2024-03-19T12:23:19-0400 [WARNING] [cli] Deprecated build option: 'docker.env' is used, please use 'envs' instead.
Error: [serve] `serve-grpc` failed: <class '_bentoml_sdk.service.factory.Service'> type doesn't support gRPC serving

I have made sure to install bentoml[grpc]. Any ideas?

[EraYaN]

According to a conversation in slack, the new 1.2 API is not yet supported by gRPC. So you would need to use the one 1.1 style bento service for gRPC to work (not a class but functions).