Using other executors

[BusyJay]

Is your feature request related to a problem? Please describe.
There are several executors in Rust world, like tokio, async-std, smol. TiKV project has developed its owned thread pool yatp. And grpcio has its own executor too, it's implemented on top of the poll engine build-in grpc c core. Due to grpc c core bind each call to the same thread (grpc/grpc#15535, for better locality), it's not recommended to do time consuming job in the executor. So people usually use one of the other executors mentioned above to get around the issue. For example, TiKV uses yatp to handle heavy computation or file IO.

But using different executors are not a good idea in some situations. For example, in resource limited environment, too many threads can have huge performance regression due to context switches. Using different executors means more threads usually. For style consideration, it's also cleaner to just keep using one executor.

grpc c core supports custom poll engine by using grpc_register_event_engine_factory. A custom poll engine can adapt other executors to fit in event_engine_factory_fn. For example, a tokio Reactor exposes similar APIs with event_engine_factory_fn. One downside is that grpc_register_event_engine_factory is a private interface, so it can't work when using dynamic link.

[BusyJay]

Related design: https://github.com/grpc/proposal/pull/245/files