Lightweight support for custom protobuf annotations on RPCs

[maja42]

🚀 Feature: Ligthweight custom annotations

I have a big need for allowing custom RPC-annotations and adding special logic in my gateway that reacts according to those annotations.

Use case 1:
The first use case are caching-annotations: I want to be able to specify caching-behaviour (no-cache/max-age/...) inside my protobuf, and have a special http middleware/interceptor in my gateway that can set HTTP caching-headers according to that definition. Ideally, the gRPC server is not involved in the Rest-API caching.

Use case 2:
Another use case is authentication. I'm handling authentication and authorization in the API gateway, not the individual gRPC servers. Therefore, I need to annotate individual API calls with the required user roles or permissions, or mark them as "no authentication needed" (for the login-request for example). An interceptor middleware can then verify that the user session (stored in the request-context) is valid for the given RPC method and disallow the call if needed.

Solution
I have already seen similar feature-requests like #410 or #1243, which turned out to "implement a fully-functional plugin system". While it seems that I'm not the only one who needs this feature, a plugin system requires a lot of time and effort to implement, and that is (apparently) not possible with the project's current resources (at least not in the near future). And to be honest, I think a plugin-system does much more than I actually need or want for my purposes and is not really neccessary.

Custom annotation support could be implemented much easier: The only thing that would be needed is the registration of interceptors for certain (or all) RPC methods. The interceptor method would wrap the client-Call and is therefore able to get the method name / protobuf request object and access protobuf annotations from there. It can then use this information to set headers, rewrite protobuf messages, check session information in the context or abort the call altogether.

I'm currently evaluating if grpc-gateway is suitable for my purposes, so I'm still new here (and this is going to be a deciding factor). I'm thinking about providing a serverMux option like this:

mux := runtime.NewServeMux(runtime.WithMethodInterceptor(methodInterceptor))

The method interceptor would receive every argument that the gRPC client call receives and has all the same return types.
Additionally, it receives a callback-function for the client-call itself:

type ClientCall        func(ctx context.Context, req proto.Message, headerMD, trailerMD metadata.MD) (proto.Message, runtime.ServerMetadata, error)
type MethodInterceptor func(ctx context.Context, req proto.Message, headerMD, trailerMD metadata.MD, client ClientCall) (proto.Message, runtime.ServerMetadata, error)

An interceptor that does nothing would simply return client(ctx, req, headerMD, trailerMD), not modifying the gateways behaviour at all.

[maja42]

Alternatively, it would also be good enough for the interceptor to only receive the context and request message. It can manipulate the response headers (or context information like the session object) and simply decide if the gRPC method should be called or not. It doesn't need to have access to gRPC metadata (although it would be nice), and doesn't need to be able to modify the response object.

[jlsumler]

Protobuf already supports custom options, which allows you to attach a variety of information to proto elements: https://developers.google.com/protocol-buffers/docs/proto#customoptions

My team uses this to attach required roles to RPC definitions / API endpoints for example, which simplifies some of our authorization checking.

[maja42]

Yes, I know. And I want to take those options from protobuf and use them inside the grpc-gateway to modify it's behaviour on a per-RPC basis.
Basically, the options are not intended for the gRPC-server, but the gateway only. Right now, it's not possible to write an interceptor/handler/hook in the grpc-gateway which can access the gRPC-method that's going to be called or the the protobuf-options associated with it.

[johanbrandhorst]

Hi @maja42, thanks for your interest in the project! I'm happy to see that you're considering using the gateway for a project. I'm wondering, having read your requirements, if this could just be implemented as a client-side gRPC interceptor? As you understand, we're keen to avoid broadening the scope of the project unless there's a very compelling use case. I'm not sure I see exactly what role the gateway generator has in any of this. If you want to create a new annotation that you can use to generate a gRPC client-side interceptor from, then you can do that an add it to the grpc.Dial call you have to make when creating the gRPC gateway.

Does that fulfill your requirements?

[maja42]

Thank you for your quick response - Registering a grpc interceptor would indeed solve the issue, I didn't know that is possible on the grpc client-side.
It might not be the cleanest solution, but I think I can work with that. In the worst case, I need to pass missing information along in the context object. I'll close the issue and reopen it, if I encounter any problems/missing functionality in the future.