Skip to content

Latest commit

 

History

History
112 lines (78 loc) · 7.08 KB

L108-node-grpc-reflection-library.md

File metadata and controls

112 lines (78 loc) · 7.08 KB
Raw

Node.js Reflection Server Library

Abstract

Create a canonical implementation of the gRPC reflection API for Node.js based on the logic from the nestjs-grpc-reflection library

Background

Since its introduction in 2017 there have been a variety of external node.js implementations for the gRPC Reflection Specification, each of which is in various states of maintenance. A few examples are linked at the bottom of this section.

This feature was initially requested in grpc/grpc-node#79

Related Proposals:

Proposal

We are proposing the creation of a new @grpc/reflection package with the following external interface:

import type { Server as GrpcServer } from '@grpc/grpc-js';
import type { PackageDefinition } from '@grpc/proto-loader';

type MinimalGrpcServer = Pick<GrpcServer, 'addService'>;

interface ReflectionServerOptions {
  services?: string[]; // whitelist of fully-qualified service names to expose. Default: expose all
}

export interface ReflectionServer {
  constructor(pkg: PackageDefinition, options?: ReflectionServerOptions);
  addToServer(server: MinimalGrpcServer);
}

this ReflectionServer class will be used to expose information about the user's gRPC package according to the gRPC Reflection Specification via their existing gRPC Server. Internally, the class will be responsible for managing incoming requests for each of the various published versions of the gRPC Reflection Specification: at the time of writing, this includes v1 and v1alpha but may include more in the future. These version-specific handlers will be isolated into their own services in order to preserve backwards-compatibility, and will look like the following:

reflection.v1.ts

import {
  ExtensionNumberResponse,
  FileDescriptorResponse,
  ListServiceResponse,
} from './proto/grpc/reflection/v1/reflection';

export interface ReflectionV1Implementation {
  constructor(pkg: PackageDefinition);

  listServices(listServices: string): ListServiceResponse;
  fileContainingSymbol(symbol: string): FileDescriptorResponse;
  fileByFilename(filename: string): FileDescriptorResponse;
  fileContainingExtension(symbol: string, field: number): FileDescriptorResponse;
  allExtensionNumbersOfType(symbol: string): ExtensionNumberResponse;
}

Usage

The user will leverage the library in a way very similar to the gRPC health check service by creating a new class to manage the reflection logic and then adding that to the gRPC server:

import { join } from 'path';

import * as grpc from '@grpc/grpc-js';
import * as protoLoader from '@grpc/proto-loader';
import { ReflectionServer } from '@grpc/reflection';

const pkg = protoLoader.loadSync(join(__dirname, 'sample.proto'));
const reflection = new ReflectionServer(pkg);

const server = new grpc.Server();
const proto = grpc.loadPackageDefinition(pkg) as any;
server.addService(proto.sample.SampleService.service, { ... });
reflection.addToServer(server)

server.bindAsync('0.0.0.0:5001', grpc.ServerCredentials.createInsecure(), () => { server.start(); });

Rationale

1. Design Decision: use of proto-loader over protoc

several reflection implementations linked above leverage protoc in order to generate a representation of the proto schema to expose on the API. In this document we propose the use of proto-loader to inspect the schema at runtime instead in order to simplify the developer experience and be consistent with the design of the grpc-health-check library.

2. Design Decision: support multiple reflection implementations

currently not all reflection clients request the v1 version of the spec so we need to include handlers for both v1 and v1alpha to support both during the transition. For this reason we separate the reflection handling logic itself to allow for reuse across multiple service versions

Implementation

I (jtimmons) will implement this once the maintainers have approved

Open issues (if applicable)

User is restricted to loading a single PackageDefinition

ideally we would be able to support the user adding multiple PackageDefinition objects at a time in a similar way to the gRPC server itself, however due to some internal protobuf behavior discussed in this thread this is currently difficult to accomplish. For that reason we will be restricting the user to loading a single PackageDefinition for the time being to avoid any confusing behavior or bugs. Practically, this will prevent the user from being able to load dynamic gRPC services that they are not aware of at startup time until this can be resolved.

The issue is described in more detail below for completeness:

background: when loading a PackageDefinition object via the protoLoader.load(...) function, proto-loader/protobufjs will rename the input .proto files based on their protobuf package name. For example a file named file.proto in the sample package will actually be referred to as sample.proto in all FileDescriptorProto objects in the resulting PackageDefinition.

This behavior can cause issues when multiple files exist within the same package as there can be confusion about what is the "real" contents of a file (which is critical information for the reflection API). Proto-loader/protobufjs handles this for a single load() call by unifying all files into a single package-file; for example: if we have files vendor/a.proto and vendor/b.proto which are both in the vendor protobuf namespace then contents from both files will be combined into a single vendor.proto file descriptor in the PackageDefinition. The issue arises when we attempt multiple invocations of protoLoader.load() as each may only fetch a subset of the package (in this example from a.proto in the first invocation and b.proto in the second). In these cases we have multiple different vendor.proto references which breaks the assumptions of the reflection specification in which files are often looked up by name.