v1.16.0

This release adds support for Protobuf Editions to many of the packages. Note, however, that the desc/protoparse package does not yet work to parse Protobuf Editions source files. This release also includes a handful of bug-fixes.

"github.com/jhump/protoreflect/desc"

Additions:

• Adds an Edition method to *desc.FileDescriptor, which returns non-zero for files that use Protobuf Editions (instead of syntax "proto2" or "proto3").

Changes/fixes:

• Fixes an issue in the various Wrap* functions, that could result in a descriptor hierarchy where the same descriptors were incorrectly present multiple times. This could happen if a file in the transitive dependency graph being wrapped had more than one incoming edge (e.g. same file appearing in import statements from two or more other files).

"github.com/jhump/protoreflect/desc/builder"

Additions:

• Adds an Editions field and SetEditions method to *builder.FileBuilder, for creating files that use Protobuf Editions.

"github.com/jhump/protoreflect/desc/protoprint"

Changes/fixes:

• Now correctly prints the contents for files that use Protobuf Editions. Previously, if given a descriptor for a file used Protobuf Editions, it could generate invalid source output.
• Fixed a bug when Printer.CustomSortFunction was set wherein enum values could be printed in incorrect order, resulting in invalid source output. In syntax "proto3" and in open enums in Protobuf Editions, the first value must be the one with a zero numeric value.

"github.com/jhump/protoreflect/grpcreflect"

Additions:

• Adds a new AllowMissingFileDescriptors method to *grpcreflect.Client that allows users to opt-in to a lenient mode that will try to return schemas even when some files are unavailable. This will only be successful in cases where the missing files only provide custom options or are unused imports. This restores some lenience that was inadvertently provided (due to bugs and missing validation checks) when versions of this repo prior to v1.15 were combined with versions of the google.golang.org/grpc module prior to v1.61.

v1.15.6

This is a bugfix release to address recently identified regressions in the desc/protoparse package -- things that worked in v1.14.1 but did not work in v1.15.x.

"github.com/jhump/protoreflect/desc/protoparse"

Changes/fixes:

• The documentation for the Parser.ImportPaths field specifies that the field is not used for calls to the Parser.ParseFilesButDoNotLink method. This is because import paths do not need to be resolved if not linking. But as of v1.15.0, the ImportPaths field was being used for that method, and the difference in behavior could break existing programs. This has been fixed, and the behavior once again matches the spec.
• As of v1.15.0, if a path provided to Parser.ParseFiles were resolved as descriptors (via the Parser.LookupImport or Parser.LookupImportProto functions) instead of as source code (via the Parser.Accessor function or the default action of os.Open), the parse operation would fail with a "no such file" error. This would succeed with v1.14.1, with the descriptor being used as the input for the file instead of the file being parsed from source. So working programs could have been broken by an upgrade to v1.15.x. The package has been changed back to the v1.14.1 behavior.

v1.15.5

This is a bugfix release to address bugs in the desc/builder, desc/protoparse, and grpcreflect packages and a compatibility issue with the most recent version of github.com/bufbuild/protocompile.

"github.com/jhump/protoreflect/desc/builder"

Changes/fixes:

• A regression was introduced in v1.15.0 of this package that could cause it to fail to include some imports in a file for custom options. If the custom option in question was in a known transitive dependency, the builder could fail to add the necessary import. This has been fixed.

"github.com/jhump/protoreflect/desc/protoparse"

Changes/fixes:

• As of v1.15.0, this package uses the github.com/bufbuild/protocompile package under the hood as the parser. But that package is still using major version zero, since it's API has not completely stabilized. A recent release (v0.8.0) included backwards-incompatible changes that caused this repo to no longer compile. This release addresses the compile errors: this repo now uses that latest release of protocompile and correctly compiles against it.
• The use of v0.8.0 of protocompile fixes some issues in the desc/protoparse package that could cause it to incorrectly accept or reject some Protobuf source files. See the release notes for protocompile v0.8.0 for more details.

"github.com/jhump/protoreflect/grpcreflect"

Changes/fixes:

• The grpcreflect.NewClientAuto function (added in v1.14.0) returns a client that automatically falls back to using v1alpha of the reflection protocol if the server does not support v1. However, it relied on the server returning an "unimplemented" error to decide to fall back, but this condition has been observed to sometimes instead manifest as an "unavailable" error (in proxies that fail to write a 404 error before closing the connection). In those cases, the client was not correctly falling back to v1alpha. This release remedies that, so it can fallback to v1alpha when this category of error occurs.

v1.15.4

This is a bugfix release to address bugs in the desc/protoprint package and a compatibility issue with the most recent version of github.com/bufbuild/protocompile.

"github.com/jhump/protoreflect/desc/protoparse"

Changes/fixes:

• As of v1.15.0, this package uses the github.com/bufbuild/protocompile package under the hood as the parser. But that package is still using major version zero, since it's API has not completely stabilized. A recent release (v0.7.0) included backwards-incompatible changes that caused this repo to no longer compile. This release addresses the compile errors: this repo now uses that latest release of protocompile and correctly compiles against it.

"github.com/jhump/protoreflect/desc/protoprint"

Changes/fixes:

• When a string literal included a double-quote character ("), it would get printed incorrectly leading to either an incorrect string literal in some cases or an invalid and uncompilable source file in others. This has been fixed.
• When sorting elements in the file, if the file used proto3 syntax and had an enum with negative numeric values, the result would include the negative values first, before the zero value. This is not allowed in proto3 syntax, which requires that the zero value be the first value, so the result would be an invalid and uncompilable source file. This has been fixed.
• When a source file is parsed into a FileDescriptorProto but never linked, the descriptor may be missing the type field of a FieldDescriptorProto (since it is not known whether a named type refers to a message, group, or enum), and it may have options left in the uninterpreted_options field of the various options messages, which has a peculiar representation for aggregate values. When printing a descriptor that had these characteristics, they were not correctly handled, so the output of the printer would be an invalid and uncompilable source file. Both of these cases have been fixed.

v1.15.3

This is a bugfix release to address a panics in the desc/builder and desc/protoparse packages.

"github.com/jhump/protoreflect/desc/builder"

Changes/fixes:

• When building an enum where some values in an EnumBuilder had numbers explicitly assigned and others did not, a panic could occur. This panic has now been fixed. When there is a mix of explicit value numbers and implicit numbers, the implicit values will be auto-assigned available numbers (starting at zero) while the explicit values will use the configured number.

"github.com/jhump/protoreflect/desc/protoparse"

Changes/fixes:

• When the protoparse package was overhauled in v1.15.0, a bug was inadvertently introduced to the functionality related to the Parser.InferImportPaths flag. If the files being processed imported a file for which an AST was not available (which is actually quite common, since they may import a standard file such as "google/protobuf/empty.proto"), a panic would occur. This panic has been fixed.
• This release includes some other improvements related to the same Parser.InferImportPaths flag which now allow it to better detect file layouts and remedy import mismatches in more scenarios.

v1.15.2

This is a bugfix release to address issues found in the desc/protoparse package.

"github.com/jhump/protoreflect/desc/protoparse"

Changes/fixes:

• A fix was added to v1.15.1 so that descriptors returned from this package would use statically known extension types for custom options, and custom options in the source that were not statically known would be represented as unrecognized fields. However, the fix was incomplete: only the files whose names were explicitly given to the Parser were updated in this fashion. Other files in the transitive dependency (i.e. imports) did not have options represented this way. As of this release, the fix is complete and all descriptors returned by this package should have the same representation for custom options as in v1.14 and earlier.
• As of v1.15.0, this package uses the github.com/bufbuild/protocompile package under the hood as the parser. But that package is still using major version zero, since it's API has not completely stabilized. A recent release (v0.6.0) included backwards-incompatible changes that caused this repo to no longer compile. This release addresses the compile errors: this repo now uses that latest release of protocompile and correctly compiles against it.

v1.15.1

This is small bugfix release to address regressions reported in v1.15.0.

It also fixes a panic bug that can occur when a program includes both the grpcreflect package from this module and v1.52.0 or newer of the "google.golang.org/grpc" module.

"github.com/jhump/protoreflect/desc/protoparse"

Changes/fixes:

• As of v1.15.0, descriptors returned from this package would include dynamic extension fields to represent option values for custom options that were defined in the set of files being compiled. This was a regression from all previous versions, where custom options were represented using statically known extension fields (i.e. extensions that were generated to Go code and linked into the calling program) or as unrecognized fields. This breaks some usages, where calling code is using type assertions of the extension value to a generated struct (which fails when the value is a dynamic message value). This has been fixed, and the behavior from v1.14.1 and before has been restored.
• As of v1.15.0, a data race was possible. This is was a regression, where #236 revealed its ugly head again. The issue would occur if a parse operation used the Parser.LookupImport or Parser.LookupImportProto field to supply some dependencies as descriptors instead of as source. The race occurred when the same descriptor was referenced from different, concurrent parse operations: one goroutine would modify the descriptor during linking, while another goroutine was trying to access it (without synchronization). It has now been fixed (via a fix in the underlying protocompile library: bufbuild/protocompile#103).

"github.com/jhump/protoreflect/grpcreflect"

Changes/fixes:

• This package includes its own internal copy of generated Go code corresponding to grpc/reflection/v1/reflection.proto. This was because, historically, there were no released Go packages that included it. But as of v1.52.0 of the grpc-go runtime, another package now provides this. If both packages are linked into the same program, a panic will occur during program initialization due to there being multiple, conflicting definitions of these types. As of v1.57.0 of grpc-go, this init-time panic is guaranteed to happen. This panic is resolved in this version (the internal copy is now registered using a non-conflicting name). An example panic stack-trace follows:

  panic: proto: file "reflection.proto" has a name conflict over grpc.reflection.v1.ErrorResponse
  	previously from: "google.golang.org/grpc/reflection/grpc_reflection_v1"
  	currently from:  "github.com/jhump/protoreflect/grpcreflect/internal/grpc_reflection_v1"
  See https://protobuf.dev/reference/go/faq#namespace-conflict
  
  goroutine 1 [running]:
  google.golang.org/protobuf/reflect/protoregistry.glob..func1({0x1075580e0?, 0xc0005e2800?}, {0x1075580e0?, 0xc0005e2840})
          /.../go/pkg/mod/google.golang.org/protobuf@v1.31.0/reflect/protoregistry/registry.go:56 +0x264
  google.golang.org/protobuf/reflect/protoregistry.(*Files).RegisterFile.func1({0x107569940, 0xc0006fc5b0})
          /.../go/pkg/mod/google.golang.org/protobuf@v1.31.0/reflect/protoregistry/registry.go:154 +0x2ec
  google.golang.org/protobuf/reflect/protoregistry.rangeTopLevelDescriptors({0x10756d6c8, 0xc0004e76c0}, 0xc00071ce28)
          /.../go/pkg/mod/google.golang.org/protobuf@v1.31.0/reflect/protoregistry/registry.go:417 +0x148
  google.golang.org/protobuf/reflect/protoregistry.(*Files).RegisterFile(0xc0005ce258, {0x10756d6c8?, 0xc0004e76c0?})
          /.../go/pkg/mod/google.golang.org/protobuf@v1.31.0/reflect/protoregistry/registry.go:149 +0x708
  google.golang.org/protobuf/internal/filedesc.Builder.Build({{0x1072414ee, 0x45}, {0x108070e00, 0x6bd, 0x6bd}, 0x0, 0x8, 0x0, 0x1, {0x10755ec98, ...}, ...})
          /.../go/pkg/mod/google.golang.org/protobuf@v1.31.0/internal/filedesc/build.go:112 +0x22c
  google.golang.org/protobuf/internal/filetype.Builder.Build({{{0x1072414ee, 0x45}, {0x108070e00, 0x6bd, 0x6bd}, 0x0, 0x8, 0x0, 0x1, {0x0, ...}, ...}, ...})
          /.../go/pkg/mod/google.golang.org/protobuf@v1.31.0/internal/filetype/build.go:138 +0x22c
  github.com/jhump/protoreflect/grpcreflect/internal/grpc_reflection_v1.file_reflection_proto_init()
          /.../go/pkg/mod/github.com/jhump/protoreflect@v1.14.0/grpcreflect/internal/grpc_reflection_v1/reflection.pb.go:947 +0x558
  github.com/jhump/protoreflect/grpcreflect/internal/grpc_reflection_v1.init.0()
          /.../go/pkg/mod/github.com/jhump/protoreflect@v1.14.0/grpcreflect/internal/grpc_reflection_v1/reflection.pb.go:818 +0x24
  

v1.15.0

This is a significant and admittedly long overdue release. This release finally provides a bridge for interoperability between packages and functionality in this repo and the new reflection packages and APIs added in the "v2 API" of the Protobuf runtime for Go (google.golang.org/protobuf), particularly the protoreflect, protodesc, protoregistry, and dynamicpb packages.

This repo began because the Protobuf runtime (at the time, the "v1 API" in github.com/golang/protobuf) had no support for Protobuf reflection and dynamic messages. The functionality slowly grew to include dynamic gRPC, interesting meta features like parsing Protobuf sources or printing Protobuf source from a descriptor, etc.

But the v2 API was released nearly three years ago. And it does provide some functionality that this repo provided, but now in the core runtime library. And yet this repo still refers to the now-deprecated v1 API.

Under the hood, this is a very big change, mainly due to the big changes (described below) in the github.com/jhump/protoreflect/desc and github.com/jhump/protoreflect/desc/protoparse packages.

This release is planned to be the final v1 release. Any other changes significant enough to warrant a new minor version will instead be added in an upcoming v2 version. A new v2 version will have zero references to the v1 API of the Protobuf runtime, and it will not duplicate any of the functionality that now exists in the v2 API.

The changes are all detailed below and include all of the changes described in the past two release candidates (v1.15.0-rc1 and v1.15.0-rc2).

---

NOTE: Zero defects were reported in the nearly six weeks since the first release candidate was created. It is entirely possible that very few users tried it, so there still could be some material bugs and compatibility issues lurking in this release. If you encounter any marked changes in behavior between the previous release (v1.14.1) and this one, please file an issue as soon as possible! However, if the issue is a performance degradation, it may not be addressed until a v2. So If your use case is very sensitive to performance, you may need to stay on v1.14.1. The under-the-hood changes in this repo for this release are expected to have some performance impact, though it will hopefully be minor or even negligible for most users.

---

"github.com/jhump/protoreflect/desc"

Changes/fixes:

• This package, the core of this whole repo, with its descriptor interfaces and related functions, has been substantially overhauled. The descriptor values provided by this package are now backed by the descriptor implementations in the google.golang.org/protobuf/reflect/protoreflect package.
  • The primary advantage of this change is that it is now easy to convert a protoreflect.Descriptor to a desc.Descriptor (via the various Wrap* functions in this package). It is similarly easy to unwrap a desc.Descriptor value, to recover the underlying protoreflectDescriptor.
  • This allows this package to be easily used in conjunction with protoreflect and accompanying packages. You can take a protoreflect.Descriptor and then easily use it with the desc/builder and desc/protoprint packages in this repo. Similarly, you can create a desc.Descriptor using the desc/protoparse package and easily turn that into a protoreflect.Descriptor, for use with the google.golang.org/protobuf/... packages.

Additions:

• This adds a new DescriptorWrapper interface, which is implemented by all descriptor implementations in this package. It contains an Unwrap() protoreflect.Descriptor function, to recover the underlying protoreflect.Descriptor.
• Also, all descriptor implementations in this package also have additional methods that are more strongly typed. For example, *desc.FileDescriptor has a method UnwrapFile() protoreflect.FileDescriptor.
• Finally, this package now has numerous Wrap* functions, which accept a protoreflect.Descriptor and wrap it, returning a desc.Descriptor. There is one function for each concrete type, for example for messages there is WrapMessage(d protoreflect.MessageDescriptor) (*MessageDescriptor, error)

"github.com/jhump/protoreflect/desc/builder"

Changes/fixes:

• Previously, not all rules of the Protobuf language were enforced when building a new descriptor with this package. Since the desc.Descriptor values returned by this package are now backed by protoreflect.Descriptor values, more rules are enforced. This is because the implementation of descriptors in the protoreflect package does perform all of those validation checks. The new checks that are now enforced that previously were not:
  1. Files with a syntax of proto3 are not allowed to have required fields.
  2. Files with a syntax of proto3 are not allowed to have messages that define extension ranges.
  3. Files with a syntax of proto3 are not allowed to use groups.
  4. Files with a syntax of proto3 are not allowed to declare default values for fields.
  5. Extension fields must use tag numbers that are in an extension range defined on the extended message.
  6. Non-extension fields are not allowed to use tags that lie in a message's extension ranges or reserved ranges.
  7. Non-extension fields are not allowed to use names that the message has marked as reserved.
  8. Extension ranges and reserved ranges must not overlap.

"github.com/jhump/protoreflect/desc/protoparse"

Changes/fixes:

• This package has been overhauled perhaps even more than the desc package. The implementation in this package has been completely replaced with the functionality of the github.com/bufbuild/protocompile package. So the exported APIs in this package are now just adapters. The actual parser/compiler is implemented in this other dependency. Most of this adaptation logic is trivial with the exception being for the Parser.ParseToAST method, which must convert from protocompile's AST model to the model defined in desc/protoparse/ast.

"github.com/jhump/protoreflect/desc/protoprint"

Changes/fixes:

• Formatting of message literals in custom option values has been improved.
  • It now encloses nested messages in curly braces ({ and }) instead of angle brackets (< and >), which is better aligned with the commonly-used and preferred syntax style.
  • Also, nested messages now respect the Printer.MessageLiteralExpansionThresholdLength, instead of this threshold only being applied for a top-level message. So small nested messages can be emitted in compact form, even if enclosed within a message literal that has been expanded.
• If printing descriptors that had no source code info and no other sort configuration was used on the Printer, then elements with multiple option declarations would have those options printed in seemingly random order. The actual order was based on a map iteration, and was thus non-deterministic. This has been fixed. In such a case, options will now be ordered by name.

"github.com/jhump/protoreflect/desc/sourceinfo"

Additions:

• A new TypeResolver interface has been added, as well as a GlobalTypes package variable that implements that interface. This is analogous to protoregistry.GlobalTypes, just as the existing sourceinfo.GlobalFiles corresponds to protoregistry.GlobalFiles.
• This adds new Wrap* functions, that accept various protoreflect types and return values that are identical to the input values except that they include source code info that was registered with this package.

"github.com/jhump/protoreflect/dynamic"

Changes/fixes:

• Previously, if protoreflect APIs were used to store a value in an extension field, using a protoreflect.FieldDescriptor that was not a generated extension (i.e. not known at compile-time), trying to convert the resulting message to a *dynamic.Message would overlook that field; it would be absent from the result, not even appearing in the unrecognized fields. This has been fixed. Such extension values will be in the resulting *dynamic.Message and appear as recognized/known fields.

v1.15.0-rc2

This is still a release candidate due to the significance of changes to under-the-hood machinery. But we expect to cut a proper v1.15.0 soon.

In addition to all of the many changes in v1.15.0-rc1, this release includes the following:

"github.com/jhump/protoreflect/desc/protoparse"

Changes/fixes:

• Fuzz testing identified a couple of potentially severe performance issues in the underlying protocompile package implementation. These were innocuous-looking issues that could become extreme if given the right kind of pathological input. These issues were fixed, and the fixed version of protocompile has been pulled into this repo.

"github.com/jhump/protoreflect/desc/protoprint"

Changes/fixes:

• If printing descriptors that had no source code info and no other sort configuration was used on the Printer, then elements with multiple option declarations would have those options printed in seemingly random order. The actual order was based on a map iteration, and was thus non-deterministic. This has been fixed. In such a case, options will now be ordered by name.

v1.15.0-rc1

This is a very big change under-the-hood, which is why this is a release candidate. Unfortunately, go get -u ... and tools like Depend-a-bot will not take release candidates/pre-release versions. So this candidate is a sort of shot in the dark, hoping that some brave souls might try it out and provide feedback before I stamp it as an official release.

All of CI is green, I've run apidiff to make sure there are no backwards-incompatible changes in the exported API, and I've done some other exploratory testing to make sure this is all good. My hope is that no one will notice a thing. If they do notice something, it might be a slight performance degradation, but hopefully not consequential enough to be a problem.

I expect v1.15 to be the last v1 version. Any other changes significant enough to warrant a new minor version will instead be added in an upcoming v2 version. This version is intended to be a stop-gap between v1 and v2 as it provides a long-needed bridge between the functionality in this repo and the newer google.golang.org/protobuf/reflect/protoreflect package (and its accompanying packages, protodesc, protoregistry, and dynamicpb).

"github.com/jhump/protoreflect/desc"

Changes/fixes:

• This package, the core of this whole repo, with its descriptor interfaces and related functions, has been substantially overhauled. The descriptor values provided by this package are now backed by the descriptor implementations in the google.golang.org/protobuf/reflect/protoreflect package.
  • The primary advantage of this change is that it is now easy to convert a protoreflect.Descriptor to a desc.Descriptor (via the various Wrap* functions in this package). It is similarly easy to unwrap a desc.Descriptor value, to recover the underlying protoreflectDescriptor.
  • This allows this package to be easily used in conjunction with protoreflect and accompanying packages. You can take a protoreflect.Descriptor and then easily use it with the desc/builder and desc/protoprint packages in this repo. Similarly, you can create a desc.Descriptor using the desc/protoparse package and easily turn that into a protoreflect.Descriptor, for use with the google.golang.org/protobuf/... packages.

Additions:

• This adds a new DescriptorWrapper interface, which is implemented by all descriptor implementations in this package. It contains an Unwrap() protoreflect.Descriptor function, to recover the underlying protoreflect.Descriptor.
• Also, all descriptor implementations in this package also have additional methods that are more strongly typed. For example, *desc.FileDescriptor has a method UnwrapFile() protoreflect.FileDescriptor.
• Finally, this package now has numerous Wrap* functions, which accept a protoreflect.Descriptor and wrap it, returning a desc.Descriptor. There is one function for each concrete type, for example for messages there is WrapMessage(d protoreflect.MessageDescriptor) (*MessageDescriptor, error)

"github.com/jhump/protoreflect/desc/builder"

Changes/fixes:

• Previously, not all rules of the Protobuf language were enforced when building a new descriptor with this package. Since the desc.Descriptor values returned by this package are now backed by protoreflect.Descriptor values, more rules are enforced. This is because the implementation of descriptors in the protoreflect package does perform all of those validation checks. The new checks that are now enforced that previously were not:
  1. Files with a syntax of proto3 are not allowed to have required fields.
  2. Files with a syntax of proto3 are not allowed to have messages that define extension ranges.
  3. Files with a syntax of proto3 are not allowed to use groups.
  4. Files with a syntax of proto3 are not allowed to declare default values for fields.
  5. Extension fields must use tag numbers that are in an extension range defined on the extended message.
  6. Non-extension fields are not allowed to use tags that lie in a message's extension ranges or reserved ranges.
  7. Non-extension fields are not allowed to use names that the message has marked as reserved.
  8. Extension ranges and reserved ranges must not overlap.

"github.com/jhump/protoreflect/desc/protoparse"

Changes/fixes:

• This package has been overhauled perhaps even more than the desc package. The implementation in this package has been completely replaced with the functionality of the github.com/bufbuild/protocompile package. So the exported APIs in this package are now just adapters. The actual parser/compiler is implemented in this other dependency. Most of this adaptation logic is trivial with the exception being for the Parser.ParseToAST method, which must convert from protocompile's AST model to the model defined in desc/protoparse/ast.

"github.com/jhump/protoreflect/desc/protoprint"

Changes/fixes:

• Formatting of message literals in custom option values has been improved.
  • It now encloses nested messages in curly braces ({ and }) instead of angle brackets (< and >), which is better aligned with the commonly-used and preferred syntax style.
  • Also, nested messages now respect the Printer.MessageLiteralExpansionThresholdLength, instead of this threshold only being applied for a top-level message. So small nested messages can be emitted in compact form, even if enclosed within a message literal that has been expanded.

"github.com/jhump/protoreflect/desc/sourceinfo"

Additions:

• A new TypeResolver interface has been added, as well as a GlobalTypes package variable that implements that interface. This is analogous to protoregistry.GlobalTypes, just as the existing sourceinfo.GlobalFiles corresponds to protoregistry.GlobalFiles.
• This adds new Wrap* functions, that accept various protoreflect types and return values that are identical to the input values except that they include source code info that was registered with this package.

"github.com/jhump/protoreflect/dynamic"

Changes/fixes:

• Previously, if protoreflect APIs were used to store a value in an extension field, using a protoreflect.FieldDescriptor that was not a generated extension (i.e. not known at compile-time), trying to convert the resulting message to a *dynamic.Message would overlook that field; it would be absent from the result, not even appearing in the unrecognized fields. This has been fixed. Such extension values will be in the resulting *dynamic.Message and appear as recognized/known fields.