/
CHANGES.txt
2822 lines (2423 loc) · 128 KB
/
CHANGES.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
2021-05-25 version 3.17.2 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
Kotlin
* Fix duplicate class error (#8653)
PHP
* Fixed SEGV in sub-message getters for well-known types when message is unset
(#8670)
2021-05-07 version 3.17.1 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
PHP
* Fixed PHP memory leaks and arginfo errors. (#8614)
* Fixed JSON parser to allow multiple values from the same oneof as long as
all but one are null.
Ruby
* Fixed memory bug: properly root repeated/map field when assigning. (#8639)
* Fixed JSON parser to allow multiple values from the same oneof as long as
all but one are null.
2021-05-07 version 3.17.0 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
Protocol Compiler
* Fix the generated source information for reserved values in Enums.
C++
* Fix -Wunused-parameter in map<string, int> fields (fixes #8494) (#8500)
* Use byteswap.h when building against musl libc (#8503)
* Fix -Wundefined-inline error when using SharedCtor() or SharedDtor() (#8532)
* Fix bug where `Descriptor::DebugString()` printed proto3 synthetic oneofs.
* Provide stable versions of `SortAndUnique()`.
* Make sure to cache proto3 optional message fields when they are cleared.
* Expose UnsafeArena methods to Reflection.
* Use std::string::empty() rather than std::string::size() > 0.
Kotlin
* Restrict extension setter and getter operators to non-nullable T.
Java
* updating GSON and Guava to more recent versions (#8524)
* Reduce the time spent evaluating isExtensionNumber by storing the extension
ranges in a TreeMap for faster queries. This is particularly relevant for
protos which define a large number of extension ranges, for example when
each tag is defined as an extension.
* Fix java bytecode estimation logic for optional fields.
* Optimize Descriptor.isExtensionNumber.
Python
* Add MethodDescriptor.CopyToProto() (#8327)
* Remove unused python_protobuf.{cc,h} (#8513)
* Start publishing python aarch64 manylinux wheels normally (#8530)
* Fix constness issue detected by MSVC standard conforming mode (#8568)
* Make JSON parsing match C++ and Java when multiple fields from the same
oneof are present and all but one is null.
Ruby
* Add support for proto3 json_name in compiler and field definitions (#8356)
* Fixed memory leak of Ruby arena objects. (#8461)
* Fix source gem compilation (#8471)
* Fix various exceptions in Ruby on 64-bit Windows (#8563)
* Fix crash when calculating Message hash values on 64-bit Windows (#8565)
Conformance Tests
* Added a conformance test for the case of multiple fields from the same
oneof.
Other
* Opensourcing kotlin protos (#8272)
* Use a newer version of rules_proto, with the new rule `proto_descriptor_set` (#8469)
3.16.0 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
C++
* The ::pb namespace is no longer exposed due to conflicts.
* Allow MessageDifferencer::TreatAsSet() (and friends) to override previous
calls instead of crashing.
* Reduce the size of generated proto headers for protos with `string` or
`bytes` fields.
* Move arena() operation on uncommon path to out-of-line routine
* For iterator-pair function parameter types, take both iterators by value.
* Code-space savings and perhaps some modest performance improvements in
RepeatedPtrField.
* Eliminate nullptr check from every tag parse.
* Remove unused _$name$_cached_byte_size_ fields.
* Serialize extension ranges together when not broken by a proto field in the
middle.
* Do out-of-line allocation and deallocation of string object in ArenaString.
* Streamline ParseContext::ParseMessage<T> to avoid code bloat and improve
performance.
* New member functions RepeatedField::Assign, RepeatedPtrField::{Add, Assign}.
* Fix undefined behavior warning due to innocuous uninitialization of value
on an error path.
* Avoid expensive inlined code space for encoding message length for messages
>= 128 bytes and instead do a procedure call to a shared out-of-line routine.
* util::DefaultFieldComparator will be final in a future version of protobuf.
Subclasses should inherit from SimpleFieldComparator instead.
Java:
* Detect invalid overflow of byteLimit and return InvalidProtocolBufferException as documented.
* Exceptions thrown while reading from an InputStream in parseFrom are now
included as causes.
* Support potentially more efficient proto parsing from RopeByteStrings.
* Clarify runtime of ByteString.Output.toStringBuffer().
Python
* Fixed a bug in text format where a trailing colon was printed for repeated field.
* When TextFormat encounters a duplicate message map key, replace the current
one instead of merging.
JavaScript
* Make Any.pack() chainable.
2021-04-07 version 3.15.8 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
Ruby
* Fixed memory leak of Ruby arena objects (#8461)
2021-04-02 version 3.15.7 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
C++
* Remove the ::pb namespace (alias) (#8423)
Ruby
* Fix unbounded memory growth for Ruby <2.7 (#8429)
* Fixed message equality in cases where the message type is different (#8434)
2021-03-10 version 3.15.6 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
Ruby
* Fixed bug in string comparison logic (#8386)
2021-03-04 version 3.15.5 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
Ruby
* Fixed quadratic memory use in array append (#8379)
PHP
* Fixed quadratic memory use in array append (#8379)
C++
* Do not disable RTTI by default in the CMake build (#8377)
2021-03-02 version 3.15.4 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
Ruby
* Fixed SEGV when users pass nil messages (#8363)
* Fixed quadratic memory usage when appending to arrays (#8364)
C++
* Create a CMake option to control whether or not RTTI is enabled (#8361)
PHP
* read_property() handler is not supposed to return NULL (#8362)
2021-02-25 version 3.15.3 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
Ruby
* Ruby <2.7 now uses WeakMap too, which prevents memory leaks. (#8341)
2021-02-23 version 3.15.2 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
Ruby
* Fix for FieldDescriptor.get(msg) (#8330)
C++
* Fix PROTOBUF_CONSTINIT macro redefinition (#8323)
2021-02-05 version 3.15.1 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
Ruby
* Bugfix for Message.[] for repeated or map fields (#8313)
2021-02-05 version 3.15.0 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
Protocol Compiler
* Optional fields for proto3 are enabled by default, and no longer require
the --experimental_allow_proto3_optional flag.
C++
* MessageDifferencer: fixed bug when using custom ignore with multiple
unknown fields
* Use init_seg in MSVC to push initialization to an earlier phase.
* Runtime no longer triggers -Wsign-compare warnings.
* Fixed -Wtautological-constant-out-of-range-compare warning.
* DynamicCastToGenerated works for nullptr input for even if RTTI is disabled
* Arena is refactored and optimized.
* Clarified/specified that the exact value of Arena::SpaceAllocated() is an
implementation detail users must not rely on. It should not be used in
unit tests.
* Change the signature of Any::PackFrom() to return false on error.
* Add fast reflection getter API for strings.
* Constant initialize the global message instances
* Avoid potential for missed wakeup in UnknownFieldSet
* Now Proto3 Oneof fields have "has" methods for checking their presence in
C++.
* Bugfix for NVCC
* Return early in _InternalSerialize for empty maps.
* Adding functionality for outputting map key values in proto path logging
output (does not affect comparison logic) and stop printing 'value' in the
path. The modified print functionality is in the
MessageDifferencer::StreamReporter.
* Fixed https://github.com/protocolbuffers/protobuf/issues/8129
* Ensure that null char symbol, package and file names do not result in a
crash.
* Constant initialize the global message instances
* Pretty print 'max' instead of numeric values in reserved ranges.
* Removed remaining instances of std::is_pod, which is deprecated in C++20.
* Changes to reduce code size for unknown field handling by making uncommon
cases out of line.
* Fix std::is_pod deprecated in C++20 (#7180)
* Fix some -Wunused-parameter warnings (#8053)
* Fix detecting file as directory on zOS issue #8051 (#8052)
* Don't include sys/param.h for _BYTE_ORDER (#8106)
* remove CMAKE_THREAD_LIBS_INIT from pkgconfig CFLAGS (#8154)
* Fix TextFormatMapTest.DynamicMessage issue#5136 (#8159)
* Fix for compiler warning issue#8145 (#8160)
* fix: support deprecated enums for GCC < 6 (#8164)
* Fix some warning when compiling with Visual Studio 2019 on x64 target (#8125)
Python
* Provided an override for the reverse() method that will reverse the internal
collection directly instead of using the other methods of the BaseContainer.
* MessageFactory.CreateProtoype can be overridden to customize class creation.
* Fix PyUnknownFields memory leak (#7928)
* Add macOS big sur compatibility (#8126)
JavaScript
* Generate `getDescriptor` methods with `*` as their `this` type.
* Enforce `let/const` for generated messages.
* js/binary/utils.js: Fix jspb.utils.joinUnsignedDecimalString to work with negative bitsLow and low but non-zero bitsHigh parameter. (#8170)
PHP
* Added support for PHP 8. (#8105)
* unregister INI entries and fix invalid read on shutdown (#8042)
* Fix PhpDoc comments for message accessors to include "|null". (#8136)
* fix: convert native PHP floats to single precision (#8187)
* Fixed PHP to support field numbers >=2**28. (#8235)
* feat: add support for deprecated fields to PHP compiler (#8223)
* Protect against stack overflow if the user derives from Message. (#8248)
* Fixed clone for Message, RepeatedField, and MapField. (#8245)
* Updated upb to allow nonzero offset minutes in JSON timestamps. (#8258)
Ruby
* Added support for Ruby 3. (#8184)
* Rewrote the data storage layer to be based on upb_msg objects from the
upb library. This should lead to much better parsing performance,
particularly for large messages. (#8184).
* Fill out JRuby support (#7923)
* [Ruby] Fix: (SIGSEGV) gRPC-Ruby issue on Windows. memory alloc infinite
recursion/run out of memory (#8195)
* Fix jruby support to handle messages nested more than 1 level deep (#8194)
Java
* Avoid possible UnsupportedOperationException when using CodedInputSteam
with a direct ByteBuffer.
* Make Durations.comparator() and Timestamps.comparator() Serializable.
* Add more detailed error information for dynamic message field type
validation failure
* Removed declarations of functions declared in java_names.h from
java_helpers.h.
* Now Proto3 Oneof fields have "has" methods for checking their presence in
Java.
* Annotates Java proto generated *_FIELD_NUMBER constants.
* Add -assumevalues to remove JvmMemoryAccessor on Android.
C#
* Fix parsing negative Int32Value that crosses segment boundary (#8035)
* Change ByteString to use memory and support unsafe create without copy (#7645)
* Optimize MapField serialization by removing MessageAdapter (#8143)
* Allow FileDescriptors to be parsed with extension registries (#8220)
* Optimize writing small strings (#8149)
2020-11-11 version 3.14.0 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
Protocol Compiler
* The proto compiler no longer requires a .proto filename when it is not
generating code.
* Added flag `--deterministic_output` to `protoc --encode=...`.
* Fixed deadlock when using google.protobuf.Any embedded in aggregate options.
C++
* Arenas are now unconditionally enabled. cc_enable_arenas no longer has
any effect.
* Removed inlined string support, which is incompatible with arenas.
* Fix a memory corruption bug in reflection when mixing optional and
non-optional fields.
* Make SpaceUsed() calculation more thorough for map fields.
* Add stack overflow protection for text format with unknown field values.
* FieldPath::FollowAll() now returns a bool to signal if an out-of-bounds
error was encountered.
* Performance improvements for Map.
* Minor formatting fix when dumping a descriptor to .proto format with
DebugString.
* UBSAN fix in RepeatedField (#2073).
* When running under ASAN, skip a test that makes huge allocations.
* Fixed a crash that could happen when creating more than 256 extensions in
a single message.
* Fix a crash in BuildFile when passing in invalid descriptor proto.
* Parser security fix when operating with CodedInputStream.
* Warn against the use of AllowUnknownExtension.
* Migrated to C++11 for-range loops instead of index-based loops where
possible. This fixes a lot of warnings when compiling with -Wsign-compare.
* Fix segment fault for proto3 optional (#7805)
* Adds a CMake option to build `libprotoc` separately (#7949)
Java
* Bugfix in mergeFrom() when a oneof has multiple message fields.
* Fix RopeByteString.RopeInputStream.read() returning -1 when told to read
0 bytes when not at EOF.
* Redefine remove(Object) on primitive repeated field Lists to avoid
autoboxing.
* Support "\u" escapes in textformat string literals.
* Trailing empty spaces are no longer ignored for FieldMask.
* Fix FieldMaskUtil.subtract to recursively remove mask.
* Mark enums with `@java.lang.Deprecated` if the proto enum has option
`deprecated = true;`.
* Adding forgotten duration.proto to the lite library (#7738)
Python
* Print google.protobuf.NullValue as null instead of "NULL_VALUE" when it is
used outside WKT Value/Struct.
* Fix bug occurring when attempting to deep copy an enum type in python 3.
* Add a setuptools extension for generating Python protobufs (#7783)
* Remove uses of pkg_resources in non-namespace packages. (#7902)
* [bazel/py] Omit google/__init__.py from the Protobuf runtime. (#7908)
* Removed the unnecessary setuptools package dependency for Python package (#7511)
* Fix PyUnknownFields memory leak (#7928)
PHP
* Added support for "==" to the PHP C extension (#7883)
* Added `==` operators for Map and Array. (#7900)
* Native C well-known types (#7944)
* Optimized away hex2bin() call in generated code (#8006)
* New version of upb, and a new hash function wyhash in third_party. (#8000)
* add missing hasOneof method to check presence of oneof fields (#8003)
Go:
* Update go_package options to reference google.golang.org/protobuf module.
C#:
* annotate ByteString.CopyFrom(ReadOnlySpan<byte>) as SecuritySafeCritical (#7701)
* Fix C# optional field reflection when there are regular fields too (#7705)
* Fix parsing negative Int32Value that crosses segment boundary (#8035)
Javascript:
* JS: parse (un)packed fields conditionally (#7379)
2020-07-14 version 3.13.0 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
PHP:
* The C extension is completely rewritten. The new C extension has significantly
better parsing performance and fixes a handful of conformance issues. It will
also make it easier to add support for more features like proto2 and proto3 presence.
* The new C extension does not support PHP 5.x. PHP 5.x users can still use pure-PHP.
C++:
* Removed deprecated unsafe arena string accessors
* Enabled heterogeneous lookup for std::string keys in maps.
* Removed implicit conversion from StringPiece to std::string
* Fix use-after-destroy bug when the Map is allocated in the arena.
* Improved the randomness of map ordering
* Added stack overflow protection for text format with unknown fields
* Use std::hash for proto maps to help with portability.
* Added more Windows macros to proto whitelist.
* Arena constructors for map entry messages are now marked "explicit"
(for regular messages they were already explicit).
* Fix subtle aliasing bug in RepeatedField::Add
* Fix mismatch between MapEntry ByteSize and Serialize with respect to unset
fields.
Python:
* JSON format conformance fixes:
* Reject lowercase t for Timestamp json format.
* Print full_name directly for extensions (no camelCase).
* Reject boolean values for integer fields.
* Reject NaN, Infinity, -Infinity that is not quoted.
* Base64 fixes for bytes fields: accept URL-safe base64 and missing padding.
* Bugfix for fields/files named "async" or "await".
* Improved the error message when AttributeError is returned from __getattr__
in EnumTypeWrapper.
Java:
* Fixed a bug where setting optional proto3 enums with setFooValue() would
not mark the value as present.
* Add Subtract function to FieldMaskUtil.
C#:
* Dropped support for netstandard1.0 (replaced by support for netstandard1.1).
This was required to modernize the parsing stack to use the `Span<byte>`
type internally. (#7351)
* Add `ParseFrom(ReadOnlySequence<byte>)` method to enable GC friendly
parsing with reduced allocations and buffer copies. (#7351)
* Add support for serialization directly to a `IBufferWriter<byte>` or
to a `Span<byte>` to enable GC friendly serialization.
The new API is available as extension methods on the `IMessage` type. (#7576)
* Add `GOOGLE_PROTOBUF_REFSTRUCT_COMPATIBILITY_MODE` define to make
generated code compatible with old C# compilers (pre-roslyn compilers
from .NET framework and old versions of mono) that do not support
ref structs. Users that are still on a legacy stack that does
not support C# 7.2 compiler might need to use the new define
in their projects to be able to build the newly generated code. (#7490)
* Due to the major overhaul of parsing and serialization internals (#7351 and #7576),
it is recommended to regenerate your generated code to achieve the best
performance (the legacy generated code will still work, but might incur
a slight performance penalty).
2020-07-28 version 3.12.4 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
This release contains no significant changes, but exists because 3.12.3 was
mistakenly tagged at the wrong commit.
2020-06-01 version 3.12.3 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
Objective-C
* Tweak the union used for Extensions to support old generated code. #7573
2020-05-26 version 3.12.2 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
C++
* Simplified the template export macros to fix the build for mingw32. (#7539)
Objective-C
* Fix for the :protobuf_objc target in the Bazel BUILD file. (#7538)
2020-05-20 version 3.12.1 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
Ruby
* Re-add binary gems for Ruby 2.3 and 2.4. These are EOL upstream, however
many people still use them and dropping support will require more
coordination.
2020-05-12 version 3.12.0 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
Protocol Compiler
* [experimental] Singular, non-message typed fields in proto3 now support
presence tracking. This is enabled by adding the "optional" field label and
passing the --experimental_allow_proto3_optional flag to protoc.
* For usage info, see docs/field_presence.md.
* During this experimental phase, code generators should update to support
proto3 presence, see docs/implementing_proto3_presence.md for instructions.
* Allow duplicate symbol names when multiple descriptor sets are passed on
the command-line, to match the behavior when multiple .proto files are passed.
* Deterministic `protoc --descriptor_set_out` (#7175)
C++
* [experimental] Added proto3 presence support.
* New descriptor APIs to support proto3 presence.
* Enable Arenas by default on all .proto files.
* Documented that users are not allowed to subclass Message or MessageLite.
* Mark generated classes as final; inheriting from protos is strongly discouraged.
* Add stack overflow protection for text format with unknown fields.
* Add accessors for map key and value FieldDescriptors.
* Add FieldMaskUtil::FromFieldNumbers().
* MessageDifferencer: use ParsePartial() on Any fields so the diff does not
fail when there are missing required fields.
* ReflectionOps::Merge(): lookup messages in the right factory, if it can.
* Added Descriptor::WellKnownTypes enum and Descriptor::well_known_type()
accessor as an easier way of determining if a message is a Well-Known Type.
* Optimized RepeatedField::Add() when it is used in a loop.
* Made proto move/swap more efficient.
* De-virtualize the GetArena() method in MessageLite.
* Improves performance of json_stream_parser.cc by factor 1000 (#7230)
* bug: #7076 undefine Windows OUT and OPTIONAL macros (#7087)
* Fixed a bug in FieldDescriptor::DebugString() that would erroneously print
an "optional" label for a field in a oneof.
* Fix bug in parsing bool extensions that assumed they are always 1 byte.
* Fix off-by-one error in FieldOptions::ByteSize() when extensions are present.
* Clarified the comments to show an example of the difference between
Descriptor::extension and DescriptorPool::FindAllExtensions.
* Add a compiler option 'code_size' to force optimize_for=code_size on all
protos where this is possible.
Java
* [experimental] Added proto3 presence support.
* Mark java enum _VALUE constants as @Deprecated if the enum field is deprecated
* reduce <clinit> size for enums with allow_alias set to true.
* Sort map fields alphabetically by the field's key when printing textproto.
* Fixed a bug in map sorting that appeared in -rc1 and -rc2 (#7508).
* TextFormat.merge() handles Any as top level type.
* Throw a descriptive IllegalArgumentException when calling
getValueDescriptor() on enum special value UNRECOGNIZED instead of
ArrayIndexOutOfBoundsException.
* Fixed an issue with JsonFormat.printer() where setting printingEnumsAsInts()
would override the configuration passed into includingDefaultValueFields().
* Implement overrides of indexOf() and contains() on primitive lists returned
for repeated fields to avoid autoboxing the list contents.
* Add overload to FieldMaskUtil.fromStringList that accepts a descriptor.
* [bazel] Move Java runtime/toolchains into //java (#7190)
Python
* [experimental] Added proto3 presence support.
* [experimental] fast import protobuf module, only works with cpp generated code linked in.
* Truncate 'float' fields to 4 bytes of precision in setters for pure-Python
implementation (C++ extension was already doing this).
* Fixed a memory leak in C++ bindings.
* Added a deprecation warning when code tries to create Descriptor objects
directly.
* Fix unintended comparison between bytes and string in descriptor.py.
* Avoid printing excess digits for float fields in TextFormat.
* Remove Python 2.5 syntax compatibility from the proto compiler generated _pb2.py module code.
* Drop 3.3, 3.4 and use single version docker images for all python tests (#7396)
JavaScript
* Fix js message pivot selection (#6813)
PHP
* Persistent Descriptor Pool (#6899)
* Implement lazy loading of php class for proto messages (#6911)
* Correct @return in Any.unpack docblock (#7089)
* Ignore unknown enum value when ignore_unknown specified (#7455)
Ruby
* [experimental] Implemented proto3 presence for Ruby. (#7406)
* Stop building binary gems for ruby <2.5 (#7453)
* Fix for wrappers with a zero value (#7195)
* Fix for JSON serialization of 0/empty-valued wrapper types (#7198)
* Call "Class#new" over rb_class_new_instance in decoding (#7352)
* Build extensions for Ruby 2.7 (#7027)
* assigning 'nil' to submessage should clear the field. (#7397)
C#
* [experimental] Add support for proto3 presence fields in C# (#7382)
* Mark GetOption API as obsolete and expose the "GetOptions()" method on descriptors instead (#7491)
* Remove Has/Clear members for C# message fields in proto2 (#7429)
* Enforce recursion depth checking for unknown fields (#7132)
* Fix conformance test failures for Google.Protobuf (#6910)
* Cleanup various bits of Google.Protobuf (#6674)
* Fix latest ArgumentException for C# extensions (#6938)
* Remove unnecessary branch from ReadTag (#7289)
Objective-C
* [experimental] ObjC Proto3 optional support (#7421)
* Block subclassing of generated classes (#7124)
* Use references to Obj C classes instead of names in descriptors. (#7026)
* Revisit how the WKTs are bundled with ObjC. (#7173)
Other
* Add a proto_lang_toolchain for javalite (#6882)
* [bazel] Update gtest and deprecate //external:{gtest,gtest_main} (#7237)
* Add application note for explicit presence tracking. (#7390)
* Howto doc for implementing proto3 presence in a code generator. (#7407)
2020-02-14 version 3.11.4 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
C#
* Fix latest ArgumentException for C# extensions (#7188)
* Enforce recursion depth checking for unknown fields (#7210)
Ruby
* Fix wrappers with a zero value (#7195)
* Fix JSON serialization of 0/empty-valued wrapper types (#7198)
2020-01-31 version 3.11.3 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
C++
* Add OUT and OPTIONAL to windows portability files (#7087)
PHP
* Refactored ulong to zend_ulong for php7.4 compatibility (#7147)
* Call register_class before getClass from desc to fix segfault (#7077)
2019-12-10 version 3.11.2 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
PHP
* Make c extension portable for php 7.4 (#6968)
2019-12-02 version 3.11.1 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
PHP
* Extern declare protobuf_globals (#6946)
2019-11-19 version 3.11.0 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
C++
* Make serialization method naming consistent
* Make proto runtime + generated code free of deprecation warnings
* Moved ShutdownProtobufLibrary() to message_lite.h. For backward compatibility a declaration is still available in stubs/common.h, but users should prefer message_lite.h
* Removed non-namespace macro EXPECT_OK()
* Removed mathlimits.h from stubs in favor of using std::numeric_limits from C++11
* Fixed bug in parser when ending on a group tag
* Add a helper function to UnknownFieldSet to deal with the changing return value of message::unknown_fields()
* Fix incorrect use of string_view iterators
* Support direct pickling of nested messages
* Skip extension tag validation for MessageSet if unknown dependencies are allowed
* Updated deprecation macros to annotate deprecated code (#6612)
* Remove conversion warning in MapEntryFuncs::ByteSizeLong (#6766)
* Revert "Make shared libraries be able to link to MSVC static runtime libraries, so that VC runtime is not required." (#6914)
Java
* Remove the usage of MethodHandle, so that Android users prior to API version 26 can use protobuf-java
* Publish ProGuard config for javalite
* Fix for StrictMode disk read violation in ExtensionRegistryLite
* Include part of the ByteString's content in its toString().
* Include unknown fields when merging proto3 messages in Java lite builders
Python
* Add float_precision option in json format printer
* Optionally print bytes fields as messages in unknown fields, if possible
* FieldPath: fix testing IsSet on root path ''
* Experimental code gen (fast import protobuf module) which only work with cpp generated code linked in
JavaScript
* Remove guard for Symbol iterator for jspb.Map
PHP
* Avoid too much overhead in layout_init (#6716)
* Lazily Create Singular Wrapper Message (#6833)
* Implement lazy loading of php class for proto messages (#6911)
Ruby
* Ruby lazy wrappers optimization (#6797)
C#
* (RepeatedField): Capacity property to resize the internal array (#6530)
* Experimental proto2 support is now officially available (#4642, #5183, #5350, #5936)
* Getting started doc: https://github.com/protocolbuffers/protobuf/blob/master/docs/csharp/proto2.md
* Add length checks to ExtensionCollection (#6759)
* Optimize parsing of some primitive and wrapper types (#6843)
* Use 3 parameter Encoding.GetString for default string values (#6828)
* Change _Extensions property to normal body rather than expression (#6856)
Objective C
* Fixed unaligned reads for 32bit arm with newer Xcode versions (#6678)
2019-09-03 version 3.10.0 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
C++
* Switch the proto parser to the faster MOMI parser.
* Properly escape Struct keys in the proto3 JSON serializer.
* Fix crash on uninitialized map entries.
* Informed the compiler of has-bit invariant to produce better code
* Unused imports of files defining descriptor extensions will now be reported
* Add proto2::util::RemoveSubranges to remove multiple subranges in linear time.
* Added BaseTextGenerator::GetCurrentIndentationSize()
* Made implicit weak fields compatible with the Apple linker
* Support 32 bit values for ProtoStreamObjectWriter to Struct.
* Removed the internal-only header coded_stream_inl.h and the internal-only methods defined there.
* Enforced no SWIG wrapping of descriptor_database.h (other headers already had this restriction).
* Implementation of the equivalent of the MOMI parser for serialization. This removes one of the two serialization routines, by making the fast array serialization routine completely general. SerializeToCodedStream can now be implemented in terms of the much much faster array serialization. The array serialization regresses slightly, but when array serialization is not possible this wins big.
* Do not convert unknown field name to snake case to accurately report error.
* Fix a UBSAN warnings. (#6333)
* Add podspec for C++ (#6404)
* protoc: fix source code info location for missing label (#6436)
* C++ Add move constructor for Reflection's SetString (#6477)
Java
* Call loadDescriptor outside of synchronized block to remove one possible source of deadlock.
* Have oneof enums implement a separate interface (other than EnumLite) for clarity.
* Opensource Android Memory Accessors
* Update TextFormat to make use of the new TypeRegistry.
* Support getFieldBuilder and getRepeatedFieldBuilder in ExtendableBuilder
* Update JsonFormat to make use of the new TypeRegistry.
* Add proguard config generator for GmmBenchmarkSuiteLite.
* Change ProtobufArrayList to use Object[] instead of ArrayList for 5-10% faster parsing
* Implement ProtobufArrayList.add(E) for 20% (5%-40%) faster overall protolite2 parsing
* Make a copy of JsonFormat.TypeRegistry at the protobuf top level package. This will eventually replace JsonFormat.TypeRegistry.
* Fix javadoc warnings in generated files (#6231)
* Java: Add Automatic-Module-Name entries to the Manifest (#6568)
Python
* Add descriptor methods in descriptor_pool are deprecated.
* Uses explicit imports to prevent multithread test failures in py3.
* Added __delitem__ for Python extension dict
* Update six version to 1.12.0 and fix legacy_create_init issue (#6391)
JavaScript
* Remove deprecated boolean option to getResultBase64String().
* Fix sint64 zig-zag encoding.
* Simplify hash64 string conversion to avoid DIGIT array. Should reduce overhead if these functions aren't used, and be more efficient by avoiding linear array searches.
* Change the parameter types of binaryReaderFn in ExtensionFieldBinaryInfo to (number, ?, ?).
* Create dates.ts and time_of_days.ts to mirror Java versions. This is a near-identical conversion of c.g.type.util.{Dates,TimeOfDays} respectively.
* Migrate moneys to TypeScript.
PHP
* Fix incorrect leap day for Timestamp (#6696)
* Initialize well known type values (#6713)
Ruby
* Fix scope resolution for Google namespace (#5878)
* Support hashes for struct initializers (#5716)
* Optimized away the creation of empty string objects. (#6502)
* Roll forward Ruby upb changes now that protobuf Ruby build is fixed (#5866)
* Optimized layout_mark() for Ruby (#6521)
* Optimization for layout_init() (#6547)
* Fix for GC of Ruby map frames. (#6533)
* Fixed leap year handling by reworking upb_mktime() -> upb_timegm(). (#6695)
Objective C
* Remove OSReadLittle* due to alignment requirements (#6678)
* Don't use unions and instead use memcpy for the type swaps. (#6672)
Other
* Override CocoaPods module to lowercase (#6464)
2019-06-28 version 3.9.0 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
C++
* Optimize and simplify implementation of RepeatedPtrFieldBase
* Don't create unnecessary unknown field sets.
* Remove branch from accessors to repeated field element array.
* Added delimited parse and serialize util.
* Reduce size by not emitting constants for fieldnumbers
* Fix a bug when comparing finite and infinite field values with explicit tolerances.
* TextFormat::Parser should use a custom Finder to look up extensions by number if one is provided.
* Add MessageLite::Utf8DebugString() to make MessageLite more compatible with Message.
* Fail fast for better performance in DescriptorPool::FindExtensionByNumber() if descriptor has no defined extensions.
* Adding the file name to help debug colliding extensions
* Added FieldDescriptor::PrintableNameForExtension() and DescriptorPool::FindExtensionByPrintableName().
The latter will replace Reflection::FindKnownExtensionByName().
* Replace NULL with nullptr
* Created a new Add method in repeated field that allows adding a range of elements all at once.
* Enabled enum name-to-value mapping functions for C++ lite
* Avoid dynamic initialization in descriptor.proto generated code
* Move stream functions to MessageLite from Message.
* Move all zero_copy_stream functionality to io_lite.
* Do not create array of matched fields for simple repeated fields
* Enabling silent mode by default to reduce make compilation noise. (#6237)
Java
* Expose TextFormat.Printer and make it configurable. Deprecate the static methods.
* Library for constructing google.protobuf.Struct and google.protobuf.Value
* Make OneofDescriptor extend GenericDescriptor.
* Expose streamingness of service methods from MethodDescriptor.
* Fix a bug where TextFormat fails to parse Any filed with > 1 embedded message sub-fields.
* Establish consistent JsonFormat behavior for nulls in oneofs, regardless of order.
* Update GSON version to 3.8.5. (#6268)
* Add `protobuf_java_lite` Bazel target. (#6177)
Python
* Change implementation of Name() for enums that allow aliases in proto2 in Python
to be in line with claims in C++ implementation (to return first value).
* Explicitly say what field cannot be set when the new value fails a type check.
* Duplicate register in descriptor pool will raise errors
* Add __slots__ to all well_known_types classes, custom attributes are not allowed anymore.
* text_format only present 8 valid digits for float fields by default
JavaScript
* Add Oneof enum to the list of goog.provide
PHP
* Make php message class final to avoid mocking. (#6277)
* Rename get/setXXXValue to get/setXXXWrapper. (#6295)
Ruby
* Remove to_hash methods. (#6166)
2019-04-29 version 3.8.0 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
C++
* Use std::atomic<int32> in case of myriad2 platform
* Always declare enums to be int-sized
* Added DebugString() and ShortDebugString() methods on MessageLite
* Specialized different parse loop control flows
* Make hasbits potentially in register. The or's start forming an obstacle because it's a read modify store on the same mem address on each iteration.
* Move to an internal MACRO for parser validity checks.
* Improve map parsing performance.
* Make MergePartialFromCodedStream non virtual. This allows direct calls, potential inlining and is also a code health improvement
* Add an overall limit to parse_context to prevent reading past it. This allows to remove a annoying level of indirection.
* Fix a mistake, we shouldn't verify map key/value strings for utf8 in opt mode for proto2.
* Further improvements to cut binary size.
* Prepare to make MergePartialFromCodedStream non-virtual.
* A report on some interesting behavior change in python (caused by b/27494216) made me realize there is a check that needs to be done in case the parse ended on a end group tag.
* Add a note of caution to the comments around skip in CodedOutputStream.
* Simplify end check.
* Add overload for ParseMessage for MessageLite/Message types. If the explicit type is not known inlining won't help de-virtualizing the virtual call.
* Reduce linker input. It turns out that ParseMessage is not inlined, producing template instantiations that are used only once and save nothing but cost more.
* Improve the parser.
* [c++17] Changed proto2::RepeatedPtrField iterators to no longer derive from the deprecated std::iterator class.
* Change the default value of case_insensitive_enum_parsing to false for JsonStringToMessage.
* Add a warning if a field name doesn't match the style guide.
* Fix TextFormat not round-trip correctly when float value is max float.
* Added locationed info for some errors at compiler
* Python reserved keywords are now working with getattr()/setattr() for most descriptors.
* Added AllowUnknownField() in text_format
* Append '_' to C++ reserved keywords for message, enum, extension
* Fix MSVC warning C4244 in protobuf's parse_context.h.
* Updating Iterators to be compatible with C++17 in MSVC.
* Use capability annotation in mutex.h
* Fix "UndefinedBehaviorSanitizer: cfi-bad-type"
* CriticalSectionLock class as a lightweight replacement for std::mutex on Windows platforms.
* Removed vestigial wire_format_lite_inl.h
C#
* Added System.Memory dependency.
Java
* Make Java protoc code generator ignore optimize_for LITE_RUNTIME. Users should instead use the Java lite protoc plugin.
* Change Extension getMessageDefaultInstance() to return Message instead of MessageLite.
* Prevent malicious input streams from leaking buffers for ByteString or ByteBuffer parsing.
* Release new Javalite runtime.
* Show warning in case potential file name conflict.
* Allow Java reserved keywords to be used in extensions.
* Added setAllowUnknownFields() in text format
* Add memoization to ExtensionRegistryLite.getEmptyRegistry()
* Improve performance of CodedOutputStream.writeUInt32NoTag
* Add an optimized mismatch-finding algorithm to UnsafeUtil.
* When serializing uint32 varints, check that we have MAX_VARINT32_SIZE bytes left, not just MAX_VARINT_SIZE.
* Minor optimization to RopeByteString.PieceIterator
JavaScript
* Simplify generated toObject code when the default value is used.
Python
* Changes implementation of Name() for enums that allow aliases in proto2 in Python to be in line with claims in C++ implementation (to return first value).
* Added double_format option in text format printer.
* Added iter and __contains__ to extension dict
* Added allow_unknown_field option in python text format parser
* Fixed Timestamp.ToDatetime() loses precision issue
* Support unknown field in text format printer.
* Float field will be convert to inf if bigger than struct.unpack('f', b'\xff\xff\x7f\x7f')[0] which is about 3.4028234664e+38,
convert to -inf if smaller than -3.4028234664e+38
* Allowed casting str->bytes in Message.__setstate__
Ruby
* Helper methods to get enum name for Ruby.
2019-01-24 version 3.7.0 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
C++
* Introduced new MOMI (maybe-outside-memory-interval) parser.
* Add an option to json_util to parse enum as case-insensitive. In the future, enum parsing in json_util will become case-sensitive.
* Added conformance test for enum aliases
* Added support for --cpp_out=speed:...
* Added use of C++ override keyword where appropriate
* Many other cleanups and fixes.
Java
* Fix illegal reflective access warning in JDK 9+
* Add BOM
Python
* Added Python 3.7 compatibility.
* Modified ParseFromString to return bytes parsed .
* Introduce Proto C API.
* FindFileContainingSymbol in descriptor pool is now able to find field and enum values.
* reflection.MakeClass() and reflection.ParseMessage() are deprecated.
* Added DescriptorPool.FindMethodByName() method in pure python (c extension already has it)
* Flipped proto3 to preserve unknown fields by default.
* Added support for memoryview in python3 proto message parsing.
* Added MergeFrom for repeated scalar fields in c extension (pure python already has it)
* Surrogates are now rejected at setters in python3.
* Added public unknown field API.
* RecursionLimit is also set to max if allow_oversize_protos is enabled.
* Disallow duplicate scalars in proto3 text_format parse.
* Fix some segment faults for c extension map field.
PHP
* Most issues for json encoding/decoding in the c extension have been fixed. There are still some edge cases not fixed. For more details, check conformance/failure_list_php_c.txt.
* Supports php 7.3
* Added helper methods to convert between enum values and names.
* Allow setting/getting wrapper message fields using primitive values.
* Various bug fixes.
Ruby
* Ruby 2.6 support.
* Drops support for ruby < 2.3.
* Most issues for json encoding/decoding in the c extension have been fixed. There are still some edge cases not fixed. For more details, check conformance/failure_list_ruby.txt.
* Json parsing can specify an option to ignore unknown fields: msg.decode_json(data, {ignore_unknown_fields: true}).
* Added support for proto2 syntax (partially).
* Various bug fixes.
Csharp
* More support for FieldMask include merge, intersect and more.
* Increasing the default recursion limit to 100.
* Support loading FileDescriptors dynamically.
* Provide access to comments from descriptors.
* Added Any.Is method.
* Compatible with C# 6
* Added IComparable and comparison operators on Timestamp.
Objective C
* Add ability to introspect list of enum values (#4678)
* Copy the value when setting message/data fields (#5215)
* Support suppressing the objc package prefix checks on a list of files (#5309)
* More complete keyword and NSObject method (via categories) checks for field names, can result in more fields being rename, but avoids the collisions at runtime (#5289)
* Small fixes to TextFormat generation for extensions (#5362)
* Provide more details/context in deprecation messages (#5412)
* Array/Dictionary enumeration blocks NS_NOESCAPE annotation for Swift (#5421)
* Properly annotate extensions for ARC when their names imply behaviors (#5427)
* Enum alias name collision improvements (#5480)
2018-07-27 version 3.6.1 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
C++
* Introduced workaround for Windows issue with std::atomic and std::once_flag
initialization (#4777, #4773).
PHP
* Added compatibility with PHP 7.3 (#4898).
Ruby
* Fixed Ruby crash involving Any encoding (#4718).
2018-06-01 version 3.6.0 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
C++
* Starting from this release, we now require C++11. For those we cannot yet
upgrade to C++11, we will try to keep the 3.5.x branch updated with
critical bug fixes only. If you have any concerns about this, please
comment on issue #2780.
* Moved to C++11 types like std::atomic and std::unique_ptr and away from our
old custom-built equivalents.
* Added support for repeated message fields in lite protos using implicit
weak fields. This is an experimental feature that allows the linker to
strip out more unused messages than previously was possible.
* Fixed SourceCodeInfo for interpreted options and extension range options.
* Fixed always_print_enums_as_ints option for JSON serialization.
* Added support for ignoring unknown enum values when parsing JSON.
* Create std::string in Arena memory.
* Fixed ValidateDateTime to correctly check the day.
* Fixed bug in ZeroCopyStreamByteSink.
* Various other cleanups and fixes.
Java
* Dropped support for Java 6.
* Added a UTF-8 decoder that uses Unsafe to directly decode a byte buffer.
* Added deprecation annotations to generated code for deprecated oneof
fields.
* Fixed map field serialization in DynamicMessage.
* Cleanup and documentation for Java Lite runtime.
* Various other fixes and cleanups
* Fixed unboxed arraylists to handle an edge case
* Improved performance for copying between unboxed arraylists
* Fixed lite protobuf to avoid Java compiler warnings
* Improved test coverage for lite runtime
* Performance improvements for lite runtime
Python
* Fixed bytes/string map key incompatibility between C++ and pure-Python
implementations (issue #4029)
* Added __init__.py files to compiler and util subpackages
* Use /MT for all Windows versions
* Fixed an issue affecting the Python-C++ implementation when used with
Cython (issue #2896)
* Various text format fixes
* Various fixes to resolve behavior differences between the pure-Python and
Python-C++ implementations
PHP
* Added php_metadata_namespace to control the file path of generated metadata
file.
* Changed generated classes of nested message/enum. E.g., Foo.Bar, which
previously generates Foo_Bar, now generates Foo/Bar
* Added array constructor. When creating a message, users can pass a php
array whose content is field name to value pairs into constructor. The
created message will be initialized according to the array. Note that
message field should use a message value instead of a sub-array.
* Various bug fixes.
Objective-C
* We removed some helper class methods from GPBDictionary to shrink the size
of the library, the functionary is still there, but you may need to do some
specific +alloc / -init… methods instead.
* Minor improvements in the performance of object field getters/setters by
avoiding some memory management overhead.
* Fix a memory leak during the raising of some errors.
* Make header importing completely order independent.
* Small code improvements for things the undefined behaviors compiler option
was flagging.
Ruby
* Added ruby_package file option to control the module of generated class.
* Various bug fixes.
Javascript
* Allow setting string to int64 field.
Csharp
* Unknown fields are now parsed and then sent back on the wire. They can be
discarded at parse time via a CodedInputStream option.
* Movement towards working with .NET 3.5 and Unity
* Expression trees are no longer used
* AOT generics issues in Unity/il2cpp have a workaround (see this commit for
details)
* Floating point values are now compared bitwise (affects NaN value
comparisons)
* The default size limit when parsing is now 2GB rather than 64MB
* MessageParser now supports parsing from a slice of a byte array
* JSON list parsing now accepts null values where the underlying proto
representation does
2017-12-20 version 3.5.1 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript)
Planned Future Changes
* Make C++ implementation C++11 only: we plan to require C++11 to build
protobuf code starting from 3.6.0 release. Please join this github issue:
https://github.com/protocolbuffers/protobuf/issues/2780 to provide your feedback.
protoc
* Fixed a bug introduced in 3.5.0 and protoc in Windows now accepts non-ascii
characters in paths again.
C++