-
Notifications
You must be signed in to change notification settings - Fork 555
Protocol Backwards Compatibility
It is possible to add fields to messages, but we have to make sure to follow the guide https://github.com/real-logic/simple-binary-encoding/wiki/Message-Versioning and check manually on every PR as there are not automatic checks.
- Adding new messages and RPC methods is okay
- Updating a message follow the guide: https://developers.google.com/protocol-buffers/docs/proto3#updating
- RPC methods cannot be changed
- We have a maven plugin in place which will check the compatibility of changes to the protocol and fail in case incompatible changes are done
Message Pack does not have a schema, so unknown fields will be ignored. New fields should always have a default value. The code has to be written in a way that it can handle non existing fields. Fields used by old code cannot be removed and have to be set to correct values.
Atomix snapshots contained static information about the primitive which can introduce breaking changes should we change it. The following is a list of non-backwards compatible changes:
As the service configuration is serialized in the snapshot, removing one of the classes that was serialized will cause the snapshot to become unreadable. The workaround here is to phase the class out; keep the class one more version without using it, so snapshots will not contain it any more, then remove it.
Snapshot serialization uses the service's Kryo namespace, which registers classes to IDs (integers); on deserialization it maps the ID back to the concrete class it has to deserialize. For backwards compatibility, we need to keep the IDs the same.
When changing the primitive service's restore/backup implementation, take care to ensure that the new implementation is backwards compatible with the previous one. For example, reordering reads/writes is a breaking change, and so is removing a read/write. On the other hand, strictly adding new reads/writes is backwards compatible, so long as they come after the older ones, and we handle the case where they could be missing.
In general better to avoid this situation at all.
If applying any custom patch from our fork that deal with Raft, take care that they do not break how snapshots would be read; this includes changes to raft session serialization, service serialization, primitive serialization, etc.
As we strict typing in our Elasticsearch templates we cannot add new fields to the indices. If a change is required we have to enforce a new index with an updated template in Elasticsearch. This will prevent a rolling upgrade and probably requires user interaction.
Note: As we now directly export our record implementation, instead of having another abstraction, we cannot change the record values fields if we do not exclude them from the toJson conversion.