More user-friendly JSON Schema compatibility checks

[big-andy-coates]

tl;dr; I'm proposed a better way to perform schema compatibility checks for JSON Schema, that allows the addition and removal of optional properties. See part 2 post for details.

They've been a few issues raised in the past by people being surprised how restrictive the compatibility checks are for JSON schema types:

• JSON-Schema forward compatibility doesn't allow the removal of an optional field. #2121
• Jsonschema Forward compatibility fails when deleting an optional field Apicurio/apicurio-registry#1710
• Unclear Schema Evolution for JSON Schemas #1778
• Unclear evolution of partially open models #2671
• Json Schema full compatibility support #1490
• Compatibility level JSON schema's #1617
• JSON serializer fails compatibility check when optional fields aren't present in object #2234
• Question: Json-schema evolution always not compatible with earlier version #1529
• etc...

Robert wrote a post explaining why it is the way it is and proposed a solution for full compatible schema changes using patternProperties. Unfortunately, using patternProperties is very restrictive and verbose, e.g. here's my stab at such a schema before I added any actual properties:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "Verbose and restrictive partially open content model",
  "$ref": "#/definitions/obj",
  "definitions": {
    "obj": {
      "type": "object",
      "additionalProperties": false,
      "patternProperties": {
       "^i_": { "type": "integer" },
       "^n_": { "type": "number" },
       "^s_": { "type": "string" },
       "^b_": { "type": "boolean" },
       "^o_": { "$ref": "#/definitions/obj"},
       "^ai_": { "type": "array", "items": {"type": "integer"} },
       "^an_": { "type": "array", "items": {"type": "number" } },
       "^as_": { "type": "array", "items": {"type": "string" } },
       "^ab_": { "type": "array", "items": {"type": "boolean" } },
       "^ao_": { "type": "array", "items": {"$ref": "#/definitions/obj"} }
      }
    }
  }
}

Without using a partially-open object model, like the one above, it is simply not possible to achieve full compatibility when making schema changes: there are no valid meaningful schema changes.

Other schema types allow optional properties to be added and removed while maintaining full compatibility.

I believe I've an improved model for JSON schema compatibility checks that does allow optional properties to be added and removed. I've detailed the approach in a post.

Thoughts very much appreciated all, @rayokota

[rayokota]

Thanks @big-andy-coates , I'll take a look!

[big-andy-coates]

Hey @rayokota, any thoughts?