Schema Validation v2

[pivotaljohn]

(this is the next feature increment of #561)

---

As a Configuration Author
I want to declare what constitutes a valid Data Value right in my schema definition
In a way that requires even less custom Starlark code from me ⭐
And I can write fewer annotations ⭐ because ytt can confidently infer for what have written.
So that*

• I can state those invariants in one central place and know they will be respected throughout the execution
• those who use my ytt library and violate one of those variants will receive a helpful error message
• and my overall maintenance effort/costs are less than before. ⭐

(⭐ = a desired improvement on top of the MVP.)

[proposal]
[getting started]

---

Backlog

• ⚠️ This track of work is still being elaborated; some scope is likely missing.⚠️
• loads of ideas for improvements can be mined from User-Test ytt Schema Validation #707's synthesis

UX:

• Use custom violation messages
  • on @assert/validate, implement 3-tuple rules where the third tuple is the violation template; to @schema/validation, add 3-tuple tules.
  • note: this is critical to improve the DX around unit testing (detailed in a conversation on #741)
• disallow @schema/validation in non-schema documents.
• disallow @assert/validate in schema documents.

Fast Follow: other must-have OOTB rules

• matches=

Programmatic Validation of YAML

• Validate a YAML document that contains @assert/validate annotations
  • expose assert.is_valid() as a method in @ytt:assert module

Include Validations in Schema Inspect

• Include validations when exporting schema to OpenAPI v3 (subset: one-for-ones)
  • need: https://kubernetes.slack.com/archives/CH8KCCKA5/p1661186311752459?thread_ts=1661185491.294779&cid=CH8KCCKA5
• Include validations when exporting schema to OpenAPI v3 (subset: needing translation)

Authoring Conveniences:

• Set a default validation for descendent Data Values (@schema/validate-string-default)
• Infer @schema/nullable for all children implicated in a @schema/validation one_not_null=

---

Resources

• Schema proposal: Part 7 — Validating Documents
• Schema Validation proposal
• Previous Epic: Schema Validation (MVP) #561

Related Issues:

• [schema, data/values] support specifying schema for data structures #103 — originating issue
• Add support for data values schema without defaults #556 — requesting "required" use-case, explicitly.