Proposal: Schema Validation #331
Conversation
pivotaljohn
force-pushed
the
004-schema-validation
branch
from
c53cb45
to
3c3ebb5
Compare
pivotaljohn
force-pushed
the
004-schema-validation
branch
5 times, most recently
from
09c7750
to
67fdd73
Compare
pivotaljohn
force-pushed
the
004-schema-validation
branch
7 times, most recently
from
d7876e5
to
8b9479e
Compare
pivotaljohn
force-pushed
the
004-schema-validation
branch
from
8b9479e
to
e8d30b1
Compare
| - exclusiveMaximum (is this useful?) | ||
| - exclusiveMinimum (is this useful?) | ||
| - [multipleOf](https://datatracker.ietf.org/doc/html/draft-wright-json-schema-validation-00#section-5.1) | ||
| - pattern (This string SHOULD be a valid regular expression, according to the ECMA 262 regular - expression dialect) |
There was a problem hiding this comment.
@cppforlife noted:
ECMA 262 != RE2
Q: what approach have others who have implemented this feature (specifically JSON Schema validation keyword pattern) in Go done?
Following discussions with [gcheadle-vmware](carvel-dev/ytt#604 (comment)) and @cppforlife, it became clear that @assert/validate ought to always be consumed in a step after overlays are applied for the current pipeline. When writing schema, the author intends not for the schema document _itself_ to be validated, but to validate the _produced_ document (of which the schema is a basis for). Authors need a way to declare the intention for a validation to be attached to the type composite: @schema/validation.
The former is a predicate (expected to return True or False), the later is an assertion.
pivotaljohn
force-pushed
the
004-schema-validation
branch
from
744ed61
to
2536c27
Compare
It seems quite likely that we'll need such a facility for more advanced cases. Placed in the "Other approaches considered" means that the ideas are found without sufficient merit; not the case here.
There was a problem hiding this comment.
I read through the proposal and it looks very well thought out, AWESOME JOB @pivotaljohn
Added some comments and questions in some of the points.
| ```python | ||
| lib.eval(output_disable_validation=True) | ||
| ``` | ||
|
|
There was a problem hiding this comment.
@vrabbi asks:
Any thought on allowing to skip a specific violation?
Not sure what the syntax would be but i can see cases where when consuming third party packages i may want to disable a specific validation that isnt valid in my case.
I don't think its a very common case but it would allow for a break glass solution that is still more secure then disabling all validations.
There was a problem hiding this comment.
At first blush, this starts to make more compelling the need for the ability for overlays to modify metadata in addition to structure (i.e. edit annotations, not just the YAML structure).... 🤔
Curious if others have thoughts about this:
- can you name specific use cases where this would occur (or speculate as to how frequent this would be)?
- ideas on a syntax for doing this via the command-line?
(from conversation with @vrabbi; that you Scott!)
There was a problem hiding this comment.
I like the proposal, and looking forward to be able to play with an initial implementation to see if all the use-cases we have can be satisfied. It's difficult to think whether all could work by just reading this proposal. But so happy to see this coming through.
| #@ end | ||
|
|
||
| #@data/values-schema | ||
| #@schema/validation ("There should be exactly one (1) registry replica if Helm Charts are stored on the filesystem.", one_registry_if_pvc_is_filesystem) |
There was a problem hiding this comment.
I think there's going to be a common pattern for this "top-node" functions where sub-nodes on the schema will be validated (or take the value from/if) a top level node is defined. An example is what CNR/TAP is defining as TLK (Top Level Keys).
There was a problem hiding this comment.
Interesting...
We should talk more about what you see, here... cooking up some examples might be useful/insightful.
| ```yaml | ||
| #@data/values-schema | ||
| --- | ||
| #@schema/validation ("", valid_service_config), when=lambda v: v.enabled |
There was a problem hiding this comment.
Is the lambda v: part required? Could also this be supported?
#@schema/validation ("", valid_service_config), when=this.enabled
service:
enabled: true
Use of keyword this to denote this node (or similar), or fully reference a value key:
#@schema/validation ("", valid_service_config), when=service.enabled
service:
enabled: true
I understand the constraint defined above that data.values are not available at pre-process, but definitely syntax is not very user friendly.
There was a problem hiding this comment.
We need some kind of "callable" that can be invoked in the validation stage: yes.
I like the intention, here.
I think this involves detecting when the argument is not a "callable" and then inject the lambda wrapper...
#@schema/validation ("", valid_service_config), when=this.enabled
is compiled to:
#@schema/validation ("", valid_service_config), when=lambda this: this.enabled
We'll need to make sure that this "auto-wrapping" doesn't accidentally cover-up or make harder-to-debug issues when the expression is invalid for some reason... 🤔
No description provided.