Promote feature CustomResourceValidationExpressions to beta

[cici37]

What type of PR is this?

/kind feature

What this PR does / why we need it:

This PR is to promote feature CustomResourceValidationExpressions to beta as it's proposed in KEP.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

The feature is in alpha since v1.23. For v1.25 cycle, we focus on performance optimization, test coverage, bug fix, metrics adding, dependency bumping, etc. The related PR FYI:

• Optimize away repeated structural schema and CEL DeclType construction in CRD validation #110135
• Pick up major changes from cel-go latest release #111071
• metrics for CEL compilation and evaluation #111035
• Bump cel-go to v0.11.2 #109804
• Add x-kubernetes-validations cost limit integration test #111005
• Add second set of additional CRD validation E2E tests #110576
• Only provide an oldSelf binding when referenced by a CEL rule. #110549
• Do test fixture setup outside cel.UnstructuredToVal benchmark loop. #110548
• Add additional CRD validation E2E tests #110330
• Use CEL EagerlyValidateDeclarations to improve compilation performance #109835
• Bump cel-go to v0.12.3 #111008
• Benchmark cel.UnstructuredToVal #108011

Graduation to beta criteria from the KEP

• Resolve topic of what support we should provide for access to the previous versions of object (ie. 'oldSelf' feature) -- resolved in

  kubernetes/staging/src/k8s.io/apiextensions-apiserver/pkg/apiserver/schema/cel/validation_test.go

  Lines 1701 to 1753 in 631a5a8

  	{name: "transition rules",
  	obj: map[string]interface{}{
  	"v": "new",
  	},
  	oldObj: map[string]interface{}{
  	"v": "old",
  	},
  	schema: objectTypePtr(map[string]schema.Structural{
  	"v": stringType,
  	}),
  	valid: []string{
  	"oldSelf.v != self.v",
  	"oldSelf.v == 'old' && self.v == 'new'",
  	},
  	},
  	{name: "skipped transition rule for nil old primitive",
  	expectSkipped: true,
  	obj: "exists",
  	oldObj: nil,
  	schema: &stringType,
  	valid: []string{
  	"oldSelf == self",
  	},
  	},
  	{name: "skipped transition rule for nil old array",
  	expectSkipped: true,
  	obj: []interface{}{},
  	oldObj: nil,
  	schema: listTypePtr(&stringType),
  	valid: []string{
  	"oldSelf == self",
  	},
  	},
  	{name: "skipped transition rule for nil old object",
  	expectSkipped: true,
  	obj: map[string]interface{}{"f": "exists"},
  	oldObj: nil,
  	schema: objectTypePtr(map[string]schema.Structural{
  	"f": stringType,
  	}),
  	valid: []string{
  	"oldSelf.f == self.f",
  	},
  	},
  	{name: "skipped transition rule for old with non-nil interface but nil value",
  	expectSkipped: true,
  	obj: []interface{}{},
  	oldObj: nilInterfaceOfStringSlice(),
  	schema: listTypePtr(&stringType),
  	valid: []string{
  	"oldSelf == self",
  	},
  	},

• x-kubernetes-int-or-string is upgraded to use a union type of just int or string, not a dynamic type (CEL go support is planned in lates 2021) -- resolved in

  kubernetes/staging/src/k8s.io/apiextensions-apiserver/pkg/apiserver/schema/cel/validation_test.go

  Lines 793 to 842 in 631a5a8

  	{name: "string in intOrString",
  	obj: map[string]interface{}{
  	"something": "25%",
  	},
  	schema: objectTypePtr(map[string]schema.Structural{
  	"something": intOrStringType(),
  	}),
  	valid: []string{
  	// In Kubernetes 1.24 and later, the CEL type returns false for an int-or-string comparison against the
  	// other type, making it safe to write validation rules like:
  	"self.something == '25%'",
  	"self.something != 1",
  	"self.something == 1 || self.something == '25%'",
  	"self.something == '25%' || self.something == 1",
  	// In Kubernetes 1.23 and earlier, all int-or-string access must be guarded by a type check to prevent
  	// a runtime error attempting an equality check between string and int types.
  	"type(self.something) == string && self.something == '25%'",
  	"type(self.something) == int ? self.something == 1 : self.something == '25%'",
  	// Because the type is dynamic it receives no type checking, and evaluates to false when compared to
  	// other types at runtime.
  	"self.something != ['anything']",
  	},
  	},
  	{name: "int in intOrString",
  	obj: map[string]interface{}{
  	"something": int64(1),
  	},
  	schema: objectTypePtr(map[string]schema.Structural{
  	"something": intOrStringType(),
  	}),
  	valid: []string{
  	// In Kubernetes 1.24 and later, the CEL type returns false for an int-or-string comparison against the
  	// other type, making it safe to write validation rules like:
  	"self.something == 1",
  	"self.something != 'some string'",
  	"self.something == 1 || self.something == '25%'",
  	"self.something == '25%' || self.something == 1",
  	// In Kubernetes 1.23 and earlier, all int-or-string access must be guarded by a type check to prevent
  	// a runtime error attempting an equality check between string and int types.
  	"type(self.something) == int && self.something == 1",
  	"type(self.something) == int ? self.something == 1 : self.something == '25%'",
  	// Because the type is dynamic it receives no type checking, and evaluates to false when compared to
  	// other types at runtime.
  	"self.something != ['anything']",
  	},
  	},

• Understanding of upper bounds of CPU/memory usage and appropriate limits set to prevent abuse.(Done in Add CEL runtime cost into CR validation #108482; Update CEL runtime cost limit #108595; Calculate CEL cost totals #108612; Reduce CEL runtime cost limits by 1/2 based on latency goals #109122)
• Build-in macro/function library is comprehensive and stable (any changes to this will be a breaking change) - The later addition of quantity appears to complicate this. See CEL Validation Use Cases where it is postponed as a use-case (Add CEL function library #108312)
• CEL numeric comparison issue is resolved (e.g. ability to compare ints to doubles) (Added test here)
• Reduce noise of invalid data messages reported from cel.UnstructuredToVal #106440 Asking for clarification of the fix
• Benchmark cel.UnstructuredToVal and optimize away repeated wrapper object construction #106438
• Demonstrate adoption and successful feature usage in the community -- confirmed against the list in CEL Validation Use Cases. quantity to be rejected by supported releases when it is added.
• Optimization on super-linear complexity growth (Optimize away repeated structural schema and CEL DeclType construction in CRD validation #110135)
• Adding metric of the latency of CEL evaluation for CRD evaluation

Does this PR introduce a user-facing change?

Graduated `CustomResourceValidationExpressions` to `beta`. The `CustomResourceValidationExpressions` feature gate is now enabled by default.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

- [KEP]: https://github.com/kubernetes/enhancements/blob/master/keps/sig-api-machinery/2876-crd-validation-expression-language/README.md

[k8s-ci-robot]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[cici37]

/sig api-machinery

[cici37]

cc @jpbetz @liggitt

[jiahuif]

Don't forget to update pkg/features/kube_features.go too

[jiahuif]

/cc

[liggitt]

updated the description to a bulleted list of the PRs, so we can see they all merged before merging this one

[cici37]

The e2e test associated with this feature will be handled in separate PR: #111448

[deads2k]

I'm diffing against the list in https://github.com/kubernetes/enhancements/blob/master/keps/sig-api-machinery/2876-crd-validation-expression-language/README.md#beta

I'm assessing "Demonstrate adoption and successful feature usage in the community" based on use-cases contained in https://docs.google.com/document/d/1KatwZJ3O0lTThs1Lg8YqY-PipbNM8aaUDDdPCLd3lic/edit?resourcekey=0-YeuQhCZlstVjL61WVqrz_w# that have specific reference and relating them back to the unit tests that ensure they work.

I'll also come back to the rest of the criteria and check them as well.

[deads2k]

note to self, int-or-string evaluation is confirmed here:

kubernetes/staging/src/k8s.io/apiextensions-apiserver/pkg/apiserver/schema/cel/validation_test.go

Lines 793 to 842 in 631a5a8

	{name: "string in intOrString",
	obj: map[string]interface{}{
	"something": "25%",
	},
	schema: objectTypePtr(map[string]schema.Structural{
	"something": intOrStringType(),
	}),
	valid: []string{
	// In Kubernetes 1.24 and later, the CEL type returns false for an int-or-string comparison against the
	// other type, making it safe to write validation rules like:
	"self.something == '25%'",
	"self.something != 1",
	"self.something == 1 || self.something == '25%'",
	"self.something == '25%' || self.something == 1",
	// In Kubernetes 1.23 and earlier, all int-or-string access must be guarded by a type check to prevent
	// a runtime error attempting an equality check between string and int types.
	"type(self.something) == string && self.something == '25%'",
	"type(self.something) == int ? self.something == 1 : self.something == '25%'",
	// Because the type is dynamic it receives no type checking, and evaluates to false when compared to
	// other types at runtime.
	"self.something != ['anything']",
	},
	},
	{name: "int in intOrString",
	obj: map[string]interface{}{
	"something": int64(1),
	},
	schema: objectTypePtr(map[string]schema.Structural{
	"something": intOrStringType(),
	}),
	valid: []string{
	// In Kubernetes 1.24 and later, the CEL type returns false for an int-or-string comparison against the
	// other type, making it safe to write validation rules like:
	"self.something == 1",
	"self.something != 'some string'",
	"self.something == 1 || self.something == '25%'",
	"self.something == '25%' || self.something == 1",
	// In Kubernetes 1.23 and earlier, all int-or-string access must be guarded by a type check to prevent
	// a runtime error attempting an equality check between string and int types.
	"type(self.something) == int && self.something == 1",
	"type(self.something) == int ? self.something == 1 : self.something == '25%'",
	// Because the type is dynamic it receives no type checking, and evaluates to false when compared to
	// other types at runtime.
	"self.something != ['anything']",
	},
	},

[deads2k]

/lgtm
/approve
/hold

holding for the comment update. Feel free to re-lgtm and unhold after that.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cici37, deads2k

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• pkg/features/OWNERS [deads2k]
• staging/src/k8s.io/apiserver/pkg/features/OWNERS [deads2k]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

New changes are detected. LGTM label has been removed.

[k8s-ci-robot]

@cici37: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-kubernetes-integration	64e6cf1	link	true	/test pull-kubernetes-integration

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[jpbetz]

The pull-kubernetes-integration failures for k8s.io/kubernetes/test/integration/apiserver/apply: TestApplyCRDUnhandledSchema are from #110135. Looking into it.

[jpbetz]

Fix for TestApplyCRDUnhandledSchema failure: #111483

[cici37]

Proposed a fix in #111504. Will hold current PR until either of them merged.

Fix for TestApplyCRDUnhandledSchema failure: #111483

[cici37]

This PR got accidentally closed and cannot be reopen due to a force-push. Open #111524 as a duplication