[Feature] Conformance tests for Kyverno upgrade

[realshuting]

Problem Statement

Currently there's no automated test for the Kyverno upgrade. This is critical for Kyverno's stability upon new releases.

All existing conformance tests target a single policy and check its expected result. While in 1.12.0 release we found an issue when applying policies concurrently and Kyverno returned inconsistent results. One of the common use cases to test out is Kyverno PSS policies. Covering Kyverno upgrade by conformance tests, specifically for PSS policies, is a good starting point.

Solution Description

This test was added to verify Kyverno PSS policies, the same test can be extended for the Kyverno upgrade.

https://github.com/kyverno/kyverno/tree/main/test/conformance/chainsaw/validate/clusterpolicy/standard/psa/concurrent-policy-execution

Alternatives

No response

Additional Context

No response

Slack discussion

No response

Research

• I have read and followed the documentation AND the troubleshooting guide.
• I have searched other issues in this repository and mine is not recorded.