[Bug] [CLI] Regression Verbosity parameter has no effect for versions above 1.10.6 when running kyverno test

[StanYago]

Kyverno CLI Version

1.11.0

Description

Up to version 1.10.6 kyverno test supported -v, --v Level option which at level 3 and above would produce additional debug output. There is no debug output generated in kyverno 1.11.x and 1.12.x

Steps to reproduce

kyverno-1.10.6/kyverno test src/policies/other/require-pdb/.kyverno-test -v3

Executing ...
I0518 13:59:22.394851 44501 common.go:289] "msg"="Defaulting request.operation to CREATE"
I0518 13:59:22.395489 44501 common.go:202] "msg"="read policies" "errors"=0 "policies"=1
applying 1 policy to 2 resources...
I0518 13:59:22.397478 44501 vars.go:376] "msg"="variable substituted" "path"="/validate/deny/conditions/any/0/key" "value"=null "variable"="{{pdb_count}}"
I0518 13:59:22.397669 44501 vars.go:376] "msg"="variable substituted" "path"="/context/0/apiCall/jmesPath" "value"=null "variable"="{{request.object.spec.template.metadata.labels}}"
I0518 13:59:22.397714 44501 vars.go:376] "msg"="variable substituted" "path"="/context/0/apiCall/urlPath" "value"=null "variable"="{{request.namespace}}"
I0518 13:59:22.397950 44501 vars.go:376] "msg"="variable substituted" "path"="/preconditions/all/0/key" "value"=null "variable"="{{request.operation || 'BACKGROUND'}}"
I0518 13:59:22.397992 44501 vars.go:376] "msg"="variable substituted" "path"="/preconditions/all/1/key" "value"=null "variable"="{{ request.object.spec.replicas }}"
I0518 13:59:22.398295 44501 common.go:425] "msg"="applying policy on resource" "policy"="require-pdb" "resource"="default/Deployment/nginx-deploy-pass"
I0518 13:59:22.398447 44501 infos.go:54] image "msg"="getting the image info" "defaultRegistry"="docker.io" "enableDefaultRegistryMutation"=true "image"="nginx:1.14.2"
I0518 13:59:22.398463 44501 infos.go:89] image "msg"="getting the image info" "digest"="" "image"="nginx:1.14.2" "name"="nginx" "path"="nginx" "registry"="docker.io" "tag"="1.14.2"
I0518 13:59:22.398617 44501 contextloaderfactory.go:108] DefaultContextLoaderFactory "msg"="disabled loading of APICall context entry %s" "pdb_count"="(MISSING)"
I0518 13:59:22.398695 44501 vars.go:376] engine.validate "msg"="variable substituted" "new.kind"="Deployment" "new.name"="nginx-deploy-pass" "new.namespace"="default" "path"="" "policy.apply"="All" "policy.name"="require-pdb" "policy.namespace"="" "rule.name"="require-pdb" "value"="CREATE" "variable"="{{request.operation || 'BACKGROUND'}}"
I0518 13:59:22.398785 44501 vars.go:376] engine.validate "msg"="variable substituted" "new.kind"="Deployment" "new.name"="nginx-deploy-pass" "new.namespace"="default" "path"="" "policy.apply"="All" "policy.name"="require-pdb" "policy.namespace"="" "rule.name"="require-pdb" "value"=4 "variable"="{{ request.object.spec.replicas }}"
I0518 13:59:22.398893 44501 vars.go:376] engine.validate "msg"="variable substituted" "new.kind"="Deployment" "new.name"="nginx-deploy-pass" "new.namespace"="default" "path"="" "policy.apply"="All" "policy.name"="require-pdb" "policy.namespace"="" "rule.name"="require-pdb" "value"="1" "variable"="{{pdb_count}}"
I0518 13:59:22.398908 44501 evaluate.go:79] engine.validate "msg"="no condition passed for 'any' block" "any"=[{"key":"{{pdb_count}}","operator":"LessThan","value":1}] "new.kind"="Deployment" "new.name"="nginx-deploy-pass" "new.namespace"="default" "policy.apply"="All" "policy.name"="require-pdb" "policy.namespace"="" "rule.name"="require-pdb"
I0518 13:59:22.399182 44501 common.go:425] "msg"="applying policy on resource" "policy"="require-pdb" "resource"="default/Deployment/nginx-deploy-skip"
I0518 13:59:22.399286 44501 infos.go:54] image "msg"="getting the image info" "defaultRegistry"="docker.io" "enableDefaultRegistryMutation"=true "image"="nginx:1.14.2"
I0518 13:59:22.399301 44501 infos.go:89] image "msg"="getting the image info" "digest"="" "image"="nginx:1.14.2" "name"="nginx" "path"="nginx" "registry"="docker.io" "tag"="1.14.2"
I0518 13:59:22.399397 44501 contextloaderfactory.go:108] DefaultContextLoaderFactory "msg"="disabled loading of APICall context entry %s" "pdb_count"="(MISSING)"
I0518 13:59:22.399464 44501 vars.go:376] engine.validate "msg"="variable substituted" "new.kind"="Deployment" "new.name"="nginx-deploy-skip" "new.namespace"="default" "path"="" "policy.apply"="All" "policy.name"="require-pdb" "policy.namespace"="" "rule.name"="require-pdb" "value"="CREATE" "variable"="{{request.operation || 'BACKGROUND'}}"
I0518 13:59:22.399538 44501 vars.go:376] engine.validate "msg"="variable substituted" "new.kind"="Deployment" "new.name"="nginx-deploy-skip" "new.namespace"="default" "path"="" "policy.apply"="All" "policy.name"="require-pdb" "policy.namespace"="" "rule.name"="require-pdb" "value"=1 "variable"="{{ request.object.spec.replicas }}"
I0518 13:59:22.399766 44501 evaluate.go:90] engine.validate "msg"="a condition failed in 'all' block" "condition"={"key":"{{ request.object.spec.replicas }}","operator":"GreaterThanOrEquals","value":3} "message"="" "new.kind"="Deployment" "new.name"="nginx-deploy-skip" "new.namespace"="default" "policy.apply"="All" "policy.name"="require-pdb" "policy.namespace"="" "rule.name"="require-pdb"

│───│─────────────│─────────────│──────────────────────────────────────│────────│
│ # │ POLICY │ RULE │ RESOURCE │ RESULT │
│───│─────────────│─────────────│──────────────────────────────────────│────────│
│ 1 │ require-pdb │ require-pdb │ default/Deployment/nginx-deploy-pass │ Pass │
│ 2 │ require-pdb │ require-pdb │ default/Deployment/nginx-deploy-skip │ Pass │
│───│─────────────│─────────────│──────────────────────────────────────│────────│

Test Summary: 2 tests passed and 0 tests failed

kyverno-1.11.5/kyverno test src/policies/other/require-pdb/.kyverno-test -v3

Loading test ( src/policies/other/require-pdb/.kyverno-test/kyverno-test.yaml ) ...
Loading values/variables ...
Loading policies ...
Loading resources ...
Applying 1 policy to 2 resources ...
Checking results ...

│────│─────────────│─────────────│──────────────────────────────│────────│────────│
│ ID │ POLICY │ RULE │ RESOURCE │ RESULT │ REASON │
│────│─────────────│─────────────│──────────────────────────────│────────│────────│
│ 1 │ require-pdb │ require-pdb │ Deployment/nginx-deploy-pass │ Pass │ Ok │
│ 2 │ require-pdb │ require-pdb │ Deployment/nginx-deploy-skip │ Pass │ Ok │
│────│─────────────│─────────────│──────────────────────────────│────────│────────│

Test Summary: 2 tests passed and 0 tests failed

Expected behavior

Expect to see debug output with verbosity level set to 3 or above in version 1.11.x and up

Screenshots

No response

Kyverno logs

No response

Slack discussion

No response

Troubleshooting

• I have read and followed the troubleshooting guide.
• I have searched other issues in this repository and mine is not recorded.

[realshuting]

This should be fixed via #10216.

[realshuting]

v1.12.2-rc.1 is available, closing.

[StanYago]

@realshuting Still no debug output available in latest releases v1.12.2 and v1.12.3

[StanYago]

Oh, I see the option is now --v=6 instead of the old -v6

[chipzoller]

It works with either one.

[StanYago]

It works with either one.

Only --v=# works, -v# option does nothing

[chipzoller]

Then that's a mistake. cc @anushkamittal2001 @realshuting