Add kubelet SeccompDefault alpha feature

[saschagrunert]

What type of PR is this?

/kind feature

What this PR does / why we need it:

This adds the gate SeccompDefault as new alpha feature. Seccomp path and field fallbacks are now passed to the helper functions, whereas unit tests covering those code paths have been added as well.

Which issue(s) this PR fixes:

Refers to kubernetes/enhancements#2413

Special notes for your reviewer:

The functions fieldProfile and fieldSeccompProfile usually never receive scmp smp *v1.SeccompProfile == nil. I decided to not cleanup those paths for security reasons.

Beside enabling the feature gate, the feature has to be enabled by the SeccompDefault kubelet configuration or its corresponding --seccomp-default CLI flag.

Another thing to mention is that the feature does not change the API at all. This means that people using the feature will not get a modified SecurityContext if they do not specify anything within it.

Does this PR introduce a user-facing change?

Added new kubelet alpha feature `SeccompDefault`. This feature enables falling back to
the `RuntimeDefault` (former `runtime/default`) seccomp profile if nothing else is specified
in the pod/container `SecurityContext` or the pod annotation level. To use the feature, enable
the feature gate as well as set the kubelet configuration option `SeccompDefault`
(`--seccomp-default`) to `true`.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

- KEP: https://github.com/kubernetes/enhancements/tree/34fa3dd/keps/sig-node/2413-seccomp-by-default

[SergeyKanzhelev]

/triage accepted
/priority important-soon

[saschagrunert]

@mrunalp @pjbgf @liggitt PTAL

[liggitt]

looking at the PRR, it is unclear to me how an operator would detect that enabling this caused an issue

How can a rollout fail? Can it impact already running workloads?

• Workloads on a node may starting to fail when (re)scheduled on the node where the feature is enabled. Required specific syscalls may be blocked by the default seccomp profile, which will cause the application to get terminated.

What specific metrics should inform a rollback?

• If a workload is starting to fail because of blocked syscalls (audit logs), then a temporarily rollback would be appropriate in production.

can you clarify how a cluster operator would know if this was causing problems?

[saschagrunert]

can you clarify how a cluster operator would know if this was causing problems?

Pods which previously run successfully would now get some syscalls blocked they either need for startup of the workload or a single feature. This means that in the worst case the pod gets stuck in a crash loop or encounter degraded functionality. So a cluster operator would see restarting containers, while the logs would indicate something like "operation not permitted".

A good pre-check would be to run an e2e test suite for the application with the runtime default profile if the application was running unconfined before.

[liggitt]

Pods which previously run successfully would now get some syscalls blocked they either need for startup of the workload or a single feature. This means that in the worst case the pod gets stuck in a crash loop or encounter degraded functionality. So a cluster operator would see restarting containers, while the logs would indicate something like "operation not permitted".

Detecting when a crash is due to this requires application-specific knowledge, right? I don't see a way for a cluster operator to detect this generically.

A good pre-check would be to run an e2e test suite for the application with the runtime default profile if the application was running unconfined before.

That is a good idea for the application owner, but the application owner and cluster operator are often two different entities.

[saschagrunert]

Pods which previously run successfully would now get some syscalls blocked they either need for startup of the workload or a single feature. This means that in the worst case the pod gets stuck in a crash loop or encounter degraded functionality. So a cluster operator would see restarting containers, while the logs would indicate something like "operation not permitted".

Detecting when a crash is due to this requires application-specific knowledge, right? I don't see a way for a cluster operator to detect this generically.

The only way to get a hint to it is when toggling the feature toggles crashing pods on a node.

A good pre-check would be to run an e2e test suite for the application with the runtime default profile if the application was running unconfined before.

That is a good idea for the application owner, but the application owner and cluster operator are often two different entities.

Agreed, mixed up the roles in my mind.

[saschagrunert]

Ah, but I see @johnbelamaric already covered this in a comment:

I don't think it's a good idea to release a feature that will break workloads on upgrade. We should require a flag to enable this even in GA.

Does that reflect the current plan, and is it expected the KEP will be updated?

We definitely have to update the KEP reflecting the flag. I think together with that I can also add a rollout plan. 🙂

[saschagrunert]

KEP update PR in kubernetes/enhancements#2773

[saschagrunert]

The KEP update PR got merged, which means this is now ready for review. @liggitt @kubernetes/sig-node-pr-reviews please take a look again

[liggitt]

config API and cli bits lgtm (one nit on documenting the feature gate requirement)

[liggitt]

tag me once the functional bits of this have approval and I can add approve for the config API

[saschagrunert]

Pinging for approval:

• cmd/kubelet/OWNERS: @mrunalp @sjenning
• pkg/features/OWNERS: @dchen1107 @derekwaynecarr
• pkg/kubelet/OWNERS: @mrunalp @sjenning

[saschagrunert]

/test pull-kubernetes-e2e-gce-ubuntu-containerd

[mrunalp]

/lgtm

[liggitt]

/approve
for API/CLI changes
based on @mrunalp lgtm (feature gate approved by @derekwaynecarr in the KEP)

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: liggitt, mrunalp, saschagrunert

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• cmd/kubelet/OWNERS [liggitt,mrunalp]
• pkg/features/OWNERS [liggitt]
• pkg/kubelet/OWNERS [liggitt,mrunalp]
• pkg/kubelet/apis/config/OWNERS [liggitt]
• staging/src/k8s.io/kubelet/config/OWNERS [liggitt]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[saschagrunert]

/test pull-kubernetes-e2e-gce-ubuntu-containerd