ReadWriteOncePod access mode for PVs and PVCs

[chrishenzie]

What type of PR is this?

/kind feature

What this PR does / why we need it:

This PR introduces the ReadWriteOncePod access mode for PersistentVolumes and PersistentVolumeClaims. This access mode restricts volume access to a single pod on a single node. Please see KEP#2485 for more context.

This access mode is enforced in two places:

First is during scheduling (see #103082). When a pod uses a PVC with ReadWriteOncePod, we check for any other consumers of this PVC in the same namespace. If any are found, consider the pod unschedulable and unresolvable since there can only be one consumer. The failing pod will have error:

0/1 nodes are available: 1 node has pod using PersistentVolumeClaim with the same name and ReadWriteOncePod access mode.

Second is during volume mounting (inside of kubelet). When a volume is mounted (or a block device is mapped), we check the actual state of the world cache to check if the provided volume is already mounted to another pod. If it is, fail the mount operation to prevent access from another pod. The failing pod will have error:

MountVolume.SetUp failed for volume "pvc-<ID>" : volume uses the ReadWriteOncePod access mode and is already in use by another pod.

Special notes for your reviewer:

I tried breaking this into individual chunks in each commit. Hopefully it will make for easier review.

Does this PR introduce a user-facing change?

Adds the ReadWriteOncePod access mode for PersistentVolumes and PersistentVolumeClaims. Restricts volume access to a single pod on a single node.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

- [KEP]: https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/2485-read-write-once-pod-pv-access-mode

/sig storage
/sig scheduling
/assign @msau42
/assign @jingxu97

[k8s-ci-robot]

@chrishenzie: This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[fejta-bot]

This PR may require API review.

If so, when the changes are ready, complete the pre-review checklist and request an API review.

Status of requested reviews is tracked in the API Review project.

[alculquicondor]

It looks like the KEP didn't go through a sig-scheduling review.

The question that comes to mind, regarding the k8s changes: why can't users who need this use pod anti-affinity along with ReadWriteOnce?

[chrishenzie]

@alculquicondor Can we retroactively do that? Any feedback is appreciated.

Users may be able to configure pod anti-affinity rules to produce similar behavior, but I feel like that's working around a lack of feature in Kubernetes that should be implemented. The issue is there is a lack of an access mode that guarantees single writer access (on a single node) to a PersistentVolume. From my understanding, most users assume this is what ReadWriteOnce does, but by definition it doesn't. By adding a new access mode to handle this case, we provide opportunities for users to express with clearer intent how they want their PersistentVolumes to be consumed.

[ahg-g]

I'm not a hug fan of the naming (no, I don't have a better one), and I am a bit surprised scheduling wants to take this on, but OK

/approve

With the new scheduling framework, it is fairly easy to add such functionality as a plugin.

[chrishenzie]

/retest

[chrishenzie]

/retest

[chrishenzie]

@jsafrane for lgtm

[jsafrane]

/lgtm
/approve
for storage side

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: chrishenzie, jsafrane, thockin

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• pkg/apis/OWNERS [thockin]
• pkg/controller/volume/persistentvolume/OWNERS [jsafrane,thockin]
• pkg/features/OWNERS [jsafrane,thockin]
• pkg/kubelet/volumemanager/OWNERS [jsafrane,thockin]
• pkg/registry/OWNERS [thockin]
• pkg/volume/OWNERS [jsafrane,thockin]
• staging/src/k8s.io/api/OWNERS [thockin]
• staging/src/k8s.io/kubectl/OWNERS [thockin]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment