Add DataSourceRef field to PVC spec #103276

bswartz · 2021-06-28T20:02:28Z

What type of PR is this?

/kind feature
/kind api-change

What this PR does / why we need it:

Modify the behavior of the AnyVolumeDataSource alpha feature gate to enable
a new field, DataSourceRef, rather than modifying the behavior of the
existing DataSource field. This allows addition Volume Populators in a way
that doesn't risk breaking backwards compatibility, although it will
result in eventually deprecating the DataSource field.

Which issue(s) this PR fixes:

Fixes #103275

Special notes for your reviewer:

A lot of test cases have been added, but I think more could be added. I'm going to keep working on those which the change itself is reviewed.

Does this PR introduce a user-facing change?

Add DataSourceRef alpha field to PVC spec, which allows contents other than PVCs and VolumeSnapshots to be data sources.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

- [KEP]: https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/1495-volume-populators

fejta-bot · 2021-06-28T21:02:49Z

This PR may require API review.

If so, when the changes are ready, complete the pre-review checklist and request an API review.

Status of requested reviews is tracked in the API Review project.

bswartz · 2021-06-30T17:16:04Z

/assign @thockin

fedebongio · 2021-07-01T20:06:54Z

/remove-sig api-machinery
/sig storage

bswartz · 2021-07-02T15:34:33Z

/assign @saad-ali

pkg/apis/core/types.go

staging/src/k8s.io/api/core/v1/types.go

pkg/registry/core/persistentvolumeclaim/storage/storage.go

pkg/api/persistentvolumeclaim/util.go

thockin

I'll be online off and on today and tomorrow. If you want to make an argument for "the tests work, let's merge it and then fix this" I am open to it, but I really REALLY think it should per made more permanent, since it is now part of the defined semantics.

pkg/api/persistentvolumeclaim/util.go

bswartz · 2021-07-05T19:01:47Z

@thockin Thanks I've been working on refactoring this code and adding more unit tests today. I'll send up another version trying to incorporate all your suggestions. The one tricky one will be the DropDisabledFields() implementation -- I understand it's intended to hold the feature-gated logic, but the existing implementation already doesn't do that. Given the ugly reality of this I will try to hold to the spirit of that goal and you can tell me what you think. I'll ping you when I push another commit this afternoon.

bswartz · 2021-07-05T20:00:31Z

That last commit represents the changes to DropDisabledFields() and more tests. I still haven't reworded the comments/docs for the new core field. That will come next.

bswartz · 2021-07-05T20:43:02Z

The most recent commit also updates the doc comments, and should cover all of the feedback so far. Please let me know if the rewrite of DropDisabledFields() is clear enough and follows the spirit of the feature-gate rule. The original logic was flawed so this new version is not only more readable but also more functional.

thockin

This actually pushed me farther down the "move it" road. IIUC, updates of PVC can't change the data source, so having this logic in DropDisabled, which has an "old" and "new" param is extra confusing. Putting it in Normalize, which is only called on Create will make it that much more obvious what the heck is going on.

See, there's a story that you need to tell here. 3, 4, 5 years from now, someone will be staring at this and asking "WTF?!", running git blame and muttering about you. Very possibly that someone will be you.

I think you should move it, and tell the whole story in Normalize. You are swimming upstream.

If you need to merge this and followup, I could buy that, but I REALLY think we will regret not doing it right.

thockin · 2021-07-05T22:13:11Z

pkg/api/persistentvolumeclaim/util.go

@@ -39,26 +65,21 @@ func dataSourceInUse(oldPVCSpec *core.PersistentVolumeClaimSpec) bool {
 	if oldPVCSpec == nil {
 		return false
 	}
-	if oldPVCSpec.DataSource != nil {
+	if oldPVCSpec.DataSource != nil || oldPVCSpec.DataSourceRef != nil {
 		return true
 	}
 	return false
 }

 func dataSourceIsEnabled(pvcSpec *core.PersistentVolumeClaimSpec) bool {


IsEnabled is now a total misnomer.

DropDisabled is called by Create and Update, and Normalize is called by Create and Get/List. I'm not opposed to more refactoring, but I think we'll end up with a 3rd function and the unit tests will need a complete rewrite as a result.

I also agree that the existing method names suck, but there's a tension between wanting to clean up bad code and wanting to avoid fixing stuff that's not broken.

It is broken, it's just broken in human-space not compiler-space :)

bswartz · 2021-07-06T03:44:04Z

Okay I have split most of the code in DropDisabledFields into another function, so Drop just handles the feature gate dropping. The new function is called EnforceDataSourceBackwardsCompatibility (long, but hopefully not confusing). I don't think it's possible to combine this new function with NormalizeDataSources, because Normalize is also used on the Get/List code path.

Lots of unit tests needed updating. And also I realized that updates needed some work so I fixed that code path too. In particular we have to be careful not to break any updates which would have previously worked, given that the validation code prevents mutations of PVC specs, and we're playing a lot of games with automatically modifying these fields.

I still want to add a raft of E2E tests that cover the combination of all the changes, to try to smoke out any corner cases, especially around combinations of updates and changing of the feature gate from off to on or on to off.

bswartz · 2021-07-06T17:30:49Z

/retest

thockin · 2021-07-06T18:23:42Z

pkg/api/persistentvolumeclaim/util.go

+// to maintain backwards compatibility with old behavior.
+// See KEP 1495 for details.
+// Specifically, if this is an update of a PVC with no data source, or a creation of a new PVC,
+// and either the AnyVolumeDataSource feature gate is disabled or the dataSourceRef field is


Nitty: I wouldn't mention the feature gate here - it's not actually checked and it is irrelevant. The field is either in use or not. The gate is handled elsewhere.

As discussed in slack - this is the right place to tell the full story about why it is called on both create and update paths, and how it handles back-rev clients

thockin

LGTM from API POV

/approve

k8s-ci-robot · 2021-07-06T20:09:41Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bswartz, thockin

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~api/OWNERS~~ [thockin]
~~pkg/api/OWNERS~~ [thockin]
~~pkg/apis/OWNERS~~ [thockin]
~~pkg/registry/OWNERS~~ [thockin]
~~staging/src/k8s.io/api/OWNERS~~ [thockin]
~~staging/src/k8s.io/client-go/OWNERS~~ [thockin]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

bswartz · 2021-07-07T01:17:09Z

I'm going to squash the commits into one so it can merge

Modify the behavior of the AnyVolumeDataSource alpha feature gate to enable a new field, DataSourceRef, rather than modifying the behavior of the existing DataSource field. This allows addition Volume Populators in a way that doesn't risk breaking backwards compatibility, although it will result in eventually deprecating the DataSource field.

bswartz · 2021-07-07T01:19:09Z

Squashed

xing-yang · 2021-07-07T14:18:13Z

/lgtm

xing-yang · 2021-07-07T15:38:04Z

/milestone v1.22

caesarxuchao · 2021-07-08T20:03:55Z

/triage accepted

k8s-ci-robot requested review from brendandburns and deads2k June 28, 2021 20:03

k8s-ci-robot added sig/apps Categorizes an issue or PR as relevant to SIG Apps. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Jun 28, 2021

k8s-ci-robot assigned thockin Jun 30, 2021

bswartz force-pushed the data-source-ref branch from da72071 to 8f527fa Compare June 30, 2021 20:54

k8s-ci-robot added the sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. label Jun 30, 2021

bswartz force-pushed the data-source-ref branch from 8f527fa to 9c6076d Compare June 30, 2021 23:30

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 30, 2021

bswartz force-pushed the data-source-ref branch from 9c6076d to d46b7a0 Compare July 1, 2021 00:33

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 1, 2021

bswartz force-pushed the data-source-ref branch 2 times, most recently from d6eb9a7 to 3bdb330 Compare July 1, 2021 16:04

k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 1, 2021

bswartz force-pushed the data-source-ref branch from 3bdb330 to abd2575 Compare July 1, 2021 18:13

k8s-ci-robot added sig/storage Categorizes an issue or PR as relevant to SIG Storage. and removed sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. labels Jul 1, 2021

thockin reviewed Jul 2, 2021

View reviewed changes

pkg/apis/core/types.go Show resolved Hide resolved

staging/src/k8s.io/api/core/v1/types.go Show resolved Hide resolved

pkg/registry/core/persistentvolumeclaim/storage/storage.go Show resolved Hide resolved

pkg/api/persistentvolumeclaim/util.go Show resolved Hide resolved

thockin reviewed Jul 5, 2021

View reviewed changes

pkg/api/persistentvolumeclaim/util.go Show resolved Hide resolved

k8s-ci-robot added the sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. label Jul 5, 2021

thockin reviewed Jul 5, 2021

View reviewed changes

thockin reviewed Jul 6, 2021

View reviewed changes

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 6, 2021

bswartz force-pushed the data-source-ref branch from ca0e57a to 00dba76 Compare July 7, 2021 01:18

k8s-ci-robot assigned xing-yang Jul 7, 2021

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 7, 2021

k8s-ci-robot added this to the v1.22 milestone Jul 7, 2021

k8s-ci-robot merged commit eaba61b into kubernetes:master Jul 7, 2021

bswartz mentioned this pull request Jul 7, 2021

Implement Reject Invalid PVC DataSource KEP #99932

Closed

k8s-ci-robot added triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Jul 8, 2021

This was referenced Jul 9, 2021

Update AnyVolumeDataSource feature gate to beta #99956

Closed

[1.22] Volume populators redesign #1495 kubernetes/website#28610

Merged

csuzhangxc mentioned this pull request Jul 18, 2023

bump K8s to v1.22.17 pingcap/tidb-operator#5182

Merged

10 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add DataSourceRef field to PVC spec #103276

Add DataSourceRef field to PVC spec #103276

bswartz commented Jun 28, 2021 •

edited

fejta-bot commented Jun 28, 2021

bswartz commented Jun 30, 2021

fedebongio commented Jul 1, 2021

bswartz commented Jul 2, 2021

thockin left a comment

bswartz commented Jul 5, 2021

bswartz commented Jul 5, 2021

bswartz commented Jul 5, 2021

thockin left a comment

thockin Jul 5, 2021

bswartz Jul 5, 2021

thockin Jul 6, 2021

bswartz commented Jul 6, 2021

bswartz commented Jul 6, 2021

thockin Jul 6, 2021

thockin Jul 6, 2021

thockin left a comment

k8s-ci-robot commented Jul 6, 2021

bswartz commented Jul 7, 2021

bswartz commented Jul 7, 2021

xing-yang commented Jul 7, 2021

xing-yang commented Jul 7, 2021

caesarxuchao commented Jul 8, 2021

Add DataSourceRef field to PVC spec #103276

Add DataSourceRef field to PVC spec #103276

Conversation

bswartz commented Jun 28, 2021 • edited

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

fejta-bot commented Jun 28, 2021

bswartz commented Jun 30, 2021

fedebongio commented Jul 1, 2021

bswartz commented Jul 2, 2021

thockin left a comment

Choose a reason for hiding this comment

bswartz commented Jul 5, 2021

bswartz commented Jul 5, 2021

bswartz commented Jul 5, 2021

thockin left a comment

Choose a reason for hiding this comment

thockin Jul 5, 2021

Choose a reason for hiding this comment

bswartz Jul 5, 2021

Choose a reason for hiding this comment

thockin Jul 6, 2021

Choose a reason for hiding this comment

bswartz commented Jul 6, 2021

bswartz commented Jul 6, 2021

thockin Jul 6, 2021

Choose a reason for hiding this comment

thockin Jul 6, 2021

Choose a reason for hiding this comment

thockin left a comment

Choose a reason for hiding this comment

k8s-ci-robot commented Jul 6, 2021

bswartz commented Jul 7, 2021

bswartz commented Jul 7, 2021

xing-yang commented Jul 7, 2021

xing-yang commented Jul 7, 2021

caesarxuchao commented Jul 8, 2021

bswartz commented Jun 28, 2021 •

edited