✨ Add cache option to filter by field #1404

qinqon · 2021-03-01T09:39:25Z

All instance for a same resources are being cached by
controller-runtime, for some use cases this consumes a lot of memory and
CPU. This change add a option to the cache so resources can be filtered
by field.

k8s-ci-robot · 2021-03-01T09:39:33Z

Hi @qinqon. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot · 2021-03-01T09:39:34Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: qinqon
To complete the pull request process, please assign mengqiy after the PR has been reviewed.
You can assign the PR to them by writing /assign @mengqiy in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

vincepri

Hi @qinqon, thanks for the contribution. Left a few comments, let's please reduce the PR to the absolute minimum set of changes needed for the requested feature addition. Also this doesn't seem to be a 🐛, but rather a new feature ✨

vincepri · 2021-03-01T18:11:37Z

pkg/cache/internal/informers_map.go

-	// Scheme maps runtime.Objects to GroupVersionKinds
-	Scheme *runtime.Scheme


This change seems outside the scope of this PR?

vincepri · 2021-03-01T18:11:48Z

pkg/cache/internal/informers_map.go

-	// mapper maps GroupVersionKinds to Resources
-	mapper meta.RESTMapper
-


Same as above, could we keep these?

vincepri · 2021-03-01T18:11:51Z

pkg/cache/internal/informers_map.go

-	// resync is the base frequency the informers are resynced
-	// a 10 percent jitter will be added to the resync period between informers
-	// so that all informers will not send list requests simultaneously.
-	resync time.Duration
-


Same as above, could we keep these?

vincepri · 2021-03-01T18:11:54Z

pkg/cache/internal/informers_map.go

-	// namespace is the namespace that all ListWatches are restricted to
-	// default or empty string means all namespaces
-	namespace string


Same as above, could we keep these?

vincepri · 2021-03-01T18:12:34Z

pkg/cache/cache.go

+
+	// FieldSelectorByResource restricts the cache's ListWatch to the desired
+	// fields per resource
+	FieldSelectorByResource map[string]string


What's the key and value expected to look like here? Would it better to use a GroupKind as key instead?

Or GroupResource?

Used GroupResource so we can match all the versions.

vincepri · 2021-03-01T18:14:00Z

pkg/cache/internal/options.go

+// Options are the optional arguments for creating a new InformersMap object
+type Options struct {
+	Scheme                  *runtime.Scheme
+	Mapper                  meta.RESTMapper
+	Resync                  *time.Duration
+	Namespace               string
+	FieldSelectorByResource map[string]string
+}


Given that this is all internal, how are users supposed to use this new feature?

Removed the internal Options

vincepri · 2021-03-01T18:14:55Z

/ok-to-test

qinqon · 2021-03-02T15:17:28Z

Remove the commit that encapsulate some internal attributes to an internal Options struct and create a small helper to find the fieldselector on the map using the resource from mapper, @vincepri you can take a look now

qinqon · 2021-03-02T15:25:00Z

/test pull-controller-runtime-test-master

alvaroaleman · 2021-03-02T20:57:56Z

pkg/cache/cache.go

+
+	// FieldSelectorByResource restricts the cache's ListWatch to the desired
+	// fields per resource
+	FieldSelectorByResource map[schema.GroupResource]string


Please add a sample field selector here and link to the kube docs about field selectors, many ppl are not aware that they only work on a very limited set of fields and not at all for CRDs

its medadata.name, not meta.name or not?

Done, correct value is metadata.name.

alvaroaleman · 2021-03-02T20:58:51Z

@qinqon this doesn't close #244 as it doesn't allow filtering by labels (which I would assume to be more common), so please remove that from the PR body

alvaroaleman · 2021-03-02T21:00:22Z

pkg/cache/internal/informers_map.go

@@ -231,6 +235,15 @@ func (ip *specificInformersMap) addInformerToMap(gvk schema.GroupVersionKind, ob
 	return i, ip.started, nil
 }

+func (ip *specificInformersMap) findFieldSelectorByGVR(gvr schema.GroupVersionResource) string {
+	gr := schema.GroupResource{Group: gvr.Group, Resource: gvr.Resource}
+	fieldSelector, ok := ip.fieldSelectorByResource[gr]


Nit: This is an empty string if there isn't a match in the map, so you do not need the ok check

qinqon · 2021-03-03T09:38:44Z

@qinqon this doesn't close #244 as it doesn't allow filtering by labels (which I would assume to be more common), so please remove that from the PR body

I can try to add labels filtering at a follow up PR to keep this small.

alvaroaleman

Looks good except for the doc sample.

I can try to add labels filtering at a follow up PR to keep this small.

If you have the time for that, that would be great but I didn't mean to say that you are in any way obligated to that, just that it would be premature to close the issue

alvaroaleman · 2021-03-03T14:20:42Z

pkg/cache/cache.go

+
+	// FieldSelectorByResource restricts the cache's ListWatch to the desired
+	// fields per resource
+	FieldSelectorByResource map[schema.GroupResource]string


its medadata.name, not meta.name or not?

qinqon · 2021-03-08T06:46:49Z

@vincepri do you know if Is the issue at pull-controller-runtime-apidiff-master important ?

qinqon · 2021-03-08T06:53:37Z

force-push: Rebased

alvaroaleman · 2021-03-08T14:08:31Z

@qinqon The one thing this change ignores entirely is what impact this has for the cache-backed client. Could you write a small design doc in a different PR that:

Outlines the motivation for this and possible alternatives
What this means for the cache-backed client
Also: Are there ways to defined a fieldSelector other than via a plain string?

Doesn't have to be anything big, but we asked for that multiple times in #244 (comment) and declined other PRs because of that. I personally like the simplicity of just saying "We allow this on the cache and if you use a custom cache on your manager, you imply that you know better" but its a bigger design question and I want to be sure there is agreement about that.

qinqon · 2021-03-09T13:14:54Z

@qinqon The one thing this change ignores entirely is what impact this has for the cache-backed client. Could you write a small design doc in a different PR

@alvaroaleman done a PR with the proposal #1419

qinqon · 2021-03-16T09:29:55Z

force-push: Rebased and converting the map value to fields.Selector

As a follow up of [1] this change instruct the ListWatcher from the informers to select by label/field but instead of using a manager/cache Options it pass the select using the builder. [1] kubernetes-sigs#1404 Signed-off-by: Quique Llorente <ellorent@redhat.com>

All instance for a same resources are being cached by controller-runtime, for some use cases this consumes a lot of memory and CPU. This change add a option to the cache so resources can be filtered by field. Signed-off-by: Quique Llorente <ellorent@redhat.com>

k8s-ci-robot · 2021-03-17T09:07:53Z

@qinqon: The following test failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
pull-controller-runtime-apidiff-master	`ff97b8f`	link	`/test pull-controller-runtime-apidiff-master`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Mar 1, 2021

k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Mar 1, 2021

k8s-ci-robot requested review from alenkacz and gerred March 1, 2021 09:39

k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Mar 1, 2021

qinqon mentioned this pull request Mar 1, 2021

Enable filtered list watches as watches #244

Closed

qinqon changed the title ~~Add cache option to filter by field 🐛~~ 🐛 Add cache option to filter by field 🐛 Mar 1, 2021

qinqon force-pushed the add-cache-filter-option branch 3 times, most recently from 250a741 to c1948c8 Compare March 1, 2021 13:53

qinqon mentioned this pull request Mar 1, 2021

Use a field name selected cache nmstate/kubernetes-nmstate#687

Closed

vincepri reviewed Mar 1, 2021

View reviewed changes

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Mar 1, 2021

qinqon changed the title ~~🐛 Add cache option to filter by field 🐛~~ ✨ Add cache option to filter by field 🐛 Mar 2, 2021

qinqon force-pushed the add-cache-filter-option branch 4 times, most recently from f5be2b0 to 2dd1373 Compare March 2, 2021 15:15

qinqon requested a review from vincepri March 2, 2021 15:16

qinqon force-pushed the add-cache-filter-option branch from 2dd1373 to 6edc30b Compare March 2, 2021 15:22

alvaroaleman reviewed Mar 2, 2021

View reviewed changes

qinqon force-pushed the add-cache-filter-option branch from 6edc30b to cf2e7dc Compare March 3, 2021 07:16

qinqon force-pushed the add-cache-filter-option branch from cf2e7dc to 948d09c Compare March 3, 2021 09:37

qinqon requested a review from alvaroaleman March 3, 2021 09:38

alvaroaleman reviewed Mar 3, 2021

View reviewed changes

qinqon force-pushed the add-cache-filter-option branch from 948d09c to 03473b4 Compare March 3, 2021 15:05

qinqon requested a review from alvaroaleman March 3, 2021 15:06

sozercan mentioned this pull request Mar 5, 2021

feat: add filtered watch for reconcile kubernetes-sigs/secrets-store-csi-driver#458

Merged

qinqon force-pushed the add-cache-filter-option branch from 03473b4 to a4e89ae Compare March 8, 2021 06:53

qinqon changed the title ~~✨ Add cache option to filter by field 🐛~~ ✨ Add cache option to filter by field Mar 8, 2021

qinqon mentioned this pull request Mar 9, 2021

🌱 Add a design document to filter cache by selectors #1419

Merged

qinqon force-pushed the add-cache-filter-option branch from a4e89ae to b6217c6 Compare March 16, 2021 09:28

qinqon mentioned this pull request Mar 16, 2021

✨ Add Selector option to Builder #1425

Closed

qinqon force-pushed the add-cache-filter-option branch from b6217c6 to ff97b8f Compare March 17, 2021 09:05

qinqon closed this Apr 6, 2021

qinqon deleted the add-cache-filter-option branch April 6, 2021 07:45

		// Scheme maps runtime.Objects to GroupVersionKinds
		Scheme *runtime.Scheme

		// mapper maps GroupVersionKinds to Resources
		mapper meta.RESTMapper

✨ Add cache option to filter by field #1404

✨ Add cache option to filter by field #1404

Conversation

qinqon commented Mar 1, 2021 • edited

k8s-ci-robot commented Mar 1, 2021

k8s-ci-robot commented Mar 1, 2021

vincepri left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qinqon Mar 2, 2021 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qinqon Mar 2, 2021 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vincepri commented Mar 1, 2021

qinqon commented Mar 2, 2021

qinqon commented Mar 2, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alvaroaleman commented Mar 2, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qinqon commented Mar 3, 2021

alvaroaleman left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qinqon commented Mar 8, 2021

qinqon commented Mar 8, 2021 • edited

alvaroaleman commented Mar 8, 2021

qinqon commented Mar 9, 2021

qinqon commented Mar 16, 2021

k8s-ci-robot commented Mar 17, 2021

qinqon commented Mar 1, 2021 •

edited

qinqon Mar 2, 2021 •

edited

qinqon Mar 2, 2021 •

edited

qinqon commented Mar 8, 2021 •

edited