New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enable filtered list watches as watches #244
Comments
+1 I've run into a similar issue with a controller I'm writing. I wound up starting a goroutine when the controller gets added to watch/act on certain objects that are created and labeled by another controller outside of mine. |
we'd have to specify this at the manager level, but it should be possible to pass restricting items. |
(since caches are shared) |
we could also let users manually specify auxiliary caches |
if anyone has ideas for a design doc, I'd love to see them. |
/kind feature this has been coming up a lot lately, and seems like a good thing to tackle in the list of related issues. |
i.e. one sketch: Have a list-options watch predicate, smartly detect the presence of that (in some generic way) and pass it down to
That last one might solve multiple problems in one fell swoop, but I'd need to see a design sketch to make sure that it's not too arcane. |
I will follow this to understand how this is properly implemented and any questions from a user or testing perspective please ask and I'll do my best to assist. In the interim I've implemented this for my own use case.
|
That's just like a predicate, though -- it doesn't actually get you most of the benefits of the filtering that people are asking for here, since it doesn't ask for server-side filtering. |
I know... As I said above the code block
|
Ack, just wanted to make sure we were on the same page :-) |
Issues go stale after 90d of inactivity. If this issue is safe to close now please do so with Send feedback to sig-testing, kubernetes/test-infra and/or fejta. |
Stale issues rot after 30d of inactivity. If this issue is safe to close now please do so with Send feedback to sig-testing, kubernetes/test-infra and/or fejta. |
/remove-lifecycle stale |
Rotten issues close after 30d of inactivity. Send feedback to sig-testing, kubernetes/test-infra and/or fejta. |
@fejta-bot: Closing this issue. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
/reopen |
@alvaroaleman: Reopened this issue. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
bumping this feature request:
I think this is a viable option, but I wonder if we could have a more straightforward pattern like a user specifies a watch with a label. If they do, then we use the label-filtered cache for that specific GVK. If another watch is created without that label-filter, then we will error out and not establish the watch. The most significant caveat that I see is the client will only use the label filtered cache. We would need to make that clear in the documentation, or is there some other mechanism to alert the user to this. |
The controller-runtime is missing filtering it's cache by labels or fields [1], this means that all the kubernetes-nmstate-handlers will read all the nodes and nodenetworkstates every period, clearly dies does not scale since kubernetes-nmstate-handler runs at as daemonset meaning that there is one handler running at every node so the bigger the cluster the bigger the problem. This change replace the default controller-runtime cache with an implementation that can be configured to use some field selectors depending on the resource, this way we can filter by "metadata.name" using the node name for "node" and "nodenetworkstate" so only one instance of them is feteched. [1] kubernetes-sigs/controller-runtime#244 Signed-off-by: Quique Llorente <ellorent@redhat.com>
The controller-runtime is missing filtering it's cache by labels or fields [1], this means that all the kubernetes-nmstate-handlers will read all the nodes and nodenetworkstates every period, clearly dies does not scale since kubernetes-nmstate-handler runs at as daemonset meaning that there is one handler running at every node so the bigger the cluster the bigger the problem. This change replace the default controller-runtime cache with an implementation that can be configured to use some field selectors depending on the resource, this way we can filter by "metadata.name" using the node name for "node" and "nodenetworkstate" so only one instance of them is feteched. [1] kubernetes-sigs/controller-runtime#244 Signed-off-by: Quique Llorente <ellorent@redhat.com>
The controller-runtime is missing filtering it's cache by labels or fields [1], this means that all the kubernetes-nmstate-handlers will read all the nodes and nodenetworkstates every period, clearly dies does not scale since kubernetes-nmstate-handler runs at as daemonset meaning that there is one handler running at every node so the bigger the cluster the bigger the problem. This change replace the default controller-runtime cache with an implementation that can be configured to use some field selectors depending on the resource, this way we can filter by "metadata.name" using the node name for "node" and "nodenetworkstate" so only one instance of them is feteched. [1] kubernetes-sigs/controller-runtime#244 Signed-off-by: Quique Llorente <ellorent@redhat.com>
Any updates on this? |
I have try a PR to specify a fieldselector at manager build time #1404. |
The controller-runtime is missing filtering it's cache by labels or fields [1], this means that all the kubernetes-nmstate-handlers will read all the nodes and nodenetworkstates every period, clearly dies does not scale since kubernetes-nmstate-handler runs at as daemonset meaning that there is one handler running at every node so the bigger the cluster the bigger the problem. This change replace the default controller-runtime cache with an implementation that can be configured to use some field selectors depending on the resource, this way we can filter by "metadata.name" using the node name for "node" and "nodenetworkstate" so only one instance of them is feteched. [1] kubernetes-sigs/controller-runtime#244 Signed-off-by: Quique Llorente <ellorent@redhat.com>
New PR: #1435 |
#1435 is now merged, so is this resolved? |
It is being worked into a release currently @invidian Looking at the tags there is a v0.9.0-beta available if you want to test. |
Thanks, will do! |
@estroz: Closing this issue. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
It seems like the current solution doesn't address the use case mentioned in It should be possible to create a namespace scoped client in ListFunc and WatchFunc if there is a field selector for |
@roee88 what was merged allows to set a FieldSelector so yes, this is possible |
To clarify, I suspect that without affecting the parameters to NamespaceIfScoped, the selector opts have no effect on the required rbac (cluster role bindings vs role bindings). For example here: controller-runtime/pkg/cache/internal/informers_map.go Lines 282 to 288 in 64b1c72
My question is whether changing the code here makes sense. Specifically for the case where ip.namespace is empty, the value from a cc @shlomitk1 |
That sounds reasonable |
When setting up watches during initialization it's currently not possible to filter by any selectors (which is possible using list watches).
For example it is not possible to only watch pods with specific labels (e.g having the label
pod-type: my-controller-type
). The current behavior results in very broad caching, which might not be desirable for large Kubernetes deployments.In some scenarios an operator could contain multiple controllers, and they all share caches, so keying caches on GVK's alone might be problematic if they want to watch the same resource type, but with different filters.
When doing
List
/Get
, how would one decide which of the caches to use? It seems that perhaps this needs to be an explicit choice by the operator developers?The text was updated successfully, but these errors were encountered: