Informers granularity

[fhalde]

Hi there, I'm looking for some guidance on our controller design

We have a controller that manages a CustomResource (CR) which has the following structure

spec:
  clusterRef: $clusterref
  configmapRef: $cmref

Our controller starts off with a single informer for this CustomResource type.
Each instance of a CR then creates a new informer for each of the Ref's listed in order to react to their changes. This is an informer specifically listening to just 1 object in k8s.

It's expected that the refs will have a high cardinality, so we are expecting a lot of informers created.
My question is, is this an OK level of granularity for setting up informers? I've noticed that each instance of a informer creates a watch & I don't know of its implications.

The other alternative is to create singleton informers for each apiGroup/kind ( the list of resource "types" we are interested in is well known in advance ) and have a handler that tracks dependencies and routes accordingly ( a router of sorts )

[csviri]

Hi @fhalde

The other alternative is to create singleton informers for each apiGroup/kind

this is the way to go.

[csviri]

Note that dynamically adding informers is quite a special thing, rarely used in controllers, and if yes usually in meta-controllers, when the types are not known upfront. Java Operator SDK supports that, see:

https://github.com/operator-framework/java-operator-sdk/blob/3c863793fdd2341d434bdfcfd6216b4023554a7b/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/EventSourceRetriever.java#L46

But as I said should be the last effort to used that.
If you are still interested to dig deeper how that is used check:
https://github.com/csviri/kubernetes-glue-operator
what is based on that.

[fhalde]

This is what it would look like on a high level. There will be several instances of these workflows running for each CR instance

  graph TD;
      A[CR instance Handler function]-->|On CR CRUD|A;
      B[Cluster instance informer]-->|When the referred Cluster object changes|A
      C[ConfigMap instance informer]-->|When the referred ConfigMap object changes|A;

[csviri]

Do you plant o use Java Operator SDK for this?

[fhalde]

Yes!

[fhalde]

this is the way to go.

is this mainly to do with performance & the number of watches it creates?

[csviri]

yes, mainly because of that