bump/set default image to grafana 10

[gunpuz]

Is your feature request related to a problem? Please describe.
It would be nice to have newest Grafana.
For example: v10.2
https://github.com/grafana/grafana/releases

Currently seems that there is only v9.1.6

grafana-operator/controllers/config/operator_constants.go

Line 6 in 172fa66

GrafanaVersion = "9.1.6"

It also could be helpful to have option to choose which Grafana version to use?

[HVBE]

Hey @gunpuz We generally avoid using the latest version of Grafana (or have up until now, mainly due to the fact that we could never dedicate the time to ensure 100% compatibility with the operator with every new release). Hopefully now that we've migrated under Grafana, officially, we can probably start aligning a bit better

You can override the Grafana image through the Grafana resource container spec: https://github.com/grafana/grafana-operator/blob/master/examples/grafana_deployment/resources.yaml#L22-L23

The operator will respect this field if set by a user, otherwise it will use the GrafanaVersion from operator_constants.go.
You can also override any other field by doing this, and the operator will keep it updated and reconciled based on your definition.

[HVBE]

Setting as triage/accepted, as we should probably start doing this going forward, but haven't yet, however, low priority as a workaround exists.

[vladimirmukhin]

@HubertStefanski I've done the upgrade before, and I can take care of maintaining the version of Grafana on a regular basis.
Several questions:

• How often should we upgrade? Should we do it with every stable release or follow a set schedule?
• Is there a standard checklist or test procedure to check compatibility, or does this change with each release? Do we need to test each new Grafana feature separately?
• Wouldn't it be simpler to make the Grafana version a configurable parameter? We could set certain stable versions as defaults, and if a user wants to upgrade, they can do so at their own risk.

[HVBE]

Hey @vladimirmukhin !

How often should we upgrade? Should we do it with every stable release or follow a set schedule?

IMHO, as soon as possible, a few days/week delay shouldn't really be an issue, as long as we're stable on the operator side.

Is there a standard checklist or test procedure to check compatibility, or does this change with each release? Do we need to test each new Grafana feature separately?

There isn't a checklist, Ideally we would have all of our supported features covered by unit tests, so we could update things automatically, with relative confidence.
Currently, the operator only supports a handful of Grafana resources (dashboards, data sources, plugins, folders etc.), as long as there isn't a breaking change on the operator-managed resources coming from the Grafana API, then we should be all good.
Albeit, now that we are under the Grafana organization, this feature set will be expanded, due to popular demand (we have an alerting proposal in the works by @theSuess in #1349)

Wouldn't it be simpler to make the Grafana version a configurable parameter? We could set certain stable versions as defaults, and if a user wants to upgrade, they can do so at their own risk.

Not sure if I understand, the Grafana version is configurable already through https://github.com/grafana/grafana-operator/blob/master/examples/grafana_deployment/resources.yaml#L22-L23
Here, the operator will read the Grafana CR, and the related container spec and use the values provided by the user, if those values aren't set we will use the default (from operator_constants). So right now it is possible to use a Grafana container image version specified by the user (you can even point it to your own mirror, if you run in disconnected or limited network modes)

[HVBE]

So, Ideally, we should prepare the operator for an automated Grafana update cycle. But that would mean quite a bit of work to get there.
If we improve our unit/integration/e2e tests, then we can pretty much update to the newest Grafana version the same day it comes out, and push a new operator version with a new default in a relatively short amount of time.

[weisdd]

@vladimirmukhin

• At the moment, I'd say, the hardcoded version of Grafana is simply the one we use in our local development environments (some of us override it). Like @HubertStefanski mentioned, users are free to specify whatever image they want, there are no restrictions;
• Grafana API changes from time to time. It's easy to catch big breaking (from operator's perspective) changes as they are tied to major versions (e.g. alerting), but minor updates can also bring some surprises (e.g. in one version you're allowed to change UID of a resource, in another - not). Until we introduce more E2E tests, it's a bit unsafe to bump versions without carefully reading changelog. At the same time, I wouldn't really expect many things to break as a result of minor version bump;
• It, probably, wouldn't make sense to bump a version each time as we don't publish daily builds of the operator, and, like it was mentioned above, users can follow their own upgrade policy;
• Some upcoming changes like docs: add proposal for grafana alerting #1349 would certainly come with version upgrades.

To be clear, I'm not against updates, just wanted to highlight that things are not that simple as long as operator development is not fully tied to Grafana itself :)

[NissesSenap]

@vladimirmukhin personally, I update to a new Grafana version as quickly as I can. So far, I haven't seen any big issues together with the operator. But if I would find any issues, I would revert the change and create an issue in this repo.

I think it would be a good idea to bump to version 10 to get it as a part of our build pipeline. But as Hubert and Igor points out, the operator isn't currently tied to a specific version of grafana.

So from my point of view, please provide a PR where we bump to the latest grafana 10 version.