[Bug] Unable to set secureJsonData.basicAuthPassword in GrafanaDatasource

[jb95x]

Describe the bug

The following manifests result in a data source being added to the grafana instance with the correct username for basic authentication, but the password field for basic authentication is empty and the datasource doesn't work.

loki-datasource.yaml

apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaDatasource
metadata:
 name: loki
 annotations:
   helm.sh/hook: post-install,post-upgrade
   helm.sh/hook-weight: "1"
spec:
 valuesFrom:
   - targetPath: basicAuthUser
     valueFrom:
       secretKeyRef:
         name: loki-gateway-credentials
         key: LOKI_USER

   - targetPath: secureJsonData.basicAuthPassword
     valueFrom:
       secretKeyRef:
         name: loki-gateway-credentials
         key: LOKI_PASSWORD

 instanceSelector:
   matchLabels:
     dashboards: 'grafana'
 datasource:
   name: Loki
   type: loki
   access: proxy
   basicAuth: true
   url: http://loki-gateway.loki.svc.cluster.local
   isDefault: true
   basicAuthUser: ${LOKI_USER}
   jsonData:
     tlsSkipVerify: true
   secureJsonData:
     basicAuthPassword: ${LOKI_PASSWORD}
   editable: true

grafana.yaml

apiVersion: grafana.integreatly.org/v1beta1
kind: Grafana
metadata:
  name: grafana
  labels:
    dashboards: "grafana"
  annotations:
    helm.sh/hook: post-install,post-upgrade
    helm.sh/hook-weight: "0"
spec:
  persistentVolumeClaim:
    spec:
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 10Gi
  config:
    log:
      mode: "console"
    auth:
      disable_login_form: "false"
  deployment:
    spec:
      template:
        spec:
          securityContext:
            fsGroup: 10001
          containers:
            - name: grafana
              image: grafana/grafana:10.1.5
              securityContext:
                allowPrivilegeEscalation: true
                readOnlyRootFilesystem: false
              readinessProbe:
                failureThreshold: 3
              env:
                - name: GF_SECURITY_ADMIN_USER
                  valueFrom:
                    secretKeyRef:
                      key: GF_SECURITY_ADMIN_USER
                      name: grafana-admin-credentials
                - name: GF_SECURITY_ADMIN_PASSWORD
                  valueFrom:
                    secretKeyRef:
                      key: GF_SECURITY_ADMIN_PASSWORD
                      name: grafana-admin-credentials
          volumes:
            - name: grafana-data
              persistentVolumeClaim:
                claimName: grafana-pvc

Version
grafana-operator v5.8.1

To Reproduce
Deploy the provided manifests

Expected behavior
Loki datasource being setup properly with the basic authentication password

Suspect component/Location where the bug might be occurring
unknown

Screenshots

Runtime (please complete the following information):

• OS: Ubuntu 22.04
• Grafana Operator Version: v5.8.1
• Environment: Kubernetes 1.28.5
• Deployment type: Deployed with ArgoCD

[matt-delaney-cruise]

I just discovered this as well and have figured out the cause.

Recently the grafana-operator moved to use the grafana-openapi-client and unfortunately it has secureJsonFields instead of secureJsonData. See this line. So, when the datasource controller attempts to unmarshal here it sets secureJsonFields to nil as it doesn't know about secureJsonData.

I'd assume that in order to not break backward compatibility, getDatasourceContent would probably need to do this mapping.

[matt-delaney-cruise]

Actually, looking more at this, SecureJSONFields is a map of bool, so actually, kinda looks like issue is that models.DataSource is the wrong thing to use as it's assuming that secure data is not available (hence bool instead of a value).

Seems like either models.AddDataSourceCommand or models.UpdateDataSourceCommand would need to be used instead (the difference being that UpdateDataSourceCommand has a Version).

[NissesSenap]

So I will close this issue since it's a duplicate of #1482.
We are aware of the bug and any help around it would be greatly appreciated.

[mdelaney]

As mentioned in #1482, I ran into this and have gotten to the bottom of it. Opened #1488 which fixes this.

edit: just realized this requires a CLA, will need to talk with my employer to see about getting this :-/