Add support to provide the input variables for dashboards

[marcofranssen]

Currently the grafana_dashboard resource does not allow to provide the input values for the dashboards.

Currently I implemented a workaround.

This is how I'm currently working around the issue for a provisioning dashboards using the Terraform Grafana provider.

variable "data_sources" {
  description = "The data source configurations for the amg workspace."
  type = map(object({
    name                     = string
    type                     = string
    url                      = optional(string, "")
    is_default               = optional(bool, false)
    basic_auth_enabled       = optional(bool, false)
    basic_auth_username      = optional(string, "")
    json_data_encoded        = optional(string, "")
    secure_json_data_encoded = optional(string, "")
  }))
  sensitive = true
}

variable "dashboards" {
  description = "The dashboards to configure."
  type = map(object({
    folder       = optional(string, "General")
    url          = string
    data_sources = optional(map(string), null)
  }))
}

locals {
  default_data_sources = {
    DS_PROMETHEUS : "prometheus"
    DS_LOKI : "loki"
  }
  dashboards = {
    for key, value in var.dashboards :
    key => {
      folder = value.folder
      json = format(
        replace(replace(data.http.dashboards[key].response_body, "%", "%%"), "/\\$${(${join("|", keys(coalesce(value.data_sources, local.default_data_sources)))})}/", "%s"),
        [
          for token in flatten(regexall("\\$${(${join("|", keys(coalesce(value.data_sources, local.default_data_sources)))})}", data.http.dashboards[key].response_body)) :
          grafana_data_source.this[lookup(coalesce(value.data_sources, local.default_data_sources), token)].uid
        ]...
      )
    }
  }
  folders = distinct([for key, value in local.dashboards : value.folder])
}

data "http" "dashboards" {
  for_each = var.dashboards
  url      = each.value.url
  lifecycle {
    postcondition {
      condition     = self.status_code == 200
      error_message = "Unable to read remote manifest ${each.value.url}"
    }
  }
}

resource "grafana_folder" "this" {
  for_each = { for f in local.folders : f => f }
  title    = each.value
}

resource "grafana_dashboard" "this" {
  for_each    = local.dashboards
  config_json = each.value.json
  folder      = grafana_folder.this[each.value.folder].id
  overwrite   = true
}

It would be great if the resource itself would allow providing these inputs.

e.g.

resource "grafana_dashboard" "this" {
  for_each    = local.dashboards
  config_json = each.value.json
  inputs = {
    DS_PROMETHEUS = grafana_datasource.prometheus.uid
    DS_LOKI       = grafana_datasource.loki.uid
  }
  folder      = grafana_folder.this[each.value.folder].id
  overwrite   = true
}

[julienduchesne]

Terraform v1.9.0 has a templatestring function (https://discuss.hashicorp.com/t/experiment-feedback-the-templatestring-function/66645). It would look something like:

resource "grafana_dashboard" "this" {
  for_each    = local.dashboards
  config_json = templatestring(each.value.json, {
    DS_PROMETHEUS = grafana_datasource.prometheus.uid
    DS_LOKI       = grafana_datasource.loki.uid
  })
  folder      = grafana_folder.this[each.value.folder].id
  overwrite   = true
}

Does that work?

[julienduchesne]

The other option is to use the external provider: https://registry.terraform.io/providers/hashicorp/external/latest/docs/data-sources/external. Write a tiny script that does the templating

Having two fields managing the JSON config of a dashboard is not going to give you a great experience. Terraform providers work well with a 1:1 field mapping.

Next steps:

If I were to implement something to complete this issue, it'd be a new datasource that, when given a dashboard JSON, it does some templating on it and spits it back out

data "grafana_dashboard_process" "my_dashboard" {
   config_json = ...
   inputs = {...}
} 
resource "grafana_dashboard" "my_dashboard" {
  config_json = data.grafana_dashboard_process.my_dashboard.processed_config_json
}

Doing this has the following benefits:

• Can't break the existing dashboard resource
• config JSON stays a 1:1 mapping so the diff keeps working well
• Easy to test

Anyone has any opinions on this proposal?

[nicolai-hornung-bl]

I would appreciate such a feature greatly! In addition to the possibility of overriding datasources what I would also like to see would be the option to inject / override dashboard settings such as UID, Tags and Links inside the config_json.

[julienduchesne]

🤔 hmm found this answer: https://stackoverflow.com/a/67378512 and I'm wondering if anyone has any dashboard mutations they want to do that aren't just search and replace. I feel like if it's possible to do with native Terraform, it doesn't deserve a new grafana specific datasource

[milldr]

I didnt want to use templatestring because it's only recently added with the latest Terraform version under BSL, so instead you can do something like this:

(note: this is using Cloud Posse's null-label context for module.this)

 tf vars
{
  "config_input": {
    "${DS_PROMETHEUS}": "myuniqueprometheusid"
  },
  "dashboard_url": "https://grafana.com/api/dashboards/315/revisions/3/download",
}

# variables.tf
variable "dashboard_url" {
  type        = string
  description = "The marketplace URL of the dashboard to be created"
}

variable "additional_config" {
  type        = map(any)
  description = "Additional dashboard configuration to be merged with the provided dashboard JSON"
  default     = {}
}

variable "config_input" {
  type        = map(string)
  description = "A map of string replacements used to supply input for the dashboard config JSON"
  default     = {}
}

# main.tf
locals {
  enabled = module.this.enabled

  # Replace each of the keys in var.config_input with the given value in the module.config_json[0].merged result
  config_json = join("", [for k in keys(var.config_input) : replace(jsonencode(module.config_json[0].merged), k, var.config_input[k])])
}

data "http" "grafana_dashboard_json" {
  count = local.enabled ? 1 : 0

  url = var.dashboard_url
}

module "config_json" {
  source  = "cloudposse/config/yaml//modules/deepmerge"
  version = "1.0.2"

  count = local.enabled ? 1 : 0

  maps = [
    jsondecode(data.http.grafana_dashboard_json[0].response_body),
    {
      "uid" : module.this.id
      "title" : module.this.id
    },
    var.additional_config
  ]
}

resource "grafana_dashboard" "this" {
  count = local.enabled ? 1 : 0

  config_json = local.config_json
}