Unable to move alert rules between rule groups

[Balsir]

Terraform Version

• Terraform: 1.5.6
• Terraform Grafana Provider: 2.11.0 (also 2.8.1)
• Grafana: 10.2.2 (also 10.3.1)

Affected Resource(s)

• grafana_rule_group

Terraform Configuration Files

resource "grafana_rule_group" "test" {
  name = "test"
  folder_uid = "apps"
  interval_seconds = 600

  rule {
    condition      = "B"
    for            = "0s"
    labels         = { }
    name           = "test" # rename to test2

    data {
      ref_id     = "A"
      query_type = ""
      relative_time_range {
        from = 600
        to   = 0
      }
      datasource_uid = "bla"
      model = jsonencode({
        hide          = false
        intervalMs    = 1000
        maxDataPoints = 43200
        refId         = "A"
      })
    }
  }
}

resource "grafana_rule_group" "test2" {
  name = "test2"
  folder_uid = "apps"
  interval_seconds = 60

  rule {
    condition      = "B"
    for            = "0s"
    labels         = { }
    name           = "test2" #rename to test

    data {
      ref_id     = "A"
      query_type = ""
      relative_time_range {
        from = 600
        to   = 0
      }
      datasource_uid = "bla"
      model = jsonencode({
        hide          = false
        intervalMs    = 1000
        maxDataPoints = 43200
        refId         = "A"
      })
    }
  }
}

Expected Behavior

Alert rules are moved between rule groups.

Actual Behavior

Apply fails with error.

│ Error: [PUT /v1/provisioning/folder/{FolderUID}/rule-groups/{Group}][400] putAlertRuleGroupBadRequest {"message":"failed to insert alert rules: a conflicting alert rule is found: rule title under the same organisation and folder should be unique"}

Steps to Reproduce

There are multiple ways to get this error:

1. Create two rule groups test with rule test and test2 with rule test2 in the same folder.
2. Switch alert rule names between the rule groups.

Alternatively:

1. Create rule group test with alert rule test.
2. Create rule group test2 with alert rule test, rename alert rule in rule group test to test2.
3. (In this case, only the first apply fails)

Or:

1. Create rule group test with alert rules test and test3.
2. Create rule group test2 with alert rule test2.
3. Move alert rule test2 to rule group test and comment out rule group test2 to destroy it.
4. (Again, the failed reply can be retried)

References

Are there any other GitHub issues (open or closed) or Pull Requests that should be linked here? For example:

• Similar to TF run produces incorrect diff and fails to apply #711

[julienduchesne]

I'd say that's expected, rule names are unique and a rule group can't take control of an existing rule. You can either destroy and recreate the resources completely or, if possible, apply an intermediate state (pretty much was is happening in your "Alternatively" steps).

What would you expect to happen here (Considering Terraform's limitation that each resource is independent)?

[Balsir]

Rule group recreation or intermediate state would be ok in a simple scenario such as in the example. However, if you manage multiple rule groups with numerous rules, I would expect moving rules between rule groups, or taking an existing rule and creating a new rule group for it without having to change the rule's name would be seamless and done via one apply.

From a technical standpoint, I would expect Terraform to either handle this error and retry (since the issue seems to be that new rule creation occurs before the previous one is destroyed, and that's why the second apply in "Alternatively" succeeds), or first destroy the rule and then try to create a new one.

[julienduchesne]

There is nothing in Terraform that allows me to fix the first case. Changes to each rule group is a conflict. Retries would not work there because none of the new resources are valid and Terraform does not give me information about other resources, so I can't know that we're switching rules between two resources. The provider also shouldn't act on anything that isn't imported into state (so I can't/shouldn't really go and delete the rule from the second group, while applying the first group), because that would be extremely unexpected behaviour.

The only thing I can think of is for you to terraform destroy -target your resources and apply them again

[Balsir]

The first case is indeed problematic.
Do you suppose something can be done regarding the second case or the third case (added to the description, basically error appears during move of last remaining rules from rule group)?

[julienduchesne]

Yes, it's possible to add a retry for that case