Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Race condition in CRS controller results in ovewriting updates #10655

Open
jimmidyson opened this issue May 21, 2024 · 1 comment · May be fixed by #10656
Open

Race condition in CRS controller results in ovewriting updates #10655

jimmidyson opened this issue May 21, 2024 · 1 comment · May be fixed by #10656
Assignees
Labels
area/clusterresourceset Issues or PRs related to clusterresourcesets kind/bug Categorizes issue or PR as related to a bug. priority/backlog Higher priority than priority/awaiting-more-evidence. triage/accepted Indicates an issue or PR is ready to be actively worked on.

Comments

@jimmidyson
Copy link
Member

jimmidyson commented May 21, 2024

What steps did you take and what happened?

Create multiple ClusterResourceSets (e.g. 5) that should apply to the same cluster. Watch how they are all successfully applied, but the ClusterResourceSetBinding for the cluster receives multiple updates that overwrite each other, resulting inaccurate data in ClusterResourceSetBinding. Also ownerReferences are also inaccurately registered, again due to being overwritten.

This seemed especially apparent when there was network latency between the management cluster and the workload cluster.

What did you expect to happen?

The ClusterResourceSetBinding should contain the correct state of all applied ClusterResourceSets for the relevant Cluster.

Cluster API version

v1.7.2

Kubernetes version

v1.29.4

Anything else you would like to add?

From https://kubernetes.slack.com/archives/C8TSNPY4T/p1716231649471649:

CRS which says it has been applied, cluster matches correctly, etc

apiVersion: addons.cluster.x-k8s.io/v1beta1
kind: ClusterResourceSet
metadata:
  creationTimestamp: "2024-05-20T18:26:47Z"
  finalizers:
  - addons.cluster.x-k8s.io
  generation: 1
  name: aws-ccm-v1.29.2-quick-start-z87nz9
  namespace: quick-start-q6ijll
  ownerReferences:
  - apiVersion: cluster.x-k8s.io/v1beta1
    kind: Cluster
    name: quick-start-z87nz9
    uid: cbb1ba25-e189-4712-8f43-e18acb81f66b
  resourceVersion: "3667"
  uid: bb63ba91-4d82-4380-93e2-45f54adfb25b
spec:
  clusterSelector:
    matchLabels:
      cluster.x-k8s.io/cluster-name: quick-start-z87nz9
  resources:
  - kind: ConfigMap
    name: aws-ccm-v1.29.2-quick-start-z87nz9
  strategy: Reconcile
status:
  conditions:
  - lastTransitionTime: "2024-05-20T18:26:51Z"
    status: "True"
    type: ResourcesApplied
  observedGeneration: 1

CRSBinding, missing binding and ownerRefs for this CRS aws-ccm-v1.29.2-quick-start-z87nz9 (other ones are correct, plus there are others missing):

apiVersion: addons.cluster.x-k8s.io/v1beta1
kind: ClusterResourceSetBinding
metadata:
  creationTimestamp: "2024-05-20T18:26:47Z"
  generation: 10
  name: quick-start-z87nz9
  namespace: quick-start-q6ijll
  ownerReferences:
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: aws-ccm-v1.29.2-quick-start-z87nz9
    uid: bb63ba91-4d82-4380-93e2-45f54adfb25b
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: calico-cni-installation-quick-start-z87nz9
    uid: 975eacbd-8f4b-48ba-8166-e7cadfabbbb8
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: aws-ebs-csi-quick-start-z87nz9
    uid: 77fd0fd2-8d3e-43a7-beb9-a63fba7ac916
  resourceVersion: "4071"
  uid: e36e93c5-9069-4767-a3b1-085c46183be1
spec:
  bindings:
  - clusterResourceSetName: calico-cni-installation-quick-start-z87nz9
    resources:
    - applied: true
      hash: sha256:627af2eec3b7a4d3dd925573294e2834552a7803ee008b34fd9edd6d9ae5caecist
      kind: ConfigMap
      lastAppliedTime: "2024-05-20T18:26:57Z"
      name: tigera-operator-quick-start-z87nz9
    - applied: true
      hash: sha256:f7a18aac5844863a256e0937a8c24cf41e88817bb31d28a68f4314fb7caa27ee
      kind: ConfigMap
      lastAppliedTime: "2024-05-20T18:26:57Z"
      name: calico-cni-installation-quick-start-z87nz9
  - clusterResourceSetName: aws-ebs-csi-quick-start-z87nz9
    resources:
    - applied: true
      hash: sha256:eeda97705092e972f72d7e79591bb8b89861dcc996caffaf75b2e266816f2a3adcf241d5512d1ab93833f332e31
      kind: ConfigMap
      lastAppliedTime: "2024-05-20T18:28:12Z"
      name: aws-ebs-csi-quick-start-z87nz9
  clusterName: quick-start-z87nz9

trace for updates to the CRSBinding (each resource output is the same but for each update):

apiVersion: addons.cluster.x-k8s.io/v1beta1
kind: ClusterResourceSetBinding
metadata:
  creationTimestamp: "2024-05-20T19:24:49Z"
  generation: 1
  name: quick-start-jeemv9
  namespace: quick-start-7l4lue
  ownerReferences:
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: aws-ebs-csi-quick-start-jeemv9
    uid: 1f6d6446-5e44-4e07-aec6-8674055595b1
  resourceVersion: "3600"
  uid: 042c69bf-456f-4142-9f74-00a04f94bf2f
spec:
  clusterName: quick-start-jeemv9
---
apiVersion: addons.cluster.x-k8s.io/v1beta1
kind: ClusterResourceSetBinding
metadata:
  creationTimestamp: "2024-05-20T19:24:49Z"
  generation: 2
  name: quick-start-jeemv9
  namespace: quick-start-7l4lue
  ownerReferences:
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: aws-ebs-csi-quick-start-jeemv9
    uid: 1f6d6446-5e44-4e07-aec6-8674055595b1
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: cluster-autoscaler-quick-start-jeemv9
    uid: 81dae048-9609-42fc-8f10-102f662e833b
  resourceVersion: "3638"
  uid: 042c69bf-456f-4142-9f74-00a04f94bf2f
spec:
  bindings:
  - clusterResourceSetName: cluster-autoscaler-quick-start-jeemv9
    resources:
    - applied: true
      hash: sha256:afc87ddd3b3243e8abebfa61454c47fe4f164d1e0c6e9affffd80d10dd347e44
      kind: ConfigMap
      lastAppliedTime: "2024-05-20T19:24:53Z"
      name: cluster-autoscaler-quick-start-jeemv9
  clusterName: quick-start-jeemv9
---
apiVersion: addons.cluster.x-k8s.io/v1beta1
kind: ClusterResourceSetBinding
metadata:
  creationTimestamp: "2024-05-20T19:24:49Z"
  generation: 3
  name: quick-start-jeemv9
  namespace: quick-start-7l4lue
  ownerReferences:
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: aws-ebs-csi-quick-start-jeemv9
    uid: 1f6d6446-5e44-4e07-aec6-8674055595b1
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: aws-ccm-v1.29.2-quick-start-jeemv9
    uid: 3feb4dce-4a80-40a5-8148-7d69b60d69be
  resourceVersion: "3640"
  uid: 042c69bf-456f-4142-9f74-00a04f94bf2f
spec:
  bindings:
  - clusterResourceSetName: aws-ccm-v1.29.2-quick-start-jeemv9
    resources:
    - applied: true
      hash: sha256:6a16618ec3ee8a84aa24d7ea22fe344fd6a73ea6b0c973c9491022e9abb6049d
      kind: ConfigMap
      lastAppliedTime: "2024-05-20T19:24:53Z"
      name: aws-ccm-v1.29.2-quick-start-jeemv9
  clusterName: quick-start-jeemv9
---
apiVersion: addons.cluster.x-k8s.io/v1beta1
kind: ClusterResourceSetBinding
metadata:
  creationTimestamp: "2024-05-20T19:24:49Z"
  generation: 4
  name: quick-start-jeemv9
  namespace: quick-start-7l4lue
  ownerReferences:
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: aws-ebs-csi-quick-start-jeemv9
    uid: 1f6d6446-5e44-4e07-aec6-8674055595b1
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: aws-ccm-v1.29.2-quick-start-jeemv9
    uid: 3feb4dce-4a80-40a5-8148-7d69b60d69be
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: cluster-autoscaler-quick-start-jeemv9
    uid: 81dae048-9609-42fc-8f10-102f662e833b
  resourceVersion: "3648"
  uid: 042c69bf-456f-4142-9f74-00a04f94bf2f
spec:
  bindings:
  - clusterResourceSetName: aws-ccm-v1.29.2-quick-start-jeemv9
    resources:
    - applied: true
      hash: sha256:6a16618ec3ee8a84aa24d7ea22fe344fd6a73ea6b0c973c9491022e9abb6049d
      kind: ConfigMap
      lastAppliedTime: "2024-05-20T19:24:53Z"
      name: aws-ccm-v1.29.2-quick-start-jeemv9
  - clusterResourceSetName: cluster-autoscaler-quick-start-jeemv9
    resources:
    - applied: true
      hash: sha256:afc87ddd3b3243e8abebfa61454c47fe4f164d1e0c6e9affffd80d10dd347e44
      kind: ConfigMap
      lastAppliedTime: "2024-05-20T19:24:54Z"
      name: cluster-autoscaler-quick-start-jeemv9
  clusterName: quick-start-jeemv9
---
apiVersion: addons.cluster.x-k8s.io/v1beta1
kind: ClusterResourceSetBinding
metadata:
  creationTimestamp: "2024-05-20T19:24:49Z"
  generation: 5
  name: quick-start-jeemv9
  namespace: quick-start-7l4lue
  ownerReferences:
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: aws-ebs-csi-quick-start-jeemv9
    uid: 1f6d6446-5e44-4e07-aec6-8674055595b1
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: node-feature-discovery-quick-start-jeemv9
    uid: 9ec6e529-ce87-4f78-b7db-3da357b173a8
  resourceVersion: "3649"
  uid: 042c69bf-456f-4142-9f74-00a04f94bf2f
spec:
  bindings:
  - clusterResourceSetName: node-feature-discovery-quick-start-jeemv9
    resources:
    - applied: true
      hash: sha256:5c965730f1982a9093e019186c6b6a03bd5644e9d516c2ab19d6bad14885a98d
      kind: ConfigMap
      lastAppliedTime: "2024-05-20T19:24:55Z"
      name: node-feature-discovery-quick-start-jeemv9
  clusterName: quick-start-jeemv9
---
apiVersion: addons.cluster.x-k8s.io/v1beta1
kind: ClusterResourceSetBinding
metadata:
  creationTimestamp: "2024-05-20T19:24:49Z"
  generation: 6
  name: quick-start-jeemv9
  namespace: quick-start-7l4lue
  ownerReferences:
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: aws-ebs-csi-quick-start-jeemv9
    uid: 1f6d6446-5e44-4e07-aec6-8674055595b1
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: node-feature-discovery-quick-start-jeemv9
    uid: 9ec6e529-ce87-4f78-b7db-3da357b173a8
  resourceVersion: "3660"
  uid: 042c69bf-456f-4142-9f74-00a04f94bf2f
spec:
  bindings:
  - clusterResourceSetName: aws-ebs-csi-quick-start-jeemv9
    resources:
    - applied: true
      hash: sha256:eeda97705092e972f72d7e79591bb8b89861dcc996caffaf75b2e266816f2a3a
      kind: ConfigMap
      lastAppliedTime: "2024-05-20T19:24:56Z"
      name: aws-ebs-csi-quick-start-jeemv9
  clusterName: quick-start-jeemv9
---
apiVersion: addons.cluster.x-k8s.io/v1beta1
kind: ClusterResourceSetBinding
metadata:
  creationTimestamp: "2024-05-20T19:24:49Z"
  generation: 7
  name: quick-start-jeemv9
  namespace: quick-start-7l4lue
  ownerReferences:
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: aws-ebs-csi-quick-start-jeemv9
    uid: 1f6d6446-5e44-4e07-aec6-8674055595b1
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: calico-cni-installation-quick-start-jeemv9
    uid: aa174738-318f-4fb5-8944-be69a0bf4289
  resourceVersion: "3666"
  uid: 042c69bf-456f-4142-9f74-00a04f94bf2f
spec:
  bindings:
  - clusterResourceSetName: calico-cni-installation-quick-start-jeemv9
    resources:
    - applied: true
      hash: sha256:627af2eec3b7a4d3dd925573294e2834552a7803ee008b34fd9edd6d9ae5caec
      kind: ConfigMap
      lastAppliedTime: "2024-05-20T19:24:57Z"
      name: tigera-operator-quick-start-jeemv9
    - applied: true
      hash: sha256:f7a18aac5844863a256e0937a8c24cf41e88817bb31d28a68f4314fb7caa27ee
      kind: ConfigMap
      lastAppliedTime: "2024-05-20T19:24:57Z"
      name: calico-cni-installation-quick-start-jeemv9
  clusterName: quick-start-jeemv9
---
apiVersion: addons.cluster.x-k8s.io/v1beta1
kind: ClusterResourceSetBinding
metadata:
  creationTimestamp: "2024-05-20T19:24:49Z"
  generation: 8
  name: quick-start-jeemv9
  namespace: quick-start-7l4lue
  ownerReferences:
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: aws-ebs-csi-quick-start-jeemv9
    uid: 1f6d6446-5e44-4e07-aec6-8674055595b1
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: calico-cni-installation-quick-start-jeemv9
    uid: aa174738-318f-4fb5-8944-be69a0bf4289
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: aws-ccm-v1.29.2-quick-start-jeemv9
    uid: 3feb4dce-4a80-40a5-8148-7d69b60d69be
  resourceVersion: "3984"
  uid: 042c69bf-456f-4142-9f74-00a04f94bf2f
spec:
  bindings:
  - clusterResourceSetName: calico-cni-installation-quick-start-jeemv9
    resources:
    - applied: true
      hash: sha256:627af2eec3b7a4d3dd925573294e2834552a7803ee008b34fd9edd6d9ae5caec
      kind: ConfigMap
      lastAppliedTime: "2024-05-20T19:24:57Z"
      name: tigera-operator-quick-start-jeemv9
    - applied: true
      hash: sha256:f7a18aac5844863a256e0937a8c24cf41e88817bb31d28a68f4314fb7caa27ee
      kind: ConfigMap
      lastAppliedTime: "2024-05-20T19:24:57Z"
      name: calico-cni-installation-quick-start-jeemv9
  - clusterResourceSetName: aws-ccm-v1.29.2-quick-start-jeemv9
    resources:
    - applied: true
      hash: sha256:6a16618ec3ee8a84aa24d7ea22fe344fd6a73ea6b0c973c9491022e9abb6049d
      kind: ConfigMap
      lastAppliedTime: "2024-05-20T19:26:03Z"
      name: aws-ccm-v1.29.2-quick-start-jeemv9
  clusterName: quick-start-jeemv9
---
apiVersion: addons.cluster.x-k8s.io/v1beta1
kind: ClusterResourceSetBinding
metadata:
  creationTimestamp: "2024-05-20T19:24:49Z"
  generation: 9
  name: quick-start-jeemv9
  namespace: quick-start-7l4lue
  ownerReferences:
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: aws-ebs-csi-quick-start-jeemv9
    uid: 1f6d6446-5e44-4e07-aec6-8674055595b1
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: calico-cni-installation-quick-start-jeemv9
    uid: aa174738-318f-4fb5-8944-be69a0bf4289
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: cluster-autoscaler-quick-start-jeemv9
    uid: 81dae048-9609-42fc-8f10-102f662e833b
  resourceVersion: "3985"
  uid: 042c69bf-456f-4142-9f74-00a04f94bf2f
spec:
  bindings:
  - clusterResourceSetName: calico-cni-installation-quick-start-jeemv9
    resources:
    - applied: true
      hash: sha256:627af2eec3b7a4d3dd925573294e2834552a7803ee008b34fd9edd6d9ae5caec
      kind: ConfigMap
      lastAppliedTime: "2024-05-20T19:24:57Z"
      name: tigera-operator-quick-start-jeemv9
    - applied: true
      hash: sha256:f7a18aac5844863a256e0937a8c24cf41e88817bb31d28a68f4314fb7caa27ee
      kind: ConfigMap
      lastAppliedTime: "2024-05-20T19:24:57Z"
      name: calico-cni-installation-quick-start-jeemv9
  - clusterResourceSetName: cluster-autoscaler-quick-start-jeemv9
    resources:
    - applied: true
      hash: sha256:afc87ddd3b3243e8abebfa61454c47fe4f164d1e0c6e9affffd80d10dd347e44
      kind: ConfigMap
      lastAppliedTime: "2024-05-20T19:26:03Z"
      name: cluster-autoscaler-quick-start-jeemv9
  clusterName: quick-start-jeemv9
---
apiVersion: addons.cluster.x-k8s.io/v1beta1
kind: ClusterResourceSetBinding
metadata:
  creationTimestamp: "2024-05-20T19:24:49Z"
  generation: 10
  name: quick-start-jeemv9
  namespace: quick-start-7l4lue
  ownerReferences:
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: aws-ebs-csi-quick-start-jeemv9
    uid: 1f6d6446-5e44-4e07-aec6-8674055595b1
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: calico-cni-installation-quick-start-jeemv9
    uid: aa174738-318f-4fb5-8944-be69a0bf4289
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: node-feature-discovery-quick-start-jeemv9
    uid: 9ec6e529-ce87-4f78-b7db-3da357b173a8
  resourceVersion: "4001"
  uid: 042c69bf-456f-4142-9f74-00a04f94bf2f
spec:
  bindings:
  - clusterResourceSetName: calico-cni-installation-quick-start-jeemv9
    resources:
    - applied: true
      hash: sha256:627af2eec3b7a4d3dd925573294e2834552a7803ee008b34fd9edd6d9ae5caec
      kind: ConfigMap
      lastAppliedTime: "2024-05-20T19:24:57Z"
      name: tigera-operator-quick-start-jeemv9
    - applied: true
      hash: sha256:f7a18aac5844863a256e0937a8c24cf41e88817bb31d28a68f4314fb7caa27ee
      kind: ConfigMap
      lastAppliedTime: "2024-05-20T19:24:57Z"
      name: calico-cni-installation-quick-start-jeemv9
  - clusterResourceSetName: node-feature-discovery-quick-start-jeemv9
    resources:
    - applied: true
      hash: sha256:5c965730f1982a9093e019186c6b6a03bd5644e9d516c2ab19d6bad14885a98d
      kind: ConfigMap
      lastAppliedTime: "2024-05-20T19:26:06Z"
      name: node-feature-discovery-quick-start-jeemv9
  clusterName: quick-start-jeemv9
---
apiVersion: addons.cluster.x-k8s.io/v1beta1
kind: ClusterResourceSetBinding
metadata:
  creationTimestamp: "2024-05-20T19:24:49Z"
  generation: 11
  name: quick-start-jeemv9
  namespace: quick-start-7l4lue
  ownerReferences:
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: aws-ebs-csi-quick-start-jeemv9
    uid: 1f6d6446-5e44-4e07-aec6-8674055595b1
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: calico-cni-installation-quick-start-jeemv9
    uid: aa174738-318f-4fb5-8944-be69a0bf4289
  - apiVersion: addons.cluster.x-k8s.io/v1beta1
    kind: ClusterResourceSet
    name: node-feature-discovery-quick-start-jeemv9
    uid: 9ec6e529-ce87-4f78-b7db-3da357b173a8
  resourceVersion: "4008"
  uid: 042c69bf-456f-4142-9f74-00a04f94bf2f
spec:
  bindings:
  - clusterResourceSetName: calico-cni-installation-quick-start-jeemv9
    resources:
    - applied: true
      hash: sha256:627af2eec3b7a4d3dd925573294e2834552a7803ee008b34fd9edd6d9ae5caec
      kind: ConfigMap
      lastAppliedTime: "2024-05-20T19:24:57Z"
      name: tigera-operator-quick-start-jeemv9
    - applied: true
      hash: sha256:f7a18aac5844863a256e0937a8c24cf41e88817bb31d28a68f4314fb7caa27ee
      kind: ConfigMap
      lastAppliedTime: "2024-05-20T19:24:57Z"
      name: calico-cni-installation-quick-start-jeemv9
  - clusterResourceSetName: aws-ebs-csi-quick-start-jeemv9
    resources:
    - applied: true
      hash: sha256:eeda97705092e972f72d7e79591bb8b89861dcc996caffaf75b2e266816f2a3a
      kind: ConfigMap
      lastAppliedTime: "2024-05-20T19:26:07Z"
      name: aws-ebs-csi-quick-start-jeemv9
  clusterName: quick-start-jeemv9

Label(s) to be applied

/kind bug
/area clusterresourceset

@k8s-ci-robot k8s-ci-robot added kind/bug Categorizes issue or PR as related to a bug. area/clusterresourceset Issues or PRs related to clusterresourcesets needs-priority Indicates an issue lacks a `priority/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels May 21, 2024
@fabriziopandini
Copy link
Member

/priority backlog
/triage accepted
given the linked PR

@k8s-ci-robot k8s-ci-robot added priority/backlog Higher priority than priority/awaiting-more-evidence. triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed needs-priority Indicates an issue lacks a `priority/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels May 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/clusterresourceset Issues or PRs related to clusterresourcesets kind/bug Categorizes issue or PR as related to a bug. priority/backlog Higher priority than priority/awaiting-more-evidence. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants