Uninstall does not delete Image automation CRD

[danielkimuipath]

Hi, I'm using Image automation CRDs in my cluster. My cluster is has a set up to fetch images from Azure ACR and there was an issue with fetching the images. So I tried to uninstall the flux and reinstallling the flux, but flux uninstall command does not clean up the image automation ACRs.

flux uninstall
Are you sure you want to delete Flux and its custom resource definitions: y
► deleting components in flux-system namespace
► deleting toolkit.fluxcd.io finalizers in all namespaces
► deleting toolkit.fluxcd.io custom resource definitions
✔ CustomResourceDefinition/imagepolicies.image.toolkit.fluxcd.io deleted
✔ CustomResourceDefinition/imagerepositories.image.toolkit.fluxcd.io deleted
✔ Namespace/flux-system deleted
✔ uninstall finished

after the delete

k get crd -A
...
imagepolicies.image.toolkit.fluxcd.io            2022-09-08T22:32:15Z
imagerepositories.image.toolkit.fluxcd.io        2022-09-08T22:32:15Z

Even I tried to force delete, it still does not work:

 k delete crd imagepolicies.image.toolkit.fluxcd.io --force --grace-period=0
warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
customresourcedefinition.apiextensions.k8s.io "imagepolicies.image.toolkit.fluxcd.io" force deleted

After that, it still shows up

k get crd -A
NAME                                             CREATED AT
..
imagepolicies.image.toolkit.fluxcd.io            2022-09-08T22:32:15Z
imagerepositories.image.toolkit.fluxcd.io        2022-09-08T22:32:15Z
...

Can someone help me how to clean up the CRD properly?

[danielkimuipath]

Describe crd:

k describe crd imagepolicies.image.toolkit.fluxcd.io
Name:         imagepolicies.image.toolkit.fluxcd.io
Namespace:
Labels:       app.kubernetes.io/instance=flux-system
              app.kubernetes.io/part-of=flux
              app.kubernetes.io/version=v0.32.0
              kustomize.toolkit.fluxcd.io/name=flux-system
              kustomize.toolkit.fluxcd.io/namespace=flux-system
Annotations:  controller-gen.kubebuilder.io/version: v0.7.0
API Version:  apiextensions.k8s.io/v1
Kind:         CustomResourceDefinition
Metadata:
  Creation Timestamp:             2022-09-08T22:32:15Z
  Deletion Grace Period Seconds:  0
  Deletion Timestamp:             2022-09-09T17:35:44Z
  Finalizers:
    customresourcecleanup.apiextensions.k8s.io
  Generation:  1
  Managed Fields:
    API Version:  apiextensions.k8s.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          f:controller-gen.kubebuilder.io/version:
        f:labels:
          f:app.kubernetes.io/instance:
          f:app.kubernetes.io/part-of:
          f:app.kubernetes.io/version:
          f:kustomize.toolkit.fluxcd.io/name:
          f:kustomize.toolkit.fluxcd.io/namespace:
      f:spec:
        f:group:
        f:names:
          f:kind:
          f:listKind:
          f:plural:
          f:singular:
        f:scope:
        f:versions:
    Manager:      kustomize-controller
    Operation:    Apply
    Time:         2022-09-08T22:32:36Z
    API Version:  apiextensions.k8s.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:controller-gen.kubebuilder.io/version:
        f:labels:
          .:
          f:app.kubernetes.io/instance:
          f:app.kubernetes.io/part-of:
          f:app.kubernetes.io/version:
      f:spec:
        f:conversion:
          .:
          f:strategy:
        f:group:
        f:names:
          f:kind:
          f:listKind:
          f:plural:
          f:singular:
        f:scope:
        f:versions:
    Manager:      HashiCorp
    Operation:    Update
    Time:         2022-09-08T22:32:15Z
    API Version:  apiextensions.k8s.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        f:acceptedNames:
          f:kind:
          f:listKind:
          f:plural:
          f:singular:
        f:conditions:
          k:{"type":"Established"}:
            .:
            f:lastTransitionTime:
            f:message:
            f:reason:
            f:status:
            f:type:
          k:{"type":"NamesAccepted"}:
            .:
            f:lastTransitionTime:
            f:message:
            f:reason:
            f:status:
            f:type:
          k:{"type":"Terminating"}:
            f:message:
            f:reason:
    Manager:         Go-http-client
    Operation:       Update
    Subresource:     status
    Time:            2022-09-09T17:35:44Z
  Resource Version:  4779062
  UID:               b50d0c66-d623-4f8a-8033-d04504c411d7
Spec:
  Conversion:
    Strategy:  None
  Group:       image.toolkit.fluxcd.io
  Names:
    Kind:       ImagePolicy
    List Kind:  ImagePolicyList
    Plural:     imagepolicies
    Singular:   imagepolicy
  Scope:        Namespaced
  Versions:
    Additional Printer Columns:
      Json Path:  .status.latestImage
      Name:       LatestImage
      Type:       string
    Name:         v1alpha1
    Schema:
      openAPIV3Schema:
        Description:  ImagePolicy is the Schema for the imagepolicies API
        Properties:
          API Version:
            Description:  APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
            Type:         string
          Kind:
            Description:  Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
            Type:         string
          Metadata:
            Type:  object
          Spec:
            Description:  ImagePolicySpec defines the parameters for calculating the ImagePolicy
            Properties:
              Filter Tags:
                Description:  FilterTags enables filtering for only a subset of tags based on a set of rules. If no rules are provided, all the tags from the repository will be ordered and compared.
                Properties:
                  Extract:
                    Description:  Extract allows a capture group to be extracted from the specified regular expression pattern, useful before tag evaluation.
                    Type:         string
                  Pattern:
                    Description:  Pattern specifies a regular expression pattern used to filter for image tags.
                    Type:         string
                Type:             object
              Image Repository Ref:
                Description:  ImageRepositoryRef points at the object specifying the image being scanned
                Properties:
                  Name:
                    Description:  Name of the referent.
                    Type:         string
                Required:
                  name
                Type:  object
              Policy:
                Description:  Policy gives the particulars of the policy to be followed in selecting the most recent image
                Properties:
                  Alphabetical:
                    Description:  Alphabetical set of rules to use for alphabetical ordering of the tags.
                    Properties:
                      Order:
                        Default:      asc
                        Description:  Order specifies the sorting order of the tags. Given the letters of the alphabet as tags, ascending order would select Z, and descending order would select A.
                        Enum:
                          asc
                          desc
                        Type:  string
                    Type:      object
                  Numerical:
                    Description:  Numerical set of rules to use for numerical ordering of the tags.
                    Properties:
                      Order:
                        Default:      asc
                        Description:  Order specifies the sorting order of the tags. Given the integer values from 0 to 9 as tags, ascending order would select 9, and descending order would select 0.
                        Enum:
                          asc
                          desc
                        Type:  string
                    Type:      object
                  Semver:
                    Description:  SemVer gives a semantic version range to check against the tags available.
                    Properties:
                      Range:
                        Description:  Range gives a semver range for the image tag; the highest version within the range that's a tag yields the latest image.
                        Type:         string
                    Required:
                      range
                    Type:  object
                Type:      object
            Required:
              imageRepositoryRef
              policy
            Type:  object
          Status:
            Description:  ImagePolicyStatus defines the observed state of ImagePolicy
            Properties:
              Conditions:
                Items:
                  Description:  Condition contains details for one aspect of the current state of this API Resource. --- This struct is intended for direct use as an array at the field path .status.conditions.  For example, type FooStatus struct{     // Represents the observations of a foo's current state.     // Known .status.conditions.type are: "Available", "Progressing", and "Degraded"     // +patchMergeKey=type     // +patchStrategy=merge     // +listType=map     // +listMapKey=type     Conditions []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,1,rep,name=conditions"`
     // other fields }
                  Properties:
                    Last Transition Time:
                      Description:  lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed.  If that is not known, then using the time when the API field changed is acceptable.
                      Format:       date-time
                      Type:         string
                    Message:
                      Description:  message is a human readable message indicating details about the transition. This may be an empty string.
                      Max Length:   32768
                      Type:         string
                    Observed Generation:
                      Description:  observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance.
                      Format:       int64
                      Minimum:      0
                      Type:         integer
                    Reason:
                      Description:  reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty.
                      Max Length:   1024
                      Min Length:   1
                      Pattern:      ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$
                      Type:         string
                    Status:
                      Description:  status of the condition, one of True, False, Unknown.
                      Enum:
                        True
                        False
                        Unknown
                      Type:  string
                    Type:
                      Description:  type of condition in CamelCase or in foo.example.com/CamelCase. --- Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be useful (see .node.status.conditions), the ability to deconflict is important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)
                      Max Length:   316
                      Pattern:      ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$
                      Type:         string
                  Required:
                    lastTransitionTime
                    message
                    reason
                    status
                    type
                  Type:  object
                Type:    array
              Latest Image:
                Description:  LatestImage gives the first in the list of images scanned by the image repository, when filtered and ordered according to the policy.
                Type:         string
              Observed Generation:
                Format:  int64
                Type:    integer
            Type:        object
        Type:            object
    Served:              true
    Storage:             false
    Subresources:
      Status:
    Additional Printer Columns:
      Json Path:  .status.latestImage
      Name:       LatestImage
      Type:       string
    Name:         v1alpha2
    Schema:
      openAPIV3Schema:
        Description:  ImagePolicy is the Schema for the imagepolicies API
        Properties:
          API Version:
            Description:  APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
            Type:         string
          Kind:
            Description:  Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
            Type:         string
          Metadata:
            Type:  object
          Spec:
            Description:  ImagePolicySpec defines the parameters for calculating the ImagePolicy
            Properties:
              Filter Tags:
                Description:  FilterTags enables filtering for only a subset of tags based on a set of rules. If no rules are provided, all the tags from the repository will be ordered and compared.
                Properties:
                  Extract:
                    Description:  Extract allows a capture group to be extracted from the specified regular expression pattern, useful before tag evaluation.
                    Type:         string
                  Pattern:
                    Description:  Pattern specifies a regular expression pattern used to filter for image tags.
                    Type:         string
                Type:             object
              Image Repository Ref:
                Description:  ImageRepositoryRef points at the object specifying the image being scanned
                Properties:
                  Name:
                    Description:  Name of the referent.
                    Type:         string
                Required:
                  name
                Type:  object
              Policy:
                Description:  Policy gives the particulars of the policy to be followed in selecting the most recent image
                Properties:
                  Alphabetical:
                    Description:  Alphabetical set of rules to use for alphabetical ordering of the tags.
                    Properties:
                      Order:
                        Default:      asc
                        Description:  Order specifies the sorting order of the tags. Given the letters of the alphabet as tags, ascending order would select Z, and descending order would select A.
                        Enum:
                          asc
                          desc
                        Type:  string
                    Type:      object
                  Numerical:
                    Description:  Numerical set of rules to use for numerical ordering of the tags.
                    Properties:
                      Order:
                        Default:      asc
                        Description:  Order specifies the sorting order of the tags. Given the integer values from 0 to 9 as tags, ascending order would select 9, and descending order would select 0.
                        Enum:
                          asc
                          desc
                        Type:  string
                    Type:      object
                  Semver:
                    Description:  SemVer gives a semantic version range to check against the tags available.
                    Properties:
                      Range:
                        Description:  Range gives a semver range for the image tag; the highest version within the range that's a tag yields the latest image.
                        Type:         string
                    Required:
                      range
                    Type:  object
                Type:      object
            Required:
              imageRepositoryRef
              policy
            Type:  object
          Status:
            Description:  ImagePolicyStatus defines the observed state of ImagePolicy
            Properties:
              Conditions:
                Items:
                  Description:  Condition contains details for one aspect of the current state of this API Resource. --- This struct is intended for direct use as an array at the field path .status.conditions.  For example, type FooStatus struct{     // Represents the observations of a foo's current state.     // Known .status.conditions.type are: "Available", "Progressing", and "Degraded"     // +patchMergeKey=type     // +patchStrategy=merge     // +listType=map     // +listMapKey=type     Conditions []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,1,rep,name=conditions"`
     // other fields }
                  Properties:
                    Last Transition Time:
                      Description:  lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed.  If that is not known, then using the time when the API field changed is acceptable.
                      Format:       date-time
                      Type:         string
                    Message:
                      Description:  message is a human readable message indicating details about the transition. This may be an empty string.
                      Max Length:   32768
                      Type:         string
                    Observed Generation:
                      Description:  observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance.
                      Format:       int64
                      Minimum:      0
                      Type:         integer
                    Reason:
                      Description:  reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty.
                      Max Length:   1024
                      Min Length:   1
                      Pattern:      ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$
                      Type:         string
                    Status:
                      Description:  status of the condition, one of True, False, Unknown.
                      Enum:
                        True
                        False
                        Unknown
                      Type:  string
                    Type:
                      Description:  type of condition in CamelCase or in foo.example.com/CamelCase. --- Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be useful (see .node.status.conditions), the ability to deconflict is important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)
                      Max Length:   316
                      Pattern:      ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$
                      Type:         string
                  Required:
                    lastTransitionTime
                    message
                    reason
                    status
                    type
                  Type:  object
                Type:    array
              Latest Image:
                Description:  LatestImage gives the first in the list of images scanned by the image repository, when filtered and ordered according to the policy.
                Type:         string
              Observed Generation:
                Format:  int64
                Type:    integer
            Type:        object
        Type:            object
    Served:              true
    Storage:             false
    Subresources:
      Status:
    Additional Printer Columns:
      Json Path:  .status.latestImage
      Name:       LatestImage
      Type:       string
    Name:         v1beta1
    Schema:
      openAPIV3Schema:
        Description:  ImagePolicy is the Schema for the imagepolicies API
        Properties:
          API Version:
            Description:  APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
            Type:         string
          Kind:
            Description:  Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
            Type:         string
          Metadata:
            Type:  object
          Spec:
            Description:  ImagePolicySpec defines the parameters for calculating the ImagePolicy
            Properties:
              Filter Tags:
                Description:  FilterTags enables filtering for only a subset of tags based on a set of rules. If no rules are provided, all the tags from the repository will be ordered and compared.
                Properties:
                  Extract:
                    Description:  Extract allows a capture group to be extracted from the specified regular expression pattern, useful before tag evaluation.
                    Type:         string
                  Pattern:
                    Description:  Pattern specifies a regular expression pattern used to filter for image tags.
                    Type:         string
                Type:             object
              Image Repository Ref:
                Description:  ImageRepositoryRef points at the object specifying the image being scanned
                Properties:
                  Name:
                    Description:  Name of the referent.
                    Type:         string
                  Namespace:
                    Description:  Namespace of the referent, when not specified it acts as LocalObjectReference.
                    Type:         string
                Required:
                  name
                Type:  object
              Policy:
                Description:  Policy gives the particulars of the policy to be followed in selecting the most recent image
                Properties:
                  Alphabetical:
                    Description:  Alphabetical set of rules to use for alphabetical ordering of the tags.
                    Properties:
                      Order:
                        Default:      asc
                        Description:  Order specifies the sorting order of the tags. Given the letters of the alphabet as tags, ascending order would select Z, and descending order would select A.
                        Enum:
                          asc
                          desc
                        Type:  string
                    Type:      object
                  Numerical:
                    Description:  Numerical set of rules to use for numerical ordering of the tags.
                    Properties:
                      Order:
                        Default:      asc
                        Description:  Order specifies the sorting order of the tags. Given the integer values from 0 to 9 as tags, ascending order would select 9, and descending order would select 0.
                        Enum:
                          asc
                          desc
                        Type:  string
                    Type:      object
                  Semver:
                    Description:  SemVer gives a semantic version range to check against the tags available.
                    Properties:
                      Range:
                        Description:  Range gives a semver range for the image tag; the highest version within the range that's a tag yields the latest image.
                        Type:         string
                    Required:
                      range
                    Type:  object
                Type:      object
            Required:
              imageRepositoryRef
              policy
            Type:  object
          Status:
            Default:
              Observed Generation:  -1
            Description:            ImagePolicyStatus defines the observed state of ImagePolicy
            Properties:
              Conditions:
                Items:
                  Description:  Condition contains details for one aspect of the current state of this API Resource. --- This struct is intended for direct use as an array at the field path .status.conditions.  For example, type FooStatus struct{     // Represents the observations of a foo's current state.     // Known .status.conditions.type are: "Available", "Progressing", and "Degraded"     // +patchMergeKey=type     // +patchStrategy=merge     // +listType=map     // +listMapKey=type     Conditions []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,1,rep,name=conditions"`
     // other fields }
                  Properties:
                    Last Transition Time:
                      Description:  lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed.  If that is not known, then using the time when the API field changed is acceptable.
                      Format:       date-time
                      Type:         string
                    Message:
                      Description:  message is a human readable message indicating details about the transition. This may be an empty string.
                      Max Length:   32768
                      Type:         string
                    Observed Generation:
                      Description:  observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance.
                      Format:       int64
                      Minimum:      0
                      Type:         integer
                    Reason:
                      Description:  reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty.
                      Max Length:   1024
                      Min Length:   1
                      Pattern:      ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$
                      Type:         string
                    Status:
                      Description:  status of the condition, one of True, False, Unknown.
                      Enum:
                        True
                        False
                        Unknown
                      Type:  string
                    Type:
                      Description:  type of condition in CamelCase or in foo.example.com/CamelCase. --- Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be useful (see .node.status.conditions), the ability to deconflict is important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)
                      Max Length:   316
                      Pattern:      ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$
                      Type:         string
                  Required:
                    lastTransitionTime
                    message
                    reason
                    status
                    type
                  Type:  object
                Type:    array
              Latest Image:
                Description:  LatestImage gives the first in the list of images scanned by the image repository, when filtered and ordered according to the policy.
                Type:         string
              Observed Generation:
                Format:  int64
                Type:    integer
            Type:        object
        Type:            object
    Served:              true
    Storage:             true
    Subresources:
      Status:
Status:
  Accepted Names:
    Kind:       ImagePolicy
    List Kind:  ImagePolicyList
    Plural:     imagepolicies
    Singular:   imagepolicy
  Conditions:
    Last Transition Time:  2022-09-08T22:32:15Z
    Message:               no conflicts found
    Reason:                NoConflicts
    Status:                True
    Type:                  NamesAccepted
    Last Transition Time:  2022-09-08T22:32:15Z
    Message:               the initial names have been accepted
    Reason:                InitialNamesAccepted
    Status:                True
    Type:                  Established
    Last Transition Time:  2022-09-09T17:35:44Z
    Message:               CustomResource deletion is in progress
    Reason:                InstanceDeletionInProgress
    Status:                True
    Type:                  Terminating
  Stored Versions:
    v1beta1
Events:  <none>

[danielkimuipath]

This seems to be the finalizer setting in the CRD - once the finalizers are removed manually, CRD were cleaned up. However, I believe this needs to be sorted by flux uninstall command

[stefanprodan]

Are you using the latest version of Flux? If not get the latest and see if it fixes the issue for you.

[danielkimuipath]

Here is the flux version of the issue:

flux --version
flux version 0.30.2

flux version
flux: v0.30.2
helm-controller: v0.22.2
image-automation-controller: v0.24.1
image-reflector-controller: v0.20.0
kustomize-controller: v0.27.0
notification-controller: v0.25.1
source-controller: v0.26.1

I will update it with newest version of flux shortly

[danielkimuipath]

k get crd
NAME                                             CREATED AT
alertproviders.flagger.app                       2022-08-31T20:41:08Z
alerts.notification.toolkit.fluxcd.io            2022-09-12T22:47:02Z
authorizationpolicies.security.istio.io          2022-08-31T20:40:38Z
buckets.source.toolkit.fluxcd.io                 2022-09-12T22:47:03Z
canaries.flagger.app                             2022-08-31T20:41:08Z
certificaterequests.cert-manager.io              2022-08-31T20:40:01Z
certificates.cert-manager.io                     2022-08-31T20:40:01Z
challenges.acme.cert-manager.io                  2022-08-31T20:40:02Z
clusterissuers.cert-manager.io                   2022-08-31T20:40:02Z
destinationrules.networking.istio.io             2022-08-31T20:40:38Z
envoyfilters.networking.istio.io                 2022-08-31T20:40:38Z
gateways.networking.istio.io                     2022-08-31T20:40:38Z
gitrepositories.source.toolkit.fluxcd.io         2022-09-12T22:47:01Z
helmcharts.source.toolkit.fluxcd.io              2022-09-12T22:47:01Z
helmreleases.helm.toolkit.fluxcd.io              2022-09-12T22:47:02Z
helmrepositories.source.toolkit.fluxcd.io        2022-09-12T22:47:01Z
imagepolicies.image.toolkit.fluxcd.io            2022-09-12T22:47:02Z
imagerepositories.image.toolkit.fluxcd.io        2022-09-12T22:47:01Z
imageupdateautomations.image.toolkit.fluxcd.io   2022-09-12T22:47:03Z
issuers.cert-manager.io                          2022-08-31T20:40:02Z
istiooperators.install.istio.io                  2022-08-31T20:40:38Z
kustomizations.kustomize.toolkit.fluxcd.io       2022-09-12T22:47:01Z
metrictemplates.flagger.app                      2022-08-31T20:41:08Z
ocirepositories.source.toolkit.fluxcd.io         2022-09-12T22:47:01Z
providers.notification.toolkit.fluxcd.io         2022-09-12T22:47:03Z
receivers.notification.toolkit.fluxcd.io         2022-09-12T22:47:02Z
...

Then I perform deleting by `flux uninstall`:

flux uninstall
Are you sure you want to delete Flux and its custom resource definitions: y
► deleting components in flux-system namespace
✔ Deployment/flux-system/helm-controller deleted
✔ Deployment/flux-system/image-automation-controller deleted
✔ Deployment/flux-system/image-reflector-controller deleted
✔ Deployment/flux-system/kustomize-controller deleted
✔ Deployment/flux-system/notification-controller deleted
✔ Deployment/flux-system/source-controller deleted
✔ Service/flux-system/notification-controller deleted
✔ Service/flux-system/source-controller deleted
✔ Service/flux-system/webhook-receiver deleted
✔ NetworkPolicy/flux-system/allow-egress deleted
✔ NetworkPolicy/flux-system/allow-scraping deleted
✔ NetworkPolicy/flux-system/allow-webhooks deleted
✔ ServiceAccount/flux-system/helm-controller deleted
✔ ServiceAccount/flux-system/image-automation-controller deleted
✔ ServiceAccount/flux-system/image-reflector-controller deleted
✔ ServiceAccount/flux-system/kustomize-controller deleted
✔ ServiceAccount/flux-system/notification-controller deleted
✔ ServiceAccount/flux-system/source-controller deleted
✔ ClusterRole/crd-controller-flux-system deleted
✔ ClusterRoleBinding/cluster-reconciler-flux-system deleted
✔ ClusterRoleBinding/crd-controller-flux-system deleted
► deleting toolkit.fluxcd.io finalizers in all namespaces
✔ GitRepository/flux-system/flux-system finalizers deleted
✔ HelmRepository/flux-system/client finalizers deleted
✔ HelmChart/flux-system/client-client finalizers deleted
✔ Kustomization/flux-system/client finalizers deleted
✔ Kustomization/flux-system/cluster-infrastructure finalizers deleted
✔ Kustomization/flux-system/flux-system finalizers deleted
✔ HelmRelease/client/client finalizers deleted
✔ ImagePolicy/client/client finalizers deleted
✔ ImageRepository/flux-system/client-helm-image-repo finalizers deleted
✔ ImageRepository/flux-system/client-image-repo finalizers deleted
► deleting toolkit.fluxcd.io custom resource definitions
✔ CustomResourceDefinition/alerts.notification.toolkit.fluxcd.io deleted
✔ CustomResourceDefinition/buckets.source.toolkit.fluxcd.io deleted
✔ CustomResourceDefinition/gitrepositories.source.toolkit.fluxcd.io deleted
✔ CustomResourceDefinition/helmcharts.source.toolkit.fluxcd.io deleted
✔ CustomResourceDefinition/helmreleases.helm.toolkit.fluxcd.io deleted
✔ CustomResourceDefinition/helmrepositories.source.toolkit.fluxcd.io deleted
✔ CustomResourceDefinition/imagepolicies.image.toolkit.fluxcd.io deleted
✔ CustomResourceDefinition/imagerepositories.image.toolkit.fluxcd.io deleted
✔ CustomResourceDefinition/imageupdateautomations.image.toolkit.fluxcd.io deleted
✔ CustomResourceDefinition/kustomizations.kustomize.toolkit.fluxcd.io deleted
✔ CustomResourceDefinition/ocirepositories.source.toolkit.fluxcd.io deleted
✔ CustomResourceDefinition/providers.notification.toolkit.fluxcd.io deleted
✔ CustomResourceDefinition/receivers.notification.toolkit.fluxcd.io deleted
✔ Namespace/flux-system deleted
✔ uninstall finished

...

After:

...
k get crd
ocirepositories.source.toolkit.fluxcd.io         2022-09-12T22:47:01Z
...

This issue still exists with ocirepositories.source.toolkit.fluxcd.io crd

[danielkimuipath]

k edit crd ocirepositories.source.toolkit.fluxcd.io

# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  annotations:
    controller-gen.kubebuilder.io/version: v0.7.0
  creationTimestamp: "2022-09-12T22:47:01Z"
  deletionGracePeriodSeconds: 0
  deletionTimestamp: "2022-09-13T19:56:51Z"
  finalizers:
  - customresourcecleanup.apiextensions.k8s.io
  generation: 1

I see finalizer is still there after the deletion, which I had to update manually.

This issue went aware with other crd like

imagepolicies.image.toolkit.fluxcd.io            2022-09-12T22:47:02Z
imagerepositories.image.toolkit.fluxcd.io        2022-09-12T22:47:01Z
imageupdateautomations.image.toolkit.fluxcd.io   2022-09-12T22:47:03Z

[danielkimuipath]

Also, I downloaded fluxcd via terrafrom provider 0.18.0 which is the latest

Also, for the flux cli, installed via choco -

flux --version
flux version 0.33.0

flux version
flux: v0.33.0
helm-controller: v0.24.0
image-automation-controller: v0.25.0
image-reflector-controller: v0.21.0
kustomize-controller: v0.28.0
notification-controller: v0.26.0
source-controller: v0.29.0

[kingdonb]

I think this may be resolved in Flux 0.34.0

There are some labels which are placed on the Flux CRDs to make them detected for the uninstaller to delete them. I noticed in a recent version that for some reason, I was not seeing OCIRepository deleted, for example. When I upgraded to Flux 0.34.0, I noticed that issue went away. (I was also testing with Flux 0.33.x prior to that when I noticed the issue.)

If the labels are missing in the current version and you've tried upgrading everything today, then it may be a bug in the Terraform provider. Please test again with the current version and let us know what you find. I think it's unconventional to use flux uninstall together with the Terraform Provider, but I don't think there is any reason why it shouldn't work.