Skip to content
This repository has been archived by the owner on Oct 12, 2023. It is now read-only.

Releases: argoproj/applicationset

v0.4.1

10 Mar 14:20
@jgwest jgwest
v0.4.1
a5712ba
Compare
Choose a tag to compare
v0.4.1 Latest
Latest

v0.4.1

Contributors

  • Alexander Matyushentsev (@alexmt)

New in this release

This is a bug-fix release that fixes an issue with Git credential acquisition by the ApplicationSet controller in the v0.4.0 ApplicationSet controller and Argo CD v2.3.0 (due to packaging ApplicationSet controller v0.4.0).

Contributors

alexmt
Assets 2

v0.4.0

08 Mar 03:27
@jgwest jgwest
v0.4.0
eab46ad
Compare
Choose a tag to compare
v0.4.0

I am happy to announce the latest release of the Argo CD ApplicationSet controller, v0.4.0. With this release, and alongside Argo CD v2.3.0, we now fully integrate with the default Argo CD install, which means you now only need to install Argo CD to get the full benefits of ApplicationSets.

Contributors

Thanks to all the folks who have contributed to the ApplicationSet controller since our last release.

Want to join us for our next release? Check out the project repository (https://github.com/argoproj/applicationset) or visit us on #argo-cd-appset on Slack (https://argoproj.github.io/community/join-slack/).

New in this release

ApplicationSet controller is now integrated with Argo CD install

The ApplicationSet controller is now installed by default with the latest release of Argo CD. This means that users of Argo CD now get all the benefits of ApplicationSets, without the requirement of a standalone install.

Contributed by @ishitasequeira and @jgwest.

Git generator: Add support for extraction of components of paths

The Git generator now supports the extraction of individual components of the path, with the new path[n] parameter:

  • {{path[n]}}: The path to the matching configuration file within the Git repository, split into array elements (n - array index).
  • For example, for a path of /clusters/clusterA, the individual components can be extracted like so: path[0]: clusters, path[1]: clusterA

Contributed by @stempher.

Git generator: Sanitize basename param by replacing unsupported characters

When using the Git generator, with a basename name param that contains an unsupported character, you may now use the {{path.basenameNormalized}} parameter to normalize these resources. This prevents rendering invalid Kubernetes resources with names like my_cluster-app1, and instead would convert them to my-cluster-app1.

Contributed by @missingcharacter.

Make webhook address configurable

When using a webhook, the address of the webhook can now be configured using the --webhook-addr parameter on the argocd-applicationset controller.

Example:

./dist/argocd-applicationset --webhook-addr=":9999" --logformat=json

Contributed by @chetan-rns.

Fixes / Chores

  • ApplicationSet CRD size reduction, by removing validation (CRD defn) of nested merge/matrix generator (#463, contributed by @jgwest)
  • Reap zombie processes in argocd-applicationset-controller pod using tini (#453, contributed by @hcelaloner)
  • Log all validation errors (#439, contributed by @crenshaw-dev)
  • Set applicationset-controller containerPort name (#444, contributed by @aabouzaid)
  • Append missing s to matchExpression (#449, contributed by @ml-)
  • Set controller logger if we don't use JSON format (#451, contributed by @mkilchhofer)
  • Remove hardcoded namespace from manifests (#474, contributed by @ishitasequeira)
  • Fix docs typo (#493 and #481, contributed by @crenshaw-dev)

Test/infrastructure improvements:

  • E2E tests should use application-controller serviceaccount, rather than applicationset-controller serviceaccount (#434, contributed by @jgwest)
  • Add GitHub action to run E2E tests against nightly Argo CD, w/ ApplicationSet master branch (#470, contributed by @jgwest)

Contributors

crenshaw-dev, missingcharacter, and 9 other contributors
Assets 2

v0.3.0

16 Dec 01:18
@jgwest jgwest
v0.3.0
849d5f0
Compare
Choose a tag to compare
v0.3.0

I am happy to announce the latest release of the Argo CD ApplicationSet controller, v0.3.0. Many new features were contributed as part of this release, including two new generators, improved error reporting and handling, support for webhook-based refresh trigger, plus doc updates, usability improvements, stability fixes, and more.

You can learn more about this release from the ApplicationSet documentation, or check out the project repository and learn how you can contribute.

Contributors

Thanks to all the folks who have contributed to the ApplicationSet controller since our last release!

Want to join us for our next release? Check out the project repository (https://github.com/argoproj-labs/applicationset) or visit us on #argo-cd-appset on Slack.

New in this release

New generator: Pull Request generator

With ApplicationSet v0.3.0, a new Pull Request generator has been contributed which uses the API of an SCMaaS provider (e.g. GitHub) to automatically discover open pull requests within an repository. This fits well with users that wish to construct a test environment based on an open pull request.

In this example, we will create an Argo CD Application resource for each open pull request:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: myapps
spec:
  generators:
  - pullRequest:
      github:
        # The GitHub organization or user.
        owner: myorg
        # The Github repository
        repo: myrepository
        # For GitHub Enterprise (optional)
        api: https://git.example.com/
        # Reference to a Secret containing an access token. (optional)
        tokenRef:
          secretName: github-token
          key: token
        # Labels is used to filter the PRs that you want to target. (optional)
        labels:
        - preview
  template:
  # (template the Application using PR generator params)...

To learn more, check out the Pull Request generator documentation for details. Contributed by @shmurata.

New generator: Merge generator

Also new in this release is the Merge generator, which is useful when you want to selectively override the parameters generated by one generator, with those generated by another.

In this example, we first gather the list of clusters from Argo CD, then we 'patch' only those clusters with label use-kakfa: false, and finally we enable redis on a specfic cluster:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: cluster-git
spec:
  generators:
    # merge 'parent' generator
    - merge:
        mergeKeys:
          - server
        generators:
          # Generate parameters for all Argo CD clusters
          - clusters:
              values:
                kafka: 'true'
                redis: 'false'
          # For clusters with a specific label, enable Kafka.
          - clusters:
              selector:
                matchLabels:
                  use-kafka: 'false'
              values:
                kafka: 'false'
          # For a specific cluster, enable Redis.
          - list:
              elements: 
                - server: https://2.4.6.8
                  values.redis: 'true'

See the Merge generator documentation for a full example, and for details on generator behaviour. Contributed by @crenshaw-dev.

Report error conditions/status for ApplicationSet CR

When the user-provided generator/template produces invalid Argo CD Applications, the ApplicationSet resource's status field will now report errors (or the lack thereof). Here is an example of the new status conditions:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: myapps
spec:
  generators: # (...)
  template: # (...)
status:
  conditions:
  - lastTransitionTime: "2021-11-23T05:47:08Z"
    type: ErrorOccurred
    status: "False"
    reason: ApplicationSetUpToDate
    message: Successfully generated parameters for all Applications
  - lastTransitionTime: "2021-11-23T05:47:08Z"
    type: ParametersGenerated
    message: Successfully generated parameters for all Applications
    reason: ParametersGenerated
    status: "True"
  - lastTransitionTime: "2021-11-23T05:47:08Z"
    type: ResourcesUpToDate
    status: "True"
    reason: ApplicationSetUpToDate
    message: ApplicationSet up to date

On parameter generation failure or templating failure, those errors will be reported under the appropriate conditions. Contributed by @ishitasequeira.

Git Generator: Refresh ApplicationSet resource with Git generator using webhook

This feature adds support for refreshing ApplicationSets via a GitHub webhook trigger. It exposes a service which listens for incoming webhook payloads, and once received triggers the ApplicationSet controller to regenerate resources. In contrast, with the previous release, the ApplicationSet controller only supported polling the Git repository used by the Git generator every 3 mins (but this is at least customizable).

See the webhook documentation for details. Contributed by @chetan-rns.

This contribution also adds general support for webhooks, which is used by the Pull Request generator webhook code, below.

Gracefully handle application validation errors

This feature changes how the ApplicationSet controller handles ApplicationSets that generate invalid Applications. Previously, if at least one Application in the ApplicationSet was invalid, the controller would refuse to proceed further and would skip all Application processing (i.e. it would 'fail fast'). Now, the controller will process valid Applications, and only skip invalid Applications (logging information about them to the console).

Contributed by @alexmt.

Pull Request generator: Support for webhooks

When using a Pull Request generator, the ApplicationSet controller polls every requeueAfterSeconds interval (defaulting to every 30 minutes) to detect changes. To eliminate this delay from polling, the ApplicationSet webhook server can be configured to receive webhook events, which will refresh the parameters generated by the Pull Request generator, and thus the corresponding Application resources.

More information on configuring webhooks with the Pull Request generator is available from the Pull Request generator documentation. Contributed by @shmurata.

Improve Git files generator performance (#355)

The Git files generator was consuming too much time (and driving up Git requests) due to inadvertently executing 'git fetch/git checkout' for each discovered file within the repository. With ApplicationSet v0.3.0, that has improved such that we will now issue a Git checkout/fetch repo once per refresh. Contributed by @alexmt.

Support -logformat=json as parameter to applicationset-controller

This feature adds a new --logformat=json parameter to the applicationset-controller, which switches the logging output of the ApplicationSet controller to JSON. Contributed by @shivjm.

SCM Generator: Provide SHA for latest commit on a branch in variables (#307)

This feature adds SHA to the list of parameters exposed by the SCM Generator, with the SHA parameter representing the latest commit. Contributed by @TMaYaD.

Fixes, test fixes, infrastructure improvements, and documentation updates

Fixes

  • Fix: new variable for the normalized version of name field (#390, contributed by @chetan-rns)
  • Fixes GitLab RepoHasPath error handling (#423, contributed by @benoitg31)

Test/infrastructure improvements:

  • Investigate Argo CD deletion failure messages when running ApplicationSet E2E tests in on-cluster configuration (#392, contributed by @jgwest)
  • Update master branch VERSION file and metadata, and pull up release changes from 0.2.0 (#343, contributed by @jgwest)
  • Skip E2E tests that require GitHub token, if not specified (#380, contributed by @jgwest)
  • API rate limit error in image publish action ([#368](https://github.com/argoproj-labs/app...
Read more

Contributors

TMaYaD, olvesh, and 14 other contributors
Assets 2

v0.2.0

09 Aug 15:36
@jgwest jgwest
v0.2.0
2fb0435
Compare
Choose a tag to compare
v0.2.0

v0.2.0

I am happy to announce the second release of the Argo CD ApplicationSet controller, v0.2.0. Many new features were contributed as part of this release, including support for combining generator parameters, support for building Argo CD Applications based on GitHub/GitLab organizations, and support for using custom resources to select clusters, plus oft requested improvements to existing generators, and of course doc updates, usability improvements, stability fixes, and more.

You can learn more about this from the ApplicationSet documentation or check out the source and learn how you can contribute.

Contributors

Many many thanks to all the folks who have contributed to the ApplicationSet controller over the past few months. These many contributions, both big and small, general and specific, help to bring a more featureful and polished experience to Argo CD users. We could not do this without all of you!

Want to join us for our next release? Check out the project repository (https://github.com/argoproj-labs/applicationset) or visit us on #argo-cd-appset on Slack (https://argoproj.github.io/community/join-slack/).

New in this Release

Matrix generator

The Matrix generator is a new generator that combines the parameters generated by two child generators, iterating through every combination of each generator's set:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: cluster-git
spec:
  generators:
    - matrix: # 'Parent' Matrix Generator
        generators:
          - git: # 'Child' generator A
              repoURL: https://github.com/argoproj-labs/applicationset.git
              revision: HEAD
              directories:
                - path: examples/matrix/cluster-addons/*
          - clusters: # 'Child' generator B
              selector:
                matchLabels:
                  argocd.argoproj.io/secret-type: cluster
  template:
  # (...)

The parameters generated by a Git generator (looking for directories within a Git Repository), and by a Cluster generator (looking for Argo CD-managed clusters), are combined: this will generate Argo CD Applications that target each of the Git directories (containing Kubernetes manifests), for each cluster managed by Argo CD.

See the Matrix generator documentation for an in-depth example of how generator parameters are combined. Contributed by @OmerKahani.

SCM Provider generator

The SCM Provider generator is a new generator that utilizes the API of GitHub/GitLab to automatically discover repositories within an organization, allowing the repository values to be used for targeting generated Argo CD Applications. This fits well with GitOps layout patterns that split microservices across many repositories, rather than those patterns that stick to a single repository (which can be handled by other ApplicationSet generators).

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: myapps
spec:
  generators:
  - scmProvider:
      github:
        organization: myorg         # The GitHub organization to scan.
        tokenRef: { } # Reference to a Secret containing an access token. (optional)
  template:
  # ...

The ApplicationSet controller will then scan the provided GitHub/GitLab organization and produce template parameters for each discovered repository/branch, which may be used to generate Argo CD Applications for each of these repositories.

See the SCM Provider generator documentation for more information. Initial implementation and GitHub support contributed by @coderanger, and GitLab support contributed by @empath-nirvana.

Cluster Decision Resource generator

The Cluster Decision Resource generator is a new generator that generates a list of Argo CD clusters based on the contents of an external custom resource (CR), with that custom resource managed by an external controller. With the Cluster Decision Resource generator, you may 'outsource' the logic (of which clusters to target) to third party controllers/CRs, such as the Open Cluster Management's Placements.

This is handled seamlessly using duck-typing, which does not require knowledge of the full shape of the referenced Kubernetes resource. The following is an example of a cluster-decision-resource-based ApplicationSet generator:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
 name: guestbook
spec:
 generators:
 - clusterDecisionResource:
    # ConfigMap with GVK information for the duck type resource
    configMapRef: my-configmap      
    name: quak  # Choose either "name" of the resource or "labelSelector"

Which might reference an external resource named quak, managed by an external controller, containing a list of clusters:

apiVersion: mallard.io/v1beta1
kind: Duck
metadata:
  name: quak
spec: {}
status:
  decisions:   # Duck-typing ignores all other aspects of the resource except the "decisions" list
  - clusterName: cluster-01
  - clusterName: cluster-02

See the Cluster Decision Resource generator documentation for more information. Cluster Decision Resource generator and labelSelector support contributed by @jnpacker.

Preserve Application child resources on deletion of parent ApplicationSet

By default, Applications created and managed by the ApplicationSet controller include the Argo CD resource deletion finalizer. This means that when an ApplicationSet is deleted, its child Applications will be deleted, as well as the cluster resources of those child Applications. This is the same behaviour as a cascade delete within Argo CD.

However, this behaviour is not always desirable: one may want to preserve the Application child resources on deletion of the parent Application. To enable this, using the .spec.syncPolicy.preserveResourcesOnDeletion value in the parent ApplicationSet:

kind: ApplicationSet
spec:
  generators:
    - clusters: {}
  template:
    # (...)
  syncPolicy:
    # Don't delete Application's child resources, on parent deletion:
    preserveResourcesOnDeletion: true

See the Application Deletion Behaviour documentation for more information. Contributed by @mmatur.

Add YAML configuration file support to Git File generator

In the previous ApplicationSet release, only JSON-formatted configuration files were supported by the Git File generator. Now, both JSON and YAML files are supported, and may be used interchangeably. Contributed by @teejaded.

Allow any key/value pair in List generator

The List generator previously only supported a fixed list of cluster name/URL values, with an optional values field for user-defined values. You can now specify any key/value pair:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
spec:
 generators:
 - list:
     elements:
     # current form, still supported:
     - cluster: engineering-dev
       url: https://kubernetes.default.svc
       values:
         additional: value
     # new form, does not require cluster/URL keys:
     - staging: true
       gitRepo: https://kubernetes.default.svc   
 template:
   # (...)

This new form is fully backwards compatible with existing v0.1.0-originated ApplicationSets, and no migration steps are needed. See the List generator documentation for more information. Contributed by @ishitasequeira.

Added additional path params to Git File generator

The Git File generator now produces {{ path }} and {{ path.basename }} parameters, containing the path of the configuration file, similar to the same parameters already produced by the Git Directory generator. Contributed by @ishitasequeira.

Add exclude path support to Git directories

The Git Directory generator scans directories within a Git repository, looking for directories that match specifi...

Read more

Contributors

coderanger, palp, and 19 other contributors
Assets 2

v0.1.0

07 Apr 13:28
@jgwest jgwest
v0.1.0
8100e6b
Compare
Choose a tag to compare
v0.1.0

v0.1.0

I am excited to announce the first release of the Argo CD ApplicationSet controller, v0.1.0, releasing now alongside Argo CD v2.0!

The ApplicationSet controller provides the ability:

  • To deploy Argo CD Applications to multiple Kubernetes clusters at once
  • To deploy multiple Argo CD applications from a single monorepo
  • Allows unprivileged cluster users (those without access to the Argo CD namespace) to deploy Argo CD applications without the need to involve cluster administrators in enabling the destination clusters/namespaces
  • Best of all, all these features are controlled by only a single ApplicationSet Kubernetes custom resource, which means no more juggling of multiple Argo CD Application resources to target those multiple clusters/repos!

Unlike with an Argo CD Application resource, which deploys resources from a single Git repository to a single destination cluster/namespace, ApplicationSet uses templated automation to create, modify, and manage multiple Argo CD applications at once.

You can learn more about this from the ApplicationSet documentation or check out the source and learn how you can contribute.

Since this is our first release, we would ❤️ if you would give ApplicationSets a shot, and let us know what cool problems you are solving with it, or what pain points you hit. Got feature requests, bug reports, or want to contribute code? Let us know on our project repository, or join us on #argo-cd-appset on Slack.

Contributors

A great deal of work has gone into bringing this project to life, from many different contributors, all the way from its inception in early 2020 until today. It is terrific to be able to bring all that work together as ApplicationSet's first release, and make it available to a wider audience… we welcome you to try it out, and let us know what you think!
A big thanks to all ApplicationSet controller contributors for their hard work over the last year, whether it be contributing code, writing design documentation, performing code reviews, writing user documentation, and opening issues and PRs:

The ApplicationSet controller would not exist without the contributions of these talented individuals! 🎉

New in this Release

List Generator

The List generator generates parameters based on a fixed list of cluster name/URL values, with those values passed as parameters into the template. This allows manual control over Application destinations via editing of a literal list with the ApplicationSet.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
spec:
 generators:
 - list:
     elements:
     - cluster: engineering-dev
       url: https://kubernetes.default.svc
#    - cluster: engineering-prod
#      url: (another cluster API url)
 template:
   spec:
     project: default
     source:
       repoURL: https://github.com/argoproj-labs/applicationset.git
       targetRevision: HEAD
       path: examples/list-generator/guestbook/{{cluster}}
     # (...)

In this example, if one wanted to add a second cluster, we could uncomment the second cluster element and the ApplicationSet controller would automatically target it with the defined application.

Cluster Generator

The cluster generator is useful if you are using Argo CD to manage cluster add-ons, such as Custom Resource Definitions (CRDs) or Ingress Controllers, across a fleet of clusters. Instead of manually creating an application for each cluster, you can delegate it to the cluster generator.

In Argo CD, managed clusters are stored within Secrets in the Argo CD namespace. The ApplicationSet controller uses those same Secrets to generate template parameters for which to target available clusters.

The Cluster generator will automatically identify clusters defined with Argo CD, and extract the cluster data as parameters:

kind: ApplicationSet
spec:
  generators:
  - clusters: {} # Automatically use all clusters defined within Argo CD
  template:
    metadata:
      name: '{{name}}-guestbook' # 'name' field of the cluster
    spec:
      source: 
        # (...)
      destination:
        server: '{{server}}' # 'server' field of the cluster
        namespace: guestbook

Git Directory Generator

It is a good practice to define a convention between Argo CD application name and the location of the deployment manifests directory with the Git repository. For example, you might choose to store all manifests of production applications under "applications/prod/" and all staging applications under "applications/staging/".

The Git Directory generator allows you to go one step further and "codify" that convention. The Git directory generator generates template parameters using the directory structure of a specified Git repository.
Whenever a new subfolder is added to the Git repository, the ApplicationSet controller will detect this change and automatically use the subfolder path to template an Argo CD application containing the manifests contained within that folder (whether they be plain YAML, Kustomize, Helm, etc).

kind: ApplicationSet
spec:
  generators:
  - git:
      repoURL: https://github.com/argoproj-labs/applicationset.git
      revision: HEAD
      directories:
      - path: examples/git-generator-directory/cluster-addons/*
# (...)

Git File Generator

Similar to the Directory generator, the Git File generator allows you to generate applications based on Git repository content but provide a bit more flexibility. The Git file generator generates template parameters using the contents of JSON files found within a specified repository.
Git commits containing changes to JSON files are automatically discovered by the Git generator, and the contents of those files are parsed and converted into template parameters.

This allows the creation of custom Argo CD Applications based on the contents of automatically discovered JSON files within the repository. As new files are added/changed, new Argo Applications are created or modified.

kind: ApplicationSet
spec:
  generators:
  - git:
      repoURL: https://github.com/argoproj-labs/applicationset.git
      revision: HEAD
      files:
      - path: "examples/git-generator-files-discovery/cluster-config/**/config.json"

Brand new documentation and examples

The ApplicationSet controller has a brand new set of documentation and examples. Topics include introduction, a quick getting started guide, use cases, the interaction with Argo CD, generators, template fields, application lifecycle, and more.

Template Override

In addition to specifying a template within the .spec.template of the ApplicationSet resource, templates may also be specified within generators. This is useful for overriding the values of the spec-level template with generator-specific values.

spec:
  generators:
  - list:
      elements:
        - cluster: engineering-dev
          url: https://kubernetes.default.svc
      template: # <--- A template under a list generator
        metadata: {}
        spec:
          project: "default"
          source:
            revision: HEAD
            repoURL: https://github.com/argoproj-labs/applicationset.git
            # New path value is generated here:
            path: 'examples/template-override/{{cluster}}-override'
          destination: {}

Support for arbitrary key/value pairs in Cluster generator and List generator

Arbitrary key/value pairs may be included within the Cluster and List generators, which will be converted into template parameters during template rendering. This is useful for providing custom parameters for a specific generator instance:

spec:
  generators:
  - clusters:
      values:
        version: '2.0.0'

Bugs, tests, and infrastructure improvements

In addition to the above new features, we delivered lots of bug fixes, new unit tests, a new end-to-end test framework, new end-to-end tests, a new release process, and build/test infrastructure improvements.

Installation

The ApplicationSet controller must be installed into the same namespace as the Argo CD it is targetting:

kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj-labs/applicationset/v0.1.0/manifests/install.yaml

Once installed, the ApplicationSet controller requires no additional setup. You can learn more about ApplicationSet controller installation from the Getting Started page.

Changelog

Features:

  • Add item list CRD Spec (#1)
  • Sketch out cluster generator in the CRD, and generator interface (#2)
  • Add list generator support (#3)
  • Add git directory crd (#4)
  • Add git directory generator (#7)
  • Cluster Generator reconciling on cluster secret events (#8)
  • ...
Read more
Assets 2