README.yaml

name: Packages

description: |-
  Cloud Posse distribution of awesome apps.

introduction: |-

  Use this repo to easily install releases of popular Open Source apps. We provide a few ways to use it.

  1. **Make Based Installer.** This installer works regardless of your OS and distribution. It downloads packages directly from their GitHub source repos and installs them to your `INSTALL_PATH`. 
  2. **Alpine Linux Packages.** Use our Alpine repository to install prebuilt packages that use the original source binary (where possible) from the maintainers' official GitHub repo releases.
  3. **Docker Image.** Use our docker image as a base-image or as part of a multi-stage docker build. The docker image always distributes the latest linux binaries for `x86_64` architectures.

  See examples below for usage.

  **Is one of our packages out of date?**

  Open up an [issue](https://github.com/cloudposse/packages/issues) or submit a PR (*preferred*). We'll review quickly!

  ## Sponsorship [<img src="https://cloudposse.com/wp-content/uploads/2020/10/cloudsmith.svg" width="250" align="right" />](https://cloudsmith.io/)

  Package repository hosting is graciously provided by [cloudsmith](https://cloudsmith.io/). Cloudsmith is the only fully hosted, cloud-native, universal package management solution, that enables your organization to create, store and share packages in any format, to any place, with total confidence. We believe there’s a better way to manage software assets and packages, and they’re making it happen!

# License of this project
license: "APACHE2"

# Canonical GitHub repo
github_repo: cloudposse/packages

# Badges to display
badges:
  - name: "Auto Update Status"
    image: "https://github.com/cloudposse/packages/actions/workflows/auto-update-packages.yml/badge.svg"
    url: "https://github.com/cloudposse/packages/actions/workflows/auto-update-packages.yml"
  - name: "Slack Community"
    image: "https://slack.cloudposse.com/badge.svg"
    url: "https://slack.cloudposse.com"

related:
  - name: "build-harness"
    description: "Collection of Makefiles to facilitate building Golang projects, Dockerfiles, Helm charts, and more"
    url: "https://github.com/cloudposse/build-harness"
  - name: "geodesic"
    description:
      "Geodesic is the fastest way to get up and running with a rock solid, production grade cloud platform built on
      strictly Open Source tools."
    url: "https://github.com/cloudposse/geodesic"

# Other files to include in this README from the project folder
include:
  - "docs/badges.md"

usage: |-

  ## Debian Repository (recommended)

  A public Debian repository is provided by [Cloud Posse](https://cloudposse.com). 
  The repository is hosted by [Cloudsmith](https://cloudsmith.com/)
  Using this Debian repository is ultimately more reliable than depending on [GitHub for availability](https://twitter.com/githubstatus)
  and provides an easier way to manage dependencies pinned at multiple versions. 

  ### Configure the Debian repository:

  #### The Easy Way

  Cloudsmith provides an installation script to configure the Debian repository for your version of Debian. 

  ```
  curl -1sLf 'https://dl.cloudsmith.io/public/cloudposse/packages/cfg/setup/bash.deb.sh' | bash
  ```
  __NOTE__: Requires `bash` and `curl` to run:

  #### For Docker

  Add the following to your `Dockerfile` near the top.
  ```
  # Install the cloudposse Debian repository
  RUN apt-get update && apt-get install -y apt-utils curl
  RUN curl -1sLf 'https://dl.cloudsmith.io/public/cloudposse/packages/cfg/setup/bash.deb.sh' | bash
  ```

  ### Installing Debian Packages

  When adding packages, we recommend using `apt-get update && apt-get install -yq $package` to update the repository index before installing packages.

  Simply install any package as normal:
  ```
  apk-get install -y terraform
  ```

  But we recommend that you use version pinning (the `-\*` is important, as it gets you the latest version of the package):
  ```
  apt-get install gomplate=3.0.0-\*
  ```

  ## RPM Repository
  
  We publish RPM packages corresponding to all the Debian packages we publish. Installing the repository is almost the same as above.
  
  ```
  curl -1sLf 'https://dl.cloudsmith.io/public/cloudposse/packages/cfg/setup/bash.rpm.sh' | bash
  ```

  Install packages as normal. `yum` automatically updates the repository index before installing packages.
  ```
  yum install -y terraform
  ```
  
  Note that unlike other package systems, `yum` automatically updates the repository index before installing packages.
  
  We still recommend version pinning, but `yum` makes it hard. You can use this command to list all the available versions of a package:
  ```
  yum --showduplicate list $package
  ```
  Then you can install the package with a version pinning:
  ```
  VERSION=1.3.0
  RELEASE=1
  yum install -y $package-$VERSION-$RELEASE.$(uname -m)
  ```
  
  ## Alpine Repository 

  A public Alpine repository is provided by [Cloud Posse](https://cloudposse.com). 
  The repository is hosted by [Cloudsmith](https://cloudsmith.com/)
  Using this Alpine repository is ultimately more reliable than depending on [GitHub for availability](https://twitter.com/githubstatus)
  and provides an easier way to manage dependencies pinned at multiple versions. 

  ### Configure the Alpine repository:

  #### The Easy Way

  Cloudsmith provides an installation script to configure the Alpine repository for your version of Alpine. 

  ```
  apk add --no-cache bash curl
  curl -1sLf \
  'https://dl.cloudsmith.io/public/cloudposse/packages/setup.alpine.sh' \
  | bash  
  ```

  ### Installing Alpine Packages

  When adding packages, we recommend using `apk add --update $package` to update the repository index before installing packages.

  Simply install any package as normal:
  ```
  apk add --update terraform
  ```

  But we recommend that you use version pinning:
  ```
  apk add --update terraform==1.0.0-r0
  ```

  And maybe even repository pinning, so you know that you get our versions:
  ```
  apk add --update terraform@cloudposse==1.0.0-r0
  ```

  ## Makefile Interface

  The `Makefile` interface works on OSX and Linux. It's a great way to distribute binaries in an OS-agnostic way which does not depend on a package manager (e.g. no `brew` or `apt-get`). 

  This method is ideal for [local development environments](https://docs.cloudposse.com/local-dev-environments/) (which is how we use it) where you need the dependencies installed natively for your OS/architecture, such as installing a package on OSX.

  See all available packages:
  ```
  make -C install help
  ```

  Install everything...
  ```
  make -C install all
  ```

  Install specific packages:
  ```
  make -C install aws-vault chamber
  ```

  Install to a specific folder:
  ```
  make -C install aws-vault INSTALL_PATH=/usr/bin
  ```

  Uninstall a specific package
  ```
  make -C uninstall yq
  ```

  ### Rebuilding GitHub Action Workflows

  The GitHub Action workflows are compiled from the `.github/package-template.yml` file by running `make -C .github workflows`. It's also run automatically when rebuilding the `README.md` with `make readme`.

  Run this make target anytime the `package-template.yml` changes or any new packages are added to the `vendor/` folder.

  __IMPORTANT__: The `package-template.yml` supports a single macro for interpolation `%PACKAGE_NAME%` which is replaced using a `sed` expression.
  Since the workflow uses a combation of gotemplate-like interpolations as well as inlines shell scripts, we used the `%VAR%` form of interpolation to avoid
  the need for endless escaping of interpolation specifiers.

  ### Testing Locally

  #### Alpine

  ```sh
  $ make docker/build/apk/shell
  $ make -C vendor/<package> apk
  ```

  #### Debian

  ```sh
  $ make docker/build/deb/shell
  $ make -C vendor/<package> deb
  ```

  #### RPM

  ```sh
  $ make docker/build/rpm/shell
  $ make -C vendor/<package> rpm
  ```

  ### Mac

  ```sh
  $ make -C vendor/<package> install
  ```

examples: |-
  ### Docker Multi-stage Build

  Add this to a `Dockerfile` to install packages using a multi-stage build process:
  ```
  FROM cloudposse/packages:latest AS packages

  COPY --from=packages /packages/bin/kubectl /usr/local/bin/
  ```

  ### Docker with Git Clone

  Or... add this to a `Dockerfile` to easily install packages on-demand:
  ```
  RUN git clone --depth=1 -b main https://github.com/cloudposse/packages.git /packages && \
      rm -rf /packages/.git && \
      make -C /packages/install kubectl
  ```

  ### Makefile Inclusion

  Sometimes it's necessary to install some binary dependencies when building projects. For example, we frequently 
  rely on `gomplate` or `helm` to build chart packages.

  Here's a stub you can include into a `Makefile` to make it easier to install binary dependencies.

  ```
  export PACKAGES_VERSION ?= main
  export PACKAGES_PATH ?= packages/
  export INSTALL_PATH ?= $(PACKAGES_PATH)/bin

  ## Install packages
  packages/install:
          @if [ ! -d $(PACKAGES_PATH) ]; then \
            echo "Installing packages $(PACKAGES_VERSION)..."; \
            rm -rf $(PACKAGES_PATH); \
            git clone --depth=1 -b $(PACKAGES_VERSION) https://github.com/cloudposse/packages.git $(PACKAGES_PATH); \
            rm -rf $(PACKAGES_PATH)/.git; \
          fi

  ## Install package (e.g. helm, helmfile, kubectl)
  packages/install/%: packages/install
          @make -C $(PACKAGES_PATH)/install $(subst packages/install/,,$@)

  ## Uninstall package (e.g. helm, helmfile, kubectl)
  packages/uninstall/%:
          @make -C $(PACKAGES_PATH)/uninstall $(subst packages/uninstall/,,$@)
  ```

  ### Contributing Additional Packages
  In addition to following the Contributing section, the following steps can be used to add new packages for review (via a PR).
  If possible (and it usually is), you want to find an existing package with similarly packaged release (`.tar`, `.gz`, uncompressed binary, etc.),
  and copy and edit its Makefile.
  1. Copy the Makefile from an existing, similar, package within the vendors directory. Name the new folder with the same name as the binary package being installed.
  2. Edit the Makefile, ensuring the `DOWNLOAD_URL` is properly formatted
  3. Run `make init` from within the directory to create the `DESCRIPTION`, `LICENSE`, `RELEASE`, and `VERSION` files.
  4. Ensure that a test task exists in the package Makefile. It should check the version number of the installed binary if possible.
  5. Test the install and ensure that it downloads and runs as expected (`make -C install <your_package> INSTALL_PATH=/tmp`)
  6. Test the apk build (see below)
  7. Update the `README.md` (`make init readme/deps readme`)

  ### Testing apk builds

  To validate that a new package will build into an apk you can use the following steps;

  ```bash
  make docker/build/apk/shell
  make -C vendor/<appname> apk
  # Some temp build files in the volume mount set user/group to nobody/nobody for apk building.
  # It is easier to remove them while within the docker container.
  rm -rf ./tmp/build.*
  exit
  ```

  ### Troubleshooting Package Addition
  Here are some solutions to several common problems that may occur when adding a new package:
  
  1. <details><summary>When adding a new app, the `make -C vendor/<app> apk` command fails, claiming it can't find the app's binary file, even though it is in the expected place.</summary>
  
      Part of the `make -C vendor/<app> apk` command is building a package for the binary file inside an Alpine Linux container. Since Alpine Linux uses `musl` as its C library, this often leads to situations where binaries built against `libc` might not function on Alpine. What's more, binaries from projects written in `Go` will not be found by the Alpine package builder at all if they are missing any necessary libraries, like `libc`. The solution to this problem is to add an `export APKBUILD_DEPENDS += libc6-compat` line to the top of your new package's associated `Makefile`.
      </details>
  2.  <details><summary>When adding a new binary, the `make builder TARGETS=readme` command fails with `Unable to find image 'cloudposse/build-harness:sha-[some_SHA_stub]' locally`.</summary>
  
      This can occur when you have the `cloudposse/build-harness` repository checked out somewhere on your machine. `make builder TARGETS=readme` will end up looking for a docker image tagged with the SHA that the `HEAD` ref of your `buld-harness` points to. To correct this behavior, just run `make init` in the `cloudposse/packages` directory prior to running `make builder TARGETS=readme`.
      </details>

# Contributors to this project
contributors:
  - name: "Erik Osterman"
    github: "osterman"
  - name: "Nuru"
    github: "Nuru"
  - name: "Igor Rodionov"
    github: "goruha"
  - name: "Andriy Knysh"
    github: "aknysh"