Working with ansible‐vault and Git

ansible-vault with Git

Scripts and Snippets for Easier .eyaml Development

The steps below will allow you to:

• use git diff (or other diffing tools that use git diff behind-the-scenes) to automatically diff the inner changes of .eyaml files.
• easily edit and and cat the inner YAML of .eyaml encrypted files with vault-edit/vault-decrypt without needing to deal with ansible-vault.
• automatically merge most changes to .eyaml files and easily resolve conflicts in your chosen EDITOR without needing to manually merge during a git rebase or git merge, using vault-merge (courtesy of this gist)

1. Add to ~/.gitattributes

   *.eyaml diff=ansible-vault

2. Add to ~/.gitconfig

   attributesfile = ~/.gitattributes
   [diff "ansible-vault"]
   textconv = "/Users/<USERNAME>/.bin/vault-decrypt"
   cachetextconv = false

3. Add the following scripts to any PATH-available bin directory

   Directory for this document is ~/.bin. This directory is not in PATH by default, so you will need to add it.

   1. vault-pass

      #!/bin/bash
      
      SSM_VAULT_PASSWORD_PATH="/bfd/mgmt/jenkins/sensitive/ansible_vault_password"
      
      aws ssm get-parameter \
        --region us-east-1 \
        --output text \
        --with-decryption \
        --query 'Parameter.Value' \
        --name "$SSM_VAULT_PASSWORD_PATH"

   2. vault-edit

      #!/usr/bin/env bash
      ansible-vault edit --vault-password-file=~/.bin/vault-pass "${1}"

   3. vault-decrypt

      #!/usr/bin/env bash
      
      ansible-vault decrypt --vault-password-file=~/.bin/vault-pass "${1}" --output -

   4. vault-merge (adapted from this gist)

      sponge is not included by default on Mac OS, so you will need to install it with the moreutils or sponge package using brew, i.e.:

      brew install moreutils

      Additionally, non-repo-root-relative paths must be prefixed with ./ (i.e. ops/terraform/services/base/values/prod.eyaml can be usually be specified relatively as prod.eyaml if the working directory is ops/terraform/services/base/values, but git show requires the ./ be explicit to indicate that the working directory relative prod.eyaml is desired; as in, ./prod.eyaml). For example:

      # Current working directory is ops/terraform/services/base/values
      vault-merge ./prod.eyaml

      #!/bin/sh
      
      # vault-merge
      # Benjamin Ragheb <ben@benzado.com>
      
      # This shell script handles conflicts generated by attempts to merge encrypted
      # Ansible Vault files. Run `git merge` as usual; when git warns of a merge
      # conflict, run this command to attempt a merge on the unencrypted versions of
      # the file. If there are conflicts, you will be given a chance to correct them
      # in $EDITOR.
      
      # First, we ensure we are inside the working directory of a git repo.
      
      GIT_ROOT=$(git rev-parse --show-toplevel)
      if [ $? != 0 ]; then
          exit $?
      fi
      
      VAULT_FILE=$1
      
      # If no vault has been provided, abort!
      
      if [ -z "$VAULT_FILE" ]; then
          echo "Usage: $0 VAULT_FILE"
          exit 1
      fi
      
      # Fetch the base (common ancestor) version of the encrypted vault file, save
      # it to a temporary location, and decrypt it. (Hat Tip to the git-merge manual
      # page for tipping me off to the `git show :1:path` notation.)
      
      BASE=$(mktemp "${VAULT_FILE}".base.XXXX)
      git show :1:"${VAULT_FILE}" > "$BASE" 2> /dev/null
      if [ $? != 0 ]; then
          echo "Path '${VAULT_FILE}' does not have any conflicts."
          rm "$BASE"
          exit 1
      fi
      vault-decrypt "$BASE" | sponge "$BASE" || exit $?
      
      # Do the same with the current (branch we are merging INTO) version of the vault
      # file.
      
      CURRENT=$(mktemp "${VAULT_FILE}".current.XXXX)
      git show :2:"${VAULT_FILE}" > "$CURRENT" 2> /dev/null
      vault-decrypt "$CURRENT" | sponge "$CURRENT" || exit $?
      
      # And finally, with the other (branch we a merging FROM) version of the vault.
      
      OTHER=$(mktemp "${VAULT_FILE}".other.XXXX)
      git show :3:"${VAULT_FILE}" > "$OTHER" 2> /dev/null
      vault-decrypt "$OTHER" | sponge "$OTHER" || exit $?
      
      # Now that we have all three versions decrypted, ask git to attempt the merge
      # again. If it fails again due to a conflict, open $EDITOR and let the user
      # perform a manual merge.
      
      git merge-file "$CURRENT" "$BASE" "$OTHER"
      if [ $? == 0 ]; then
          echo "Merge OK"
      else
          echo "Merge conflict; opening editor to resolve."
          $EDITOR "$CURRENT"
      fi
      
      # Now that we're done, encrypt the file and move it into the repo, and clean up
      # the temporary files (they contain secrets!).
      
      ansible-vault encrypt --vault-password-file=~/.bin/vault-pass "$CURRENT"
      cp "$CURRENT" "$VAULT_FILE"
      rm "$BASE" "$CURRENT" "$OTHER"
      
      echo "$VAULT_FILE has been updated."
      echo "    (use \"git add $VAULT_FILE\" to mark as resolved)"
      echo "    (or re-run this command to retry the merge)"
      exit 0