No tilde expansion of token_path when configured in ansible.cfg

[michaelin]

SUMMARY

When I set token_path=~/.config/ansible and create a .vault-token file there, the vault_read plugin can't find the token file. If I change token_path to a hard-coded path like /home/my_user/.config/ansible, the plugin can read it.

The reasoning for pointing to a folder under the users home dir is to minimize the risk of accidentally committing a users vault token. The default location for token_file is ./.vault-token, which will usually be added to your .gitignore. But it is easy to mis-type the filename when you create it. If mis-typed, it is not covered by .gitignore and can easily be added and committed by mistake.

I'm aware that .vault-token will be picked up by default in my home dir, but to reduce clutter and follow the XDG Base Directory Specification I'd like to have it in ~/.config/ansible instead.

ISSUE TYPE

• Bug Report

COMPONENT NAME

community.hashi_vault.vault_read

ANSIBLE VERSION

ansible [core 2.15.6]
  config file = /home/michael_ingeman/code/ansible-main/ansible.cfg
  configured module search path = ['/home/michael_ingeman/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/michael_ingeman/code/ansible-main/.venv/lib/python3.9/site-packages/ansible
  ansible collection location = /home/michael_ingeman/code/ansible-main
  executable location = /home/michael_ingeman/code/ansible-main/.venv/bin/ansible
  python version = 3.9.5 (default, Nov 23 2021, 15:27:38) [GCC 9.3.0] (/home/michael_ingeman/code/ansible-main/.venv/bin/python)
  jinja version = 3.1.2
  libyaml = True

COLLECTION VERSION

Collection            Version
--------------------- -------
community.hashi_vault 6.0.0  

CONFIGURATION

COLLECTIONS_PATHS(/home/myuser/code/ansible_bug/ansible.cfg) = ['/home/myuser/code/ansible_bug/collections']
COLLECTIONS_SCAN_SYS_PATH(/home/myuser/code/ansible_bug/ansible.cfg) = False
CONFIG_FILE() = /home/myuser/code/ansible_bug/ansible.cfg
DEFAULT_ROLES_PATH(/home/myuser/code/ansible_bug/ansible.cfg) = ['/home/myuser/code/ansible_bug/roles']

[defaults]
callback_result_format = yaml
roles_path=./roles/
collections_path=./collections
collections_scan_sys_path=False

[hashi_vault_collection]
token_path=/home/myuser/.config/ansible
url=https://vault.mycompany
ca_cert=/etc/ssl/certs/ca-certificates.crt

OS / ENVIRONMENT

Linux michael-ws-linux 5.4.0-167-generic #184-Ubuntu SMP Tue Oct 31 09:21:49 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

STEPS TO REPRODUCE

Create a .vault-token file in ~/.config/ansible:

mkdir -p ~/.config/ansible
echo <my_token_value> > ~/.config/ansible/.vault-token

Configure ansible.cfg:

[hashi_vault_collection]
token_path=~/.config/ansible
url=https://vault.mycompany

Then try to fetch a secret:

ansible 'localhost,'  -m debug -a "msg={{ lookup('community.hashi_vault.vault_read', 'secret/my_vault_secret') }}"

Change the token_path to a hardcoded value:

[hashi_vault_collection]
token_path=/home/myuser/.config/ansible
url=https://vault.mycompany

EXPECTED RESULTS

If the secret exists, the lookup should succeed.

localhost | SUCCESS => 
    msg:
        auth: null
        data:
            secret_value_1: My first secret value
            secret_value_2: My second secret value
        lease_duration: 8294400
        lease_id: ''
        renewable: false
        request_id: 89d93467-1d29-029e-36c8-b37fg5ecb2a3
        warnings: null
        wrap_info: null

If the secret doesn't exist, the lookup will will of course fail, but with a message like this instead:

localhost | FAILED! => 
    msg: 'An unhandled exception occurred while running the lookup plugin ''community.hashi_vault.vault_read''.
        Error was a <class ''ansible.errors.AnsibleError''>, original message: Forbidden:
        Permission Denied to path ''secret/my_vault_secret''.. Forbidden: Permission
        Denied to path ''secret/my_vault_secret''.'

This is the current functionality when using a path that doesn't require expansion.

ACTUAL RESULTS

The lookup fails with the message No Vault Token specified or discovered.

ansible [core 2.15.6]
  config file = /home/myuser/code/ansible-main/ansible.cfg
  configured module search path = ['/home/myuser/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/myuser/code/ansible-main/.venv/lib/python3.9/site-packages/ansible
  ansible collection location = /home/myuser/code/ansible-main
  executable location = /home/myuser/code/ansible-main/.venv/bin/ansible
  python version = 3.9.5 (default, Nov 23 2021, 15:27:38) [GCC 9.3.0] (/home/myuser/code/ansible-main/.venv/bin/python)
  jinja version = 3.1.2
  libyaml = True
Using /home/myuser/code/ansible-main/ansible.cfg as config file
Reading vault password file: /home/myuser/.config/ansible/toolchestpoc.vaultpw
setting up inventory plugins
Loading collection ansible.builtin from 
host_list declined parsing /home/myuser/code/ansible-main/hosts as it did not pass its verify_file() method
script declined parsing /home/myuser/code/ansible-main/hosts as it did not pass its verify_file() method
auto declined parsing /home/myuser/code/ansible-main/hosts as it did not pass its verify_file() method
Parsed /home/myuser/code/ansible-main/hosts inventory source with ini plugin
Loading callback plugin minimal of type stdout, v2.0 from /home/myuser/code/ansible-main/.venv/lib/python3.9/site-packages/ansible/plugins/callback/minimal.py
Attempting to use 'default' callback.
Skipping callback 'default', as we already have a stdout callback.
Attempting to use 'junit' callback.
Attempting to use 'minimal' callback.
Skipping callback 'minimal', as we already have a stdout callback.
Attempting to use 'oneline' callback.
Skipping callback 'oneline', as we already have a stdout callback.
Attempting to use 'tree' callback.
Loading collection community.hashi_vault from /home/myuser/code/ansible-main/ansible_collections/community/hashi_vault
exception during Jinja2 execution: Traceback (most recent call last):
  File "/home/myuser/code/ansible-main/ansible_collections/community/hashi_vault/plugins/lookup/vault_read.py", line 121, in run
    self.authenticator.validate()
  File "/home/myuser/code/ansible-main/ansible_collections/community/hashi_vault/plugins/module_utils/_authenticator.py", line 98, in validate
    method.validate(*args, **kwargs)
  File "/home/myuser/code/ansible-main/ansible_collections/community/hashi_vault/plugins/module_utils/_auth_method_token.py", line 81, in validate
    raise HashiVaultValueError("No Vault Token specified or discovered.")
ansible_collections.community.hashi_vault.plugins.module_utils._hashi_vault_common.HashiVaultValueError: No Vault Token specified or discovered.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/myuser/code/ansible-main/.venv/lib/python3.9/site-packages/ansible/template/__init__.py", line 868, in _lookup
    ran = instance.run(loop_terms, variables=self._available_variables, **kwargs)
  File "/home/myuser/code/ansible-main/ansible_collections/community/hashi_vault/plugins/lookup/vault_read.py", line 124, in run
    raise AnsibleError(e)
ansible.errors.AnsibleError: No Vault Token specified or discovered.
localhost | FAILED! => 
    msg: 'An unhandled exception occurred while running the lookup plugin ''community.hashi_vault.vault_read''.
        Error was a <class ''ansible.errors.AnsibleError''>, original message: No Vault
        Token specified or discovered.. No Vault Token specified or discovered.'

The problem seems to be that the plugin doesn't do tilde expansion when validating token_path. The token_path and token_file values are simply path-joined and handed to read(), which doesn't natively handle tilde expansion. That should be handled beforehand by calling os.path.expanduser() and os.path.expandvars(). That is how it is handled by the built-in plugins. Maybe we can use unfrackpath() from ansible.utils.path here as well?

[briantist]

Hi @michaelin welcome and thank you for the very detailed report! I'll look into using one of the methods you suggested.
If you are interested in putting up a PR yourself, let's chat here first about which method we might use.

---

A possible workaround for you at the moment might be to resolve the path you want before passing it, for example using the env lookup: https://docs.ansible.com/ansible/latest/collections/ansible/builtin/env_lookup.html#examples

- name: read a path
  vars:
    homedir: "{{ lookup('ansible.builtin.env', 'HOME') }}"
    vault_token_path: "{{ homedir }}/.config/ansible"
  ansible.builtin.debug:
    msg: "{{ lookup('community.hashi_vault.vault_read', 'secret/my_vault_secret', token_path=vault_token_path) }}"

I realize that this is not ideal, since you wanted to set the directory within the config file, and this requires that you set vars somewhere, and execute additional lookups (in the above the vars on the task, but they could be on a block, play, role entry, etc.), but it is a possible workaround.

Also important is that this particular workaround is only appropriate for lookups or modules executed on the controller, since it gets the value of HOME from the controller. It's possible you could do a similar workaround with modules on remote hosts by using fact gathering.