Allow token auto renew when playbook is executing

[1ovsss]

SUMMARY

It would be nice if hashi_vault will auto renew token during ansible execution uses token with configured token_ttl and token_max_ttl.

ISSUE TYPE

• Feature Idea

COMPONENT NAME

lookup

ADDITIONAL INFORMATION

let's say i have following vault configuration:

      token_ttl           = 900   #15m
      token_max_ttl     = 10800 #3h

i get token like this:

% curl -X POST -H 'Content-Type: application/json' -d \ 
    '{ "password": "'"$LDAP_PASS"'" }' $VAULT_ADDR/v1/auth/ldap/login/${LDAP_USER}
---
{
  "request_id": "bxxx3-xxx-xxxx-xxxx-9xxxx4031",
  "lease_id": "",
  "renewable": false,
  "lease_duration": 0,
  "data": {},
  "wrap_info": null,
  ],
  "auth": {
    "client_token": "hvs.token",
    "accessor": "RRRxxxxhfaj",
    "policies": [
      "policies"
    ],
    "token_policies": [
      "policies"
    ],
    "metadata": {
      "username": "myusername"
    },
    "lease_duration": 900,
    "renewable": true,
    "entity_id": "5xxxb9-xxx-xxx-xxx-x593xxxd8e",
    "token_type": "service",
    "orphan": true,
    "mfa_requirement": null,
    "num_uses": 0
  }
}

This token has access to /sys/token/* so it can renew itself (i checked with curl) and from output above i assume "renewable": true also makes it possible.

but tasks below:

tasks:
    - name: message
      block:
        - name: sec1
          debug:
            msg: "{{ lookup('hashi_vault', 'secret=some/secret/path:sec1') }}"

        - name: sleep
          command: sleep 800

        - name: sec2
          debug:
            msg: "{{ lookup('hashi_vault', 'secret=some/secret/path:sec2') }}"

        - name: sleep
          command: sleep 200

        - name: sec3
          debug:
            msg: "{{ lookup('hashi_vault', 'secret=some/secret/path:sec3') }}"
      
      tags: [ sleep-vlt ]

where 800+200 > 900, gives me:

TASK [sec1] ********************************************************************
[WARNING]: Collection community.hashi_vault does not support Ansible version
2.12.10
ok: [test.com] => {
    "msg": "secret1"
}

TASK [sleep] *******************************************************************
changed: [test.com]

TASK [sec2] ********************************************************************
[WARNING]: Collection community.hashi_vault does not support Ansible version
2.12.10
ok: [test.com] => {
    "msg": "secret2"
}

TASK [sleep] *******************************************************************
changed: [test.com]

TASK [sec3] ********************************************************************
[WARNING]: Collection community.hashi_vault does not support Ansible version
2.12.10
fatal: [test.com]: FAILED! => {"msg": "An unhandled exception occurred while running the lookup plugin 'hashi_vault'. Error was a <class 'ansible.errors.AnsibleError'>, original message: Forbidden: Permission Denied to secret 'some/secret/path'.. Forbidden: Permission Denied to secret 'some/secret/path'."}

PLAY RECAP *********************************************************************
test.com          : ok=11   changed=2    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

[briantist]

Hi @1ovsss , welcome!

This is similar/related to an older request related to caching auth:

• Support "caching" token after initial login #104

In particular this explanation about using Vault Agent with auto-auth:

• Support "caching" token after initial login #104 (comment)

---

Why renewal is difficult

Since the code for each lookup and module executes independently, each invocation has to somehow discover or know that a token needs to be renewed before it could make the extra call. Alternatively, we can ask for forgiveness by first trying to use the token and then attempting renewal if it ends up invalid, but we don't know whether the token is renewable when we try to do that. If we try to discover first, like looking up the token, then we do an extra, probably unnecessary roundtrip to the server on every invocation just to see if we should renew or not. Additionally, some tokens do not have the ability to lookup information about themselves so that method won't work on those at all. Some of this can be mitigated with options to control behavior, which unfortunately also bloats the number of options, even more than it already is. If we had extra metadata passed in other than just the token itself, like when it was issued, what the TTL is, etc., it could help, but there's no way to pass it in now, and it means either changing the way token is accepted, or adding another option for the extra data. Then we still need to handle the case where it's missing. If the token is created by one of our plugins, it might seem like at least in that case we could keep that information, but there's no reliable persistent data store between ansible plugin runs, so we'd be designating something for that too (probably a file, or files, because we can't actually assume a single token is used on all hosts).

So, it is possible that it could be achievable, but there are a lot of thorny issues around it, so I don't have much in the way of plans for it right now.

Vault Agent is designed specifically to handle renewals, so it works well running alongside Ansible with this collection.

---

More workarounds

Another option might be to intersperse your tasks with calls to vault_write, use that to renew your token. I haven't tested this but it might look something like this.

- name: renew
  run_once: true  # if you use different tokens per host, this should be false; true prevents unnecessary renewals against hosts in parallel 
  ansible.builtin.set_fact:
    ansible_hashi_vault_token: "{{ lookup('community.hashi_vault.vault_write', 'auth/token/renew-self').auth.client_token }}"

You could repeat that task wherever, you just have to be careful about variable precedence.

It would be nicer if we could set it once in a vars section, but the problem right now is that lookups don't execute templating on the vars that are used for options automatically, so the following won't work even though it should:

vars:
  ansible_hashi_vault_token: "{{ lookup('community.hashi_vault.vault_write', 'auth/token/renew-self').auth.client_token }}"
tasks:
    - name: message
      block:
        - name: sec1
          debug:
            msg: "{{ lookup('hashi_vault', 'secret=some/secret/path:sec1') }}"

        - name: sleep
          command: sleep 800

        - name: sec2
          debug:
            msg: "{{ lookup('hashi_vault', 'secret=some/secret/path:sec2') }}"

        - name: sleep
          command: sleep 200

        - name: sec3
          debug:
            msg: "{{ lookup('hashi_vault', 'secret=some/secret/path:sec3') }}"
      
      tags: [ sleep-vlt ]

(does not work because the lookup is never executed)

---

If you're willing to modify every task, you can force the templating by adding the token explicitly:

vars:
  my_token: "{{ lookup('community.hashi_vault.vault_write', 'auth/token/renew-self').auth.client_token }}"
tasks:
    - name: message
      block:
        - name: sec1
          debug:
            msg: "{{ lookup('hashi_vault', 'secret=some/secret/path:sec1', token=my_token) }}"

        - name: sleep
          command: sleep 800

        - name: sec2
          debug:
            msg: "{{ lookup('hashi_vault', 'secret=some/secret/path:sec2', token=my_token) }}"

        - name: sleep
          command: sleep 200

        - name: sec3
          debug:
            msg: "{{ lookup('hashi_vault', 'secret=some/secret/path:sec3', token=my_token) }}"
      
      tags: [ sleep-vlt ]