Hiding secrets in logs

[FooBarTrixibell]

I don't know if I am just missing something, but is there a way, other than setting no_log on the play, to hide secrets in the hashi_vault collection?

With non-debug output in AWX, whenever you use an AWX credential it hides the passwords in the logs -

  "secret_id": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",

But reading and writing secrets with the hashi_vault plugin, the secret by default is written in plain text to the log. I assume there is a setting I am missing?

[briantist]

Hi @FooBarTrixibell , are you referring to a lookup plugin or a module?

[PostChrO]

This thread is a bit dated, but we see the same behavior here.

Using community.hashi_vault.vault_read module, we can see the whole structure read from the vault in clear text in the Ansible Logs (AAP).

For now we are using "no_log: true" as a workaround. But this is error prone.

Is there any way to hide secret information read from the vault, by default?

Thanks.
Cheers,
Chris

[briantist]

Hi @PostChrO , ultimately this comes down to how Ansible works. Modules execute outside the Ansible controller process (usually on a remote host, but even when executed on the controller, they are in a fork). The sensitive information read by the module must be returned to ansible in order to be able to use it or do anything with it, and it's the controller that's doing the logging.

There is currently no way to mark specific return values as sensitive in a module.

We can mark options (inputs) as sensitive, which actually ends up preventing us from returning that specific value as well, but that won't help for what you're describing.

no_log is the correct (only) way to handle this. You can also template the value of no_log so that you can control it with a variable if that helps you no_log: '{{ not (skip_no_log | default(False) | bool) }}'.

How has no_log been error prone for you?

[PostChrO]

Hey Brian,

thank you for the fast reply and the insights into the reasons why things work as they work.
So in fact this should be a feature request for Ansible then, but that's a different topic.

The "problem" about no_log is, that it tends to be forgotten or just to be unknonwn by people (occassionally) writing playbooks. And even when running them on AAP it's not very obvious to them, that something sensitive is actually leaked into the logs, as it is not directly written out to stdout. Instead they would have to open the detailed "JSON output" of the specific task, to see the leaked information. And if the person doing the code review, before merging the code back into the main development branch, also overlooks it, or is not aware of all the possibly sensitive cases, they can slip through.

Without having it rechecked: Using the lookup function if possible, should be the saver way to go, I guess? (There should not be any logs written during the lookup) Still have to make sure other modules hide sensitive input parameters properly. ;)

Thanks.
Cheers,
Chris

[briantist]

I'm not a user of AAP so I don't know its feature set very well, but it's possible it has something to help with this? There's also been a feature long in development in ansible-core called data "tagging" or something that could possibly help at some point but I don't have a lot of detail on that.

I might suggest posting about this on the Ansible forum since you'll find more information there about you might handle sensitive data in ansible/AAP.

[PostChrO]

ok, thanks for pointing me there. For hiding the sensitive information, ansible would have to know, that it's sensitive first. And as long as there's no machenism for the module to mark returned information as "sensitive", we're back to no_log. I think I might implement a custom ansible_lint rule, to check if no_log is active for some relevant modules.

Thanks so far.
Cheers,
Chris

[briantist]

For hiding the sensitive information, ansible would have to know, that it's sensitive first

It's possible that there are other features, like "always mark output of module <ns.collection.module> as no_log" or something (I don't think this exists just giving an example).

A custom ansible-lint rule might work.

You might also be able to use a custom callback plugin to implement some logic to actually scrub values from the logs, you'd have to look a little more deeply at it.

The forum will be a good place to discuss since it has people from the core engineering team and from across the ansible community.