"chezmoi update" prompts for password even though encrypted file has not changed

[lefth]

Describe the bug

Two machines were pretty much in sync. On one machine I added a directory with "chezmoi add" and "chezmoi cd" then git pushing. On the next machine I tried to get these files with "chezmoi update". I was prompted for the password.

To reproduce

1. Configure chezmoi to use gpg with symmetric encryption on two machines. (I used two WSL distros on the same machine.)
2. Add a file to chezmoi with encryption.
3. Push the repository to git and get both machines in sync.
4. On machine A, add an unencrypted file, for example chezmoi add ~/.bashrc and push the change to the git remote.
5. On machine B, run chezmoi update

Expected behavior

I expect to not be prompted when encrypted files haven't been changed. Perhaps this could be accomplished by (optionally) storing a checksum along with the encrypted data.

I suggest an option because in some circumstances that would be too much information leakage.

In any case, so much password prompting is not reasonable--I have several encrypted files, so I get prompted several times, even if only one file needs to be updated (and it's not encrypted).

Output of command with the --verbose flag

$ chezmoi update --verbose
Enter passphrase for key '/home/user/.ssh/id_rsa':
Already up to date.
gpg: AES256.CFB encrypted data
gpg: gcry_kdf_derive failed: Invalid data
gpg: encrypted with 1 passphrase
gpg: decryption failed: Bad session key
chezmoi: home/user/.secret: exit status 2

Output of chezmoi doctor

$ chezmoi doctor
RESULT   CHECK                       MESSAGE
ok       version                     v2.48.0, commit c758a1c57fb6084631736a7dda5ac0aaeeffa946, built at 2024-04-26T20:38:57Z, built by goreleaser
ok       latest-version              v2.48.0
ok       os-arch                     linux/amd64 (Ubuntu 24.04 LTS (Noble Numbat))
ok       uname                       Linux LAPTOP-DMH3HPFI 5.15.146.1-microsoft-standard-WSL2 #1 SMP Thu Jan 11 04:09:03 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
ok       go-version                  go1.22.2 (gc)
ok       executable                  /snap/chezmoi/876/chezmoi
ok       upgrade-method              snap-refresh
ok       config-file                 ~/.config/chezmoi/chezmoi.toml, last modified 2024-05-02T01:57:03+08:00
ok       source-dir                  ~/.local/share/chezmoi is a git working tree (clean)
ok       suspicious-entries          no suspicious entries
ok       working-tree                ~/.local/share/chezmoi is a git working tree (clean)
ok       dest-dir                    / is a directory
ok       umask                       022
ok       cd-command                  found /bin/zsh
ok       cd-args                     /bin/zsh
info     diff-command                not set
ok       edit-command                found /usr/bin/nvim
ok       edit-args                   /usr/bin/nvim
ok       git-command                 found /usr/bin/git, version 2.43.0
ok       merge-command               found /usr/bin/vimdiff
ok       shell-command               found /bin/zsh
ok       shell-args                  /bin/zsh
info     age-command                 age not found in $PATH
ok       gpg-command                 found /usr/bin/gpg, version 2.4.4
info     pinentry-command            not set
info     1password-command           op not found in $PATH
info     bitwarden-command           bw not found in $PATH
info     bitwarden-secrets-command   bws not found in $PATH
info     dashlane-command            dcli not found in $PATH
info     doppler-command             doppler not found in $PATH
info     gopass-command              gopass not found in $PATH
info     keepassxc-command           keepassxc-cli not found in $PATH
info     keepassxc-db                not set
info     keeper-command              keeper not found in $PATH
info     lastpass-command            lpass not found in $PATH
info     pass-command                pass not found in $PATH
info     passhole-command            ph not found in $PATH
info     rbw-command                 rbw not found in $PATH
info     vault-command               vault not found in $PATH
info     vlt-command                 vlt not found in $PATH
info     secret-command              not set

Additional context

Note: I set the destination to be / in the config file. I don't think that is relevant to the problem.

[halostatue]

This is not a bug. Your suggestion (storing a checksum/hash) is an interesting one, but it is not risk-free, either.

In general, with anything templated via a password manager or encrypted, chezmoi cannot know whether the file has changed until the values are unlocked and templated or decrypted. If you have two classes of changes, you may want to look at #2574 for suggestions on how to layer your configuration so that your default storage is free of encrypted data and you have a second repo that only has encrypted data.

Your suggestion is interesting, and it might be possible to add a checksum / digest field to the state database for encrypted data, but:

1. it will make every encryption step slower, because chezmoi would need to read the file and compute a pre-encryption digest before encrypting the file (possibly through a shell call).
2. it may incorrectly decide that something has not changed.

The pre-encryption digest needs to be used because encrypting the same file at two different times with the same password may not result in the same bytes on output (different salts, etc.).

[twpayne]

As @halostatue says, chezmoi has to retrieve the secrets from your password manager to know if the file's contents have changed. There is no way to avoid this.

In any case, so much password prompting is not reasonable--I have several encrypted files, so I get prompted several times, even if only one file needs to be updated (and it's not encrypted).

The solution here is to use a password manager that supports some kind of temporary persistent session so you don't have to enter your password every time.

As you're currently using GPG you might be able to adapt this technique from age.

However, this path leads to accidentally implementing your own password manager, which I wouldn't recommend. Better to use to an existing off-the-shelf solution, which is likely to be much more secure.

Another option is encrypt entire files and then use the --exclude=encrypted flag to skip encrypted files when running chezmoi apply, but this will you to know when your files containing secrets have changed.

[lefth]

@twpayne @halostatue Thanks for all the tips, I will try your suggestions!

As for knowing whether an encrypted file has changed, there is no need to decrypt the encrypted contents if checksums are stored. If the tips provided work well (secure, good UX, cross platform) there is no need to pursue this, but the strategy that jumps out is: during encryption dot_ssh/id_rsa.chezmoichecksum would be created containing three lines: "v1", a random salt, and the sha256sum of ~/.ssh/id_rsa plus the salt.

A checksum would indeed need to be performed to check whether the file is changed, but checksums are generally faster than decryption. (So extra computation would only be needed when the file is changed--in that case both checksumming and encryption and/or would be needed.)

Checksum collisions aren't currently a concern. The negative media coverage a few years ago about md5sum being broken was exaggerated (tampering with one file to force a checksum collision still isn't possible), and sha256sum is much stronger.

But like I said, I will experiment with the setups you suggested.

[halostatue]

As for knowing whether an encrypted file has changed, there is no need to decrypt the encrypted contents if checksums are stored. If the tips provided work well (secure, good UX, cross platform) there is no need to pursue this, but the strategy that jumps out is: during encryption dot_ssh/id_rsa.chezmoichecksum would be created containing three lines: "v1", a random salt, and the sha256sum of ~/.ssh/id_rsa plus the salt.

A checksum would indeed need to be performed to check whether the file is changed, but checksums are generally faster than decryption. (So extra computation would only be needed when the file is changed--in that case both checksumming and encryption and/or would be needed.)

If we were to pursue a digest approach on this, it would need to be stored in the state file, not side-by-side with the decrypted file or in the source tree. The difficulty would then be determining how the checksum is associated with the encrypted file in such a way as to (a) not interfere with the decryption process and (b) not require additional files.

I think that this is an interesting concept, but should only be pursued as a possible v3 concept as there would likely need to be changes in how chezmoi handles encryption/decryption to add chezmoi envelopes for such a digest. I don’t know if it’s something we would pursue in the end, because right now the encryption is (more or less) pass-through to the underlying libraries and/or binaries.