Issues parsing $ symbol

[modem7]

Hey guys,

Got a weird one, might be user error.

I've got a service I'm trying to parse a bcrypt password through:

  timetagger:
    image: ghcr.io/almarklein/timetagger
    container_name: Timetagger
    networks:
      pihole:
        ipv4_address: '172.22.0.137'
    ports:
      - "43841:80"
    volumes:
      - $USERDIR/Timetagger:/root/_timetagger
    environment:
      - TIMETAGGER_BIND=0.0.0.0:80
      - TIMETAGGER_DATADIR=/root/_timetagger
      - TIMETAGGER_LOG_LEVEL=info
      - TIMETAGGER_CREDENTIALS=myuser:$$2a$$08$$qFFehmqHMmIwR91g77N9xOc.eA8pkRE64zXwJo/AR/fWTbGrEgHEm

If I enter the credentials as above, it works as expected.

If I try entering it as an env var it does not:

  timetagger:
    image: ghcr.io/almarklein/timetagger
    container_name: Timetagger
    networks:
      pihole:
        ipv4_address: '172.22.0.137'
    ports:
      - "43841:80"
    volumes:
      - $USERDIR/Timetagger:/root/_timetagger
    environment:
      - TIMETAGGER_BIND=0.0.0.0:80
      - TIMETAGGER_DATADIR=/root/_timetagger
      - TIMETAGGER_LOG_LEVEL=info
      - TIMETAGGER_CREDENTIALS=$TIMETAGUSER:$TIMETAGPASS

.env:

#Timetagger
TIMETAGUSER="myuser"
TIMETAGPASS="$$2a$$08$$qFFehmqHMmIwR91g77N9xOc.eA8pkRE64zXwJo/AR/fWTbGrEgHEm"

docker compose show config outputs the following:

  timetagger:
    container_name: Timetagger
    environment:
      TIMETAGGER_BIND: 0.0.0.0:80
      TIMETAGGER_CREDENTIALS: myuser:$$a$$$$$$qFFehmqHMmIwR91g77N9xOc.eA8pkRE64zXwJo/AR/fWTbGrEgHEm
      TIMETAGGER_DATADIR: /root/_timetagger
      TIMETAGGER_LOG_LEVEL: info

And obviously, if I don't escape the "$", I get the expected problem:

TIMETAGGER_CREDENTIALS: myuser:a$$qFFehmqHMmIwR91g77N9xOc.eA8pkRE64zXwJo/AR/fWTbGrEgHEm

Any thoughts?

Docker Compose version v2.6.0

Client: Docker Engine - Community
 Version:           20.10.17
 API version:       1.41
 Go version:        go1.17.11
 Git commit:        100c701
 Built:             Mon Jun  6 23:02:57 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.17
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.17.11
  Git commit:       a89b842
  Built:            Mon Jun  6 23:01:03 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.6
  GitCommit:        10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1
 runc:
  Version:          1.1.2
  GitCommit:        v1.1.2-0-ga916309
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

OS: Ubuntu 20.04 LTS

[kiere]

Since I upgraded to Docker Desktop for Mac 4.11 (from 4.9, I believe), this is now happening to me. Compose gives "Invalid template" errors if there are $ in the values of Environment variables. I have two passwords in my .env file that have dollar signs and Compose is trying to interpolate them as substitutions.

This only started happening after the upgrade, but we also moved from .envrc with direnv to just a .env since Python's pipenv and other tools work with .env. So it's possible this was happening before, but we didn't notice because we were telling compose to use .envrc via the env_file option.

Actually, I just tested reverting to .envrc and renaming .env to env.backup and docker compose version will now work fine, but docker compose down will blow up with the "Invalid template" error same as the .env (the both have the same password in the same ENV var).

So this is definitely new post-upgrade.

[laurazard]

Thanks for the report!

As per @ulyssessouza's message here:

In Docker Compose V2 dollar signs have to be escaped by doubling it ($$).
Another solution would be surrounding the whole string with single quotes (''), like in:

JWS_KEY='WSӠ$8É*X&*8HѲk!^£'

Can you try that and see if it works?

[modem7]

Hi @laurazard,

Alrighty, looks like we're cooking.

The documentation is a bit confusing as it mentions it requires escaping.

But your solution of single quote in the env file rather than double, without escaping seems to have worked well.

Thank you.

[laurazard]

I'm glad it worked! We'll look into making the documentation clearer.

[modem7]

I'm glad it worked! We'll look into making the documentation clearer.

Thank you!

In future, for the .env file in Docker Compose V2, would you recommend single quoting or double quoting the content?

E.g. (in a normal, non-edge case unlike the above):

#HDDTemp
HDDTEMPARGS="-q -d -F /dev/sd*"

or

#HDDTemp
HDDTEMPARGS='-q -d -F /dev/sd*'

[Glioburd]

Thanks, I was struggling on a similar problem in one of my .env files since I upgraded Compose to 2.9. You saved me a lot of time 😬 .

[kiere]

@laurazard Can you elaborate on why this just started happening though? Other tools use the .env file and have no such requirement to use single quotes. I can see this being confusing as new team members come on board using double quotes as usual only to have us explain they need to switch the entire .env file to single quotes. Rinse and repeat. I'm wondering why a change like this wouldn't be an opt-in with a command-line flag or other setting instead of being the default.

[accforgithubtest]

wow, this is such an obscure and hidden requirement that I must guess is wasting a lot of peoples time (at-least in my case it has been nothing but a frustrating chase to find the cause where all docker applications lost all db connectivity) !

There are many cases where password generation is offloaded to scripts that actually enforce that a password must contain symbols (additionally $ is probably also used quite a lot manually). This creates a scenario where it makes it very hard to find out that a $ somewhere is causing issues with the compose file.

1. Is there any chance compose will handle $ properly in the future without needing escapes / workarounds ?
2. Can compose at-least warn users that it has found the use of a special character that might be parsed incorrectly and send the users to a clear documentation ?
3. Are there any other symbols / characters that need to be escaped ?

Also as requested above, would like to understand if there is any reasoning behind this obscure issue with the $ needing to be escaped.