Double dollar signs in a container's args or env section are incorrectly treated as escape characters

[MartinKanters]

What happened:

I'm trying to pass two dollar sign in the args or env fields of a pod, but they are reduced to one dollar sign.

What you expected to happen:

The two dollar signs should be propagated to the container.

How to reproduce it (as minimally and precisely as possible):

1. Apply this pod:

apiVersion: v1
kind: Pod
metadata:
  name: dollar-dollar-example
  namespace: default
spec:
  containers:
    - name: client-container
      image: k8s.gcr.io/busybox
      env: 
      - name: DOLLAR_DOLLAR_ENV
        value: "$$"
      command: ["sh", "-c"]
      args: 
      - "echo 'dollar dollar from args: $$';
         echo 'dollar dollar from env:' $DOLLAR_DOLLAR_ENV"

2. Check logs

kubectl logs dollar-dollar-example
dollar dollar from args: $
dollar dollar from env: $

Anything else we need to know?:

Kubernetes can expand variables used in the args and env section with the following syntax: $(var). $$(var) is a way of escaping this variable, resulting in it not being expanded. I don't think '$$' should be regarded as something to be escaped, though.
Here's the expansion code, I believe.

I can take a shot at fixing it, but I first want to know for sure that this is not intended behavior.

Environment:

• Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.4", GitCommit:"e87da0bd6e03ec3fea7933c4b5263d151aafd07c", GitTreeState:"clean", BuildDate:"2021-02-22T19:05:30Z", GoVersion:"go1.15.8", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.7", GitCommit:"14f897abdc7b57f0850da68bd5959c9ee14ce2fe", GitTreeState:"clean", BuildDate:"2021-01-22T17:29:38Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}

• Cloud provider or hardware configuration: Azure
• OS (e.g: cat /etc/os-release):
  Ubuntu over WSL

NAME="Ubuntu"
VERSION="18.04.4 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.4 LTS"
VERSION_ID="18.04"

• Kernel (e.g. uname -a):
  Linux MartinK 5.4.72-microsoft-standard-WSL2 #1 SMP Wed Oct 28 23:40:43 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
• Other info:
  This issue was found during debugging what we first thought to be a Tekton issue: Tekton converts double dollar symbols to one tektoncd/pipeline#3871.

[k8s-ci-robot]

@MartinKanters: This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[pacoxu]

For env, see #75907 (comment)

See documentation at https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#envvar-v1-core.

container.args or container.command are similar: https://v1-18.docs.kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#container-v1-core

[rouke-broersma]

For env, see #75907 (comment)

See documentation at https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/#envvar-v1-core.

Hi @pacoxu

The docs do say that $$ can be used to escape vars, but in this case we are not using vars. ANY $$ is escaped to just $, which seems incorrect. Without any knowledge of kubernetes internals I would say it would be simple to detect $$() and only escape this sequence, instead of all $$ sequences.

[neolit123]

/sig api-machinery

[MartinKanters]

/assign

[fedebongio]

/remove-sig api-machinery
/sig node

[thockin]

For posterity, copying comment from the PR:

I disagree that this is a bug (or at worst is a docs bug).

The behavior and syntax was modelled after make and is mostly consistent with that:

$ cat /tmp/ma
VAR := val

all:
	@echo '$(VAR)'
	@echo '$$(VAR)'
	@echo '$$$(VAR)'
	@echo '$$$$(VAR)'
	@echo '$VAR'
	@echo '$$VAR'
	@echo '$$$VAR'
	@echo '$$$$VAR'

$ make -f /tmp/ma
val
$(VAR)
$val
$$(VAR)
AR
$VAR
$AR
$$VAR

$$ -> $

The only place we are different, I think, is that we don't eat the V in $V (I think, can't run a test right now).

Worse, "fixing" this is a breaking change. Someone out there is depending on the current behavior, and to fix one app you silently break another. We can't do that.

[sftim]

I recommend opening a new issue in this repo that covers updating API documentation such as

kubernetes/pkg/apis/core/types.go

Lines 2103 to 2107 in 2112bdd

	// Optional: The docker image's entrypoint is used if this is not provided; cannot be updated.
	// Variable references $(VAR_NAME) are expanded using the container's environment. If a variable
	// cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax
	// can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded,
	// regardless of whether the variable exists or not.

[MartinKanters]

Thanks @sftim , I've created a PR.