Control characters printed to shell prompt with 'docker logs' command

[pcgeek86]

Description

Control characters are being printed to the shell prompt when running docker logs.

Steps to reproduce the issue:

1. Run command: docker run --name ps --interactive --tty microsoft/powershell
2. In PowerShell Core interactive prompt, run command: 1..999 | % { echo $_; sleep 1 }
3. Detach from the container CTRL + P, CTRL + Q
4. Run docker logs ps

Describe the results you received:

After running step #4 in the repro, I receive some control characters printed to the shell prompt.

Describe the results you expected:

I would expect the shell prompt to be empty, after running the docker logs command.

Output of docker version:

Docker Version Output

Client:
 Version:      1.13.0-rc5
 API version:  1.25
 Go version:   go1.7.3
 Git commit:   43cc971
 Built:        Thu Jan  5 03:07:30 2017
 OS/Arch:      darwin/amd64

Server:
 Version:      1.13.0-rc5
 API version:  1.25 (minimum version 1.12)
 Go version:   go1.7.3
 Git commit:   43cc971
 Built:        Thu Jan  5 03:07:30 2017
 OS/Arch:      linux/amd64
 Experimental: true

Output of docker info:

Docker Info Output

Containers: 5
 Running: 2
 Paused: 0
 Stopped: 3
Images: 5
Server Version: 1.13.0-rc5
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host ipvlan macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 03e5862ec0d8d3b3f750e19fca3ee367e13c090e
runc version: 51371867a01c467f08af739783b8beafc154c4d7
init version: 949e6fa
Security Options:
 seccomp
  Profile: default
Kernel Version: 4.8.15-moby
Operating System: Alpine Linux v3.5
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.952 GiB
Name: moby
ID: QORF:4VZP:OGR2:LSRT:BMRM:BAZA:QK2G:CHEU:TNFK:LANG:NQD2:IVHP
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 29
 Goroutines: 36
 System Time: 2017-01-13T16:28:34.777352585Z
 EventsListeners: 1
Registry: https://index.docker.io/v1/
Experimental: true
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.):

• Macbook Pro Early 2015
• Mac OS X Sierra
• Docker for Mac

Cheers,
Trevor Sullivan
Docker Captain
Microsoft MVP: Cloud & Data Center Management
https://trevorsullivan.net
https://twitter.com/pcgeek86

[thaJeztah]

I'm able to reproduce this; tested both on Docker for Mac and on an Ubuntu machine

ping @ijc25 @dperny @johnstep any ideas? May be specific to powershell?

[thaJeztah]

(fwiw - testing on 17.06.0-rc1)

[dperny]

Same, I can repro. Almost certainly related to powershell doing something weird w/ control characters. Getting logs for a TTY is a mess if there are control characters involved; try doing docker logs on a container running top with tty set and see what madness that can bring.

[dperny]

Yeah it's 100% control characters borking the hell out of the terminal.

[ijc]

I see it too. Also:

$ docker run --interactive --tty --rm microsoft/powershell '1..2 | % { echo $_; sleep 1 }'
1
2
$ docker run --interactive --tty --rm microsoft/powershell '1..2 | % { echo $_; sleep 1 }' | cat -vet
^[[?1h^[=1^M$
             ^[[?1h^[=^[]0;^G2^M$
                                 ^[[?1h^[=^[]0;^G

Need to lookup what those escape codes do but apparently one of them is a TIOCSTI type thing which pushes stuff into the terminals input...

[ijc]

The issue is that the Powershell prompt includes several ^[[6n (where ^[ is an escape, 0x1b). This is, according to this page a DSR or Get Cursor Position, which elicits the response ^[[«LINE»;«COLUMN»R (where «LINE» and «COLUMN» are digits), i.e. ^[[8;1R as seen here. You can try this with echo -e '\e[6n'.

AFAICT the 1..999 | % { echo $_; sleep 1 } is not necessary. It is sufficient to:

• Run docker run --name ps --interactive --tty microsoft/powershell
• Press Enter once
• Press Ctrl-P then Ctrl-Q to detach
• Run docker log ps

There are several instance of the ^[[6n pattern in the prompt.

[ijc]

Not PS specific, you can reproduce e.g. with:

$ PS1='«PROMPT»'
«PROMPT»docker run --name bash debian:jessie /bin/bash -c 'echo -e \\e[6n'

^[[10;1R«PROMPT»;1R
bash: syntax error near unexpected token `;'
«PROMPT»docker logs bash

^[[14;1R«PROMPT»;1R
bash: syntax error near unexpected token `;'
«PROMPT»

So neither --interactive nor --tty are necessary either.

[ijc]

It's not clear to me that this is a bug, or if it were what docker could do about it. One person's escape code is another person's binary data, so in trying to "fix" the former we would inevitably be corrupting the latter.

[thaJeztah]

@ijc25 is it that PowerShell uses different escape codes than a Linux shell? So, when printing the same logs from a PowerShell session on the host, it properly "translates"?

[ijc]

On bash the prompt is just whatever is in $PS1, which defaults to something fairly conservative and which doesn't contain any escape codes.

I don't know much about PowerShell, including where gets its prompt from (it's nothing I can see in the output of env), but evidently it does include some escape codes, for whatever reason...

[ijc]

BTW I'm not 100% sure this is about the "prompt" per-se, but at the least it is about outputting of escape codes in the normal course of using the shell.

[dperny]

Yeah I'm gonna agree and says notabug/wontfix.

[thaJeztah]

Agreed; still curious how it acts on a PowerShell shell on the host (but that's just my curiosity) 😄

@pcgeek86 I'm closing this per the discussion above, but thanks so much for reporting; also feel free to continue the discussion 👍

[ijc]

A feature request for a docker logs option which escaped control characters (a la cat -vet, which I see now is equivalent to the less memorable cat -et or cat -vET) would not be unreasonably IMHO.

@thaJeztah When running on the host and shell (including PowerShell) will have those escape codes captured and interpreted by the terminal and PowerShell will, presumably, be expecting and thus consume/act upon the answering escape codes rather than accepting them as regular input. The issue with docker logs is that the host shell is not generating the escape codes (they are just in the output stream) so it isn't expecting the answer and so as far as it knows it is just weird user input.

[thaJeztah]

something like docker logs --clean ? Or reverse it to docker logs --raw to include control-characters

[ijc]

I have no particular opinion on the naming or the default, but yes, that sort of thing.

[solvingj]

I believe we've encountered a related problem when running:

• On a Windows Agent
• A Jenkins "freestyle" job
• Which executes a Batch Script
• Which calls "docker exec" commands

�[H�]0;C:\Windows\system32\cmd.exe��[?25h�[?25l�[80X�[80C
�[80X�[80C
�[80X�[80C
�[80X�[80C

Of note, "pipeline" jobs which use a Jenkinsfile don't exhibit the problem.

We've opened an issue with Cloudbee's who do not seem to have encountered this issue before.

Can anyone confirm that this might be related to this issue? Or should I open a new issue?

[thaJeztah]

@solvingj if that's the output of docker exec (and it's running attached) that's expected, as it will attach to the container and output the raw terminal output

[DaweiCai]

@thaJeztah we also met this issue on windows run "docker exec", how to mitigate this issue? I mean remove the unwanted from output

[DaweiCai]

@solvingj may I know have you resolved the issue?

[thaJeztah]

These answers may help if you're using PowerShell; https://stackoverflow.com/a/51681675 https://stackoverflow.com/a/51683794

[thaJeztah]

I see there's some discussion around ANSI escape sequences in the PowerShell issue tracker here; PowerShell/PowerShell#13071

[solvingj]

For posterity, my issue was simply because I was passing -t , not related to other issues linked here. I removed the flag and don't get the undesired characters in jenkins.

[thaJeztah]

Yes, processes in the container may produce different output depending on if h there's a TTY attached to the container or not.

Good to hear you resolved the situation for your use case

[diepes]

I just had the same issue, running linux container with aws cli in it.
alias aws='docker run --rm -it -v ${HOME}/.aws:/root/.aws -v $(pwd):/aws -e AWS_PROFILE amazon/aws-cli:2.1.4'
aws sts get-session-token --serial-number ${mfa_device_arn} --token-code ${mfa_token_code} --output json

removing the "t" (from -it) fixed my issue.

e.g.

$ cat -v delme.txt 
^[[?1h^[= {^[[m 
    "Credentials": {^[[m 

remove -t

$ cat -v delme.txt 
{
    "Credentials": {