Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

VS Code's terminal has started to eat unicode #110506

Closed
Peilonrayz opened this issue Nov 12, 2020 · 6 comments
Closed

VS Code's terminal has started to eat unicode #110506

Peilonrayz opened this issue Nov 12, 2020 · 6 comments
Assignees
@meganrogge
Labels
bug Issue identified by VS Code Team member as probable bug terminal Integrated terminal issues
Milestone
Backlog

Comments

@Peilonrayz
Copy link

  • VSCode Version: 1.50.1 d2e414d x64
  • OS Version: Linux-5.9.8-arch1-1-x86_64-with-glibc2.2.5

Does this issue occur when all extensions are disabled?: Yes

I updated VS Code yesterday and now I'm getting missing glyphs in my prompt. These worked fine before the update.

Steps to Reproduce:

  1. Install fish

    pacman -Syu fish

  2. Install oh-my-fish

    curl -L https://get.oh-my.fish | fish

  3. Install shellder

    omf i shellder

  4. Open VS Code

    code --disable-extensions

Here is a comparison of the hex output that VS Code's terminal, kitty and xterm produce. I typed exit to exit fish in each. I also have a preview of what this looks like in each terminal.

  • VS Code

    VSCode

    Full hexdump 
     /d/p/g/P/darkray ? ? master ● ? fish | xxd
<W> fish: Current terminal parameters have rows and/or columns set to zero.
<W> fish: The stty command can be used to correct this (e.g., stty rows 80 columns 24).
00000000: 071b 5b33 306d 1b28 421b 5b6d 1b5d 303b  ..[30m.(B.[m.]0;
00000010: 6669 7368 202f 6461 7461 2f70 726f 6772  fish /data/progr
00000020: 616d 6d69 6e67 2f67 6974 6875 622f 5065  amming/github/Pe
00000030: 696c 6f6e 7261 797a 2f64 6172 6b72 6179  ilonrayz/darkray
00000040: 1b5b 326d 5e4a 1b28 421b 5b6d 2020 2020  .[2m^J.(B.[m    
00000050: 2020 2020 2020 2020 2020 2020 2020 2020                  
00000060: 2020 2020 2020 2020 2020 2020 2020 2020                  
00000070: 2020 2020 2020 2020 2020 2020 2020 2020                  
00000080: 2020 2020 2020 2020 2020 2020 2020 2020                  
00000090: 2020 2020 2020 2020 2020 0d5e 4a20 200d            .^J  .
000000a0: 1b5b 4b1b 5b3f 3230 3034 6807 1b5b 3330  .[K.[?2004h..[30
000000b0: 6d1b 2842 1b5b 6d1b 5d30 3b66 6973 6820  m.(B.[m.]0;fish 
000000c0: 2f64 6174 612f 7072 6f67 7261 6d6d 696e  /data/programmin
000000d0: 672f 6769 7468 7562 2f50 6569 6c6f 6e72  g/github/Peilonr
000000e0: 6179 7a2f 6461 726b 7261 790d 0d1b 5b34  ayz/darkray...[4
000000f0: 383b 323b 3238 3b32 383b 3238 6d1b 5b33  8;2;28;28;28m.[3
00000100: 383b 323b 3235 353b 3235 353b 3235 356d  8;2;255;255;255m
00000110: 202f 642f 702f 672f 502f 6461 726b 7261   /d/p/g/P/darkra
00000120: 7920 1b5b 3433 6d1b 5b33 383b 323b 3238  y .[43m.[38;2;28
00000130: 3b32 383b 3238 6d3f 201b 5b34 336d 1b5b  ;28;28m? .[43m.[
00000140: 3330 6d3f 206d 6173 7465 7220 e297 8f20  30m? master ... 
00000150: 1b28 421b 5b6d 1b5b 3333 6d3f 201b 2842  .(B.[m.[33m? .(B
00000160: 1b5b 6d1b 5b4b 6508 1b5b 3338 3b32 3b32  .[m.[Ke..[38;2;2
00000170: 3535 3b30 3b30 6d65 1b5b 3330 6d1b 2842  55;0;0me.[30m.(B
00000180: 1b5b 6d1b 5b33 383b 323b 3835 3b38 353b  .[m.[38;2;85;85;
00000190: 3835 6d78 6974 0808 081b 5b33 306d 1b28  85mxit....[30m.(
000001a0: 421b 5b6d 1b5b 3338 3b32 3b32 3535 3b30  B.[m.[38;2;255;0
000001b0: 3b30 6d78 1b5b 3338 3b32 3b38 353b 3835  ;0mx.[38;2;85;85
000001c0: 3b38 356d 6974 0808 1b5b 3330 6d1b 2842  ;85mit...[30m.(B
000001d0: 1b5b 6d1b 5b33 383b 323b 3235 353b 303b  .[m.[38;2;255;0;
000001e0: 306d 691b 5b33 383b 323b 3835 3b38 353b  0mi.[38;2;85;85;
000001f0: 3835 6d74 081b 5b33 306d 1b28 421b 5b6d  85mt..[30m.(B.[m
00000200: 1b5b 3338 3b32 3b32 3535 3b30 3b30 6d74  .[38;2;255;0;0mt
00000210: 1b5b 3330 6d1b 2842 1b5b 6d08 0808 081b  .[30m.(B.[m.....
00000220: 5b33 383b 323b 303b 3935 3b32 3135 6d65  [38;2;0;95;215me
00000230: 7869 741b 5b33 306d 1b28 421b 5b6d 0a1b  xit.[30m.(B.[m..
00000240: 5b33 306d 1b28 421b 5b6d 1b5b 3f32 3030  [30m.(B.[m.[?200
00000250: 346c 071b 5b33 306d 1b28 421b 5b6d 0d1b  4l..[30m.(B.[m..
00000260: 5b33 306d 1b28 421b 5b6d 1b5b 3f32 3030  [30m.(B.[m.[?200
00000270: 346c                                     4l

    000000e0: 6179 7a2f 6461 726b 7261 790d 0d1b 5b34  ayz/darkray...[4
000000f0: 383b 323b 3238 3b32 383b 3238 6d1b 5b33  8;2;28;28;28m.[3
00000100: 383b 323b 3235 353b 3235 353b 3235 356d  8;2;255;255;255m
00000110: 202f 642f 702f 672f 502f 6461 726b 7261   /d/p/g/P/darkra
00000120: 7920 1b5b 3433 6d1b 5b33 383b 323b 3238  y .[43m.[38;2;28
00000130: 3b32 383b 3238 6d3f 201b 5b34 336d 1b5b  ;28;28m? .[43m.[
00000140: 3330 6d3f 206d 6173 7465 7220 e297 8f20  30m? master ... 
00000150: 1b28 421b 5b6d 1b5b 3333 6d3f 201b 2842  .(B.[m.[33m? .(B
00000160: 1b5b 6d1b 5b4b 6508 1b5b 3338 3b32 3b32  .[m.[Ke..[38;2;2

  • kitty

    kitty

    Full hexdump 
     /d/p/g/P/darkray   master ●  fish | xxd
<W> fish: Current terminal parameters have rows and/or columns set to zero.
<W> fish: The stty command can be used to correct this (e.g., stty rows 80 columns 24).
00000000: 071b 5b33 306d 1b28 421b 5b6d 1b5d 303b  ..[30m.(B.[m.]0;
00000010: 6669 7368 202f 6461 7461 2f70 726f 6772  fish /data/progr
00000020: 616d 6d69 6e67 2f67 6974 6875 622f 5065  amming/github/Pe
00000030: 696c 6f6e 7261 797a 2f64 6172 6b72 6179  ilonrayz/darkray
00000040: 1b5b 326d e28f 8e1b 2842 1b5b 6d20 2020  .[2m....(B.[m   
00000050: 2020 2020 2020 2020 2020 2020 2020 2020                  
00000060: 2020 2020 2020 2020 2020 2020 2020 2020                  
00000070: 2020 2020 2020 2020 2020 2020 2020 2020                  
00000080: 2020 2020 2020 2020 2020 2020 2020 2020                  
00000090: 2020 2020 2020 2020 2020 2020 0de2 8f8e              ....
000000a0: 200d 1b5b 4b1b 5b3f 3230 3034 6807 1b5b   ..[K.[?2004h..[
000000b0: 3330 6d1b 2842 1b5b 6d1b 5d30 3b66 6973  30m.(B.[m.]0;fis
000000c0: 6820 2f64 6174 612f 7072 6f67 7261 6d6d  h /data/programm
000000d0: 696e 672f 6769 7468 7562 2f50 6569 6c6f  ing/github/Peilo
000000e0: 6e72 6179 7a2f 6461 726b 7261 790d 0d1b  nrayz/darkray...
000000f0: 5b34 383b 323b 3238 3b32 383b 3238 6d1b  [48;2;28;28;28m.
00000100: 5b33 383b 323b 3235 353b 3235 353b 3235  [38;2;255;255;25
00000110: 356d 202f 642f 702f 672f 502f 6461 726b  5m /d/p/g/P/dark
00000120: 7261 7920 1b5b 3433 6d1b 5b33 383b 323b  ray .[43m.[38;2;
00000130: 3238 3b32 383b 3238 6dee 82b0 201b 5b34  28;28;28m... .[4
00000140: 336d 1b5b 3330 6dee 82a0 206d 6173 7465  3m.[30m... maste
00000150: 7220 e297 8f20 1b28 421b 5b6d 1b5b 3333  r ... .(B.[m.[33
00000160: 6dee 82b0 201b 2842 1b5b 6d1b 5b4b 6508  m... .(B.[m.[Ke.
00000170: 1b5b 3338 3b32 3b32 3535 3b30 3b30 6d65  .[38;2;255;0;0me
00000180: 1b5b 3330 6d1b 2842 1b5b 6d1b 5b33 383b  .[30m.(B.[m.[38;
00000190: 323b 3835 3b38 353b 3835 6d78 6974 0808  2;85;85;85mxit..
000001a0: 081b 5b33 306d 1b28 421b 5b6d 1b5b 3338  ..[30m.(B.[m.[38
000001b0: 3b32 3b32 3535 3b30 3b30 6d78 1b5b 3338  ;2;255;0;0mx.[38
000001c0: 3b32 3b38 353b 3835 3b38 356d 6974 0808  ;2;85;85;85mit..
000001d0: 1b5b 3330 6d1b 2842 1b5b 6d1b 5b33 383b  .[30m.(B.[m.[38;
000001e0: 323b 3235 353b 303b 306d 691b 5b33 383b  2;255;0;0mi.[38;
000001f0: 323b 3835 3b38 353b 3835 6d74 081b 5b33  2;85;85;85mt..[3
00000200: 306d 1b28 421b 5b6d 1b5b 3338 3b32 3b32  0m.(B.[m.[38;2;2
00000210: 3535 3b30 3b30 6d74 1b5b 3330 6d1b 2842  55;0;0mt.[30m.(B
00000220: 1b5b 6d08 0808 081b 5b33 383b 323b 303b  .[m.....[38;2;0;
00000230: 3935 3b32 3135 6d65 7869 741b 5b33 306d  95;215mexit.[30m
00000240: 1b28 421b 5b6d 0a1b 5b33 306d 1b28 421b  .(B.[m..[30m.(B.
00000250: 5b6d 1b5b 3f32 3030 346c 071b 5b33 306d  [m.[?2004l..[30m
00000260: 1b28 421b 5b6d 0d1b 5b33 306d 1b28 421b  .(B.[m..[30m.(B.
00000270: 5b6d 1b5b 3f32 3030 346c                 [m.[?2004l

    000000e0: 6e72 6179 7a2f 6461 726b 7261 790d 0d1b  nrayz/darkray...
000000f0: 5b34 383b 323b 3238 3b32 383b 3238 6d1b  [48;2;28;28;28m.
00000100: 5b33 383b 323b 3235 353b 3235 353b 3235  [38;2;255;255;25
00000110: 356d 202f 642f 702f 672f 502f 6461 726b  5m /d/p/g/P/dark
00000120: 7261 7920 1b5b 3433 6d1b 5b33 383b 323b  ray .[43m.[38;2;
00000130: 3238 3b32 383b 3238 6dee 82b0 201b 5b34  28;28;28m... .[4
00000140: 336d 1b5b 3330 6dee 82a0 206d 6173 7465  3m.[30m... maste
00000150: 7220 e297 8f20 1b28 421b 5b6d 1b5b 3333  r ... .(B.[m.[33
00000160: 6dee 82b0 201b 2842 1b5b 6d1b 5b4b 6508  m... .(B.[m.[Ke.

  • xterm

    xterm

    Full hexdump 
     /d/p/g/P/darkray   master ●  fish | xxd
<W> fish: Current terminal parameters have rows and/or columns set to zero.
<W> fish: The stty command can be used to correct this (e.g., stty rows 80 columns 24).
00000000: 071b 5b33 306d 1b28 421b 5b6d 1b5d 303b  ..[30m.(B.[m.]0;
00000010: 6669 7368 202f 6461 7461 2f70 726f 6772  fish /data/progr
00000020: 616d 6d69 6e67 2f67 6974 6875 622f 5065  amming/github/Pe
00000030: 696c 6f6e 7261 797a 2f64 6172 6b72 6179  ilonrayz/darkray
00000040: 1b5b 326d e28f 8e1b 2842 1b5b 6d20 2020  .[2m....(B.[m   
00000050: 2020 2020 2020 2020 2020 2020 2020 2020                  
00000060: 2020 2020 2020 2020 2020 2020 2020 2020                  
00000070: 2020 2020 2020 2020 2020 2020 2020 2020                  
00000080: 2020 2020 2020 2020 2020 2020 2020 2020                  
00000090: 2020 2020 2020 2020 2020 2020 0de2 8f8e              ....
000000a0: 200d 1b5b 4b1b 5b3f 3230 3034 6807 1b5b   ..[K.[?2004h..[
000000b0: 3330 6d1b 2842 1b5b 6d1b 5d30 3b66 6973  30m.(B.[m.]0;fis
000000c0: 6820 2f64 6174 612f 7072 6f67 7261 6d6d  h /data/programm
000000d0: 696e 672f 6769 7468 7562 2f50 6569 6c6f  ing/github/Peilo
000000e0: 6e72 6179 7a2f 6461 726b 7261 790d 0d1b  nrayz/darkray...
000000f0: 5b34 383b 353b 3233 346d 1b5b 3338 3b35  [48;5;234m.[38;5
00000100: 3b32 3331 6d20 2f64 2f70 2f67 2f50 2f64  ;231m /d/p/g/P/d
00000110: 6172 6b72 6179 201b 5b34 336d 1b5b 3338  arkray .[43m.[38
00000120: 3b35 3b32 3334 6dee 82b0 201b 5b34 336d  ;5;234m... .[43m
00000130: 1b5b 3330 6dee 82a0 206d 6173 7465 7220  .[30m... master 
00000140: e297 8f20 1b28 421b 5b6d 1b5b 3333 6dee  ... .(B.[m.[33m.
00000150: 82b0 201b 2842 1b5b 6d1b 5b4b 6508 1b5b  .. .(B.[m.[Ke..[
00000160: 3338 3b35 3b31 3936 6d65 1b5b 3330 6d1b  38;5;196me.[30m.
00000170: 2842 1b5b 6d1b 5b33 383b 353b 3234 306d  (B.[m.[38;5;240m
00000180: 7869 7408 0808 1b5b 3330 6d1b 2842 1b5b  xit....[30m.(B.[
00000190: 6d1b 5b33 383b 353b 3139 366d 781b 5b33  m.[38;5;196mx.[3
000001a0: 383b 353b 3234 306d 6974 0808 1b5b 3330  8;5;240mit...[30
000001b0: 6d1b 2842 1b5b 6d1b 5b33 383b 353b 3139  m.(B.[m.[38;5;19
000001c0: 366d 691b 5b33 383b 353b 3234 306d 7408  6mi.[38;5;240mt.
000001d0: 1b5b 3330 6d1b 2842 1b5b 6d1b 5b33 383b  .[30m.(B.[m.[38;
000001e0: 353b 3139 366d 741b 5b33 306d 1b28 421b  5;196mt.[30m.(B.
000001f0: 5b6d 0808 0808 1b5b 3338 3b35 3b32 366d  [m.....[38;5;26m
00000200: 6578 6974 1b5b 3330 6d1b 2842 1b5b 6d0a  exit.[30m.(B.[m.
00000210: 1b5b 3330 6d1b 2842 1b5b 6d1b 5b3f 3230  .[30m.(B.[m.[?20
00000220: 3034 6c07 1b5b 3330 6d1b 2842 1b5b 6d0d  04l..[30m.(B.[m.
00000230: 1b5b 3330 6d1b 2842 1b5b 6d1b 5b3f 3230  .[30m.(B.[m.[?20
00000240: 3034 6c                                  04l

    000000e0: 6e72 6179 7a2f 6461 726b 7261 790d 0d1b  nrayz/darkray...
000000f0: 5b34 383b 353b 3233 346d 1b5b 3338 3b35  [48;5;234m.[38;5
00000100: 3b32 3331 6d20 2f64 2f70 2f67 2f50 2f64  ;231m /d/p/g/P/d
00000110: 6172 6b72 6179 201b 5b34 336d 1b5b 3338  arkray .[43m.[38
00000120: 3b35 3b32 3334 6dee 82b0 201b 5b34 336d  ;5;234m... .[43m
00000130: 1b5b 3330 6dee 82a0 206d 6173 7465 7220  .[30m... master 
00000140: e297 8f20 1b28 421b 5b6d 1b5b 3333 6dee  ... .(B.[m.[33m.
00000150: 82b0 201b 2842 1b5b 6d1b 5b4b 6508 1b5b  .. .(B.[m.[Ke..[

We can see some differences in the ANSI escape codes. However the core of the prompt are the same. I have padded the ? to make this easier to see.

vscode m??? .[43m.[30m??? master ... .(B.[m.[33m??? .(B.[m.[K
kitty  m... .[43m.[30m... master ... .(B.[m.[33m... .(B.[m.[K
xterm  m... .[43m.[30m... master ... .(B.[m.[33m... .(B.[m.[K

I have also made the hex easier to compare. This is the hex for the above and we can see VS Code is eating the ee82b0 and ee82a0. The output for this section between xterm and kitty is again identical.

vscode
3f     20 1b5b 3433 6d1b 5b33 306d 3f
  20 6d61 7374 6572 20e2 978f 201b 2842
1b5b 6d1b 5b33 336d 3f     20 1b28 421b
5b6d 1b5b 4b

kitty
ee82 b020 1b5b 3433 6d1b 5b33 306d ee82
a020 6d61 7374 6572 20e2 978f 201b 2842
1b5b 6d1b 5b33 336d ee82 b020 1b28 421b
5b6d 1b5b 4b

xterm
ee82 b020 1b5b 3433 6d1b 5b33 306d ee82
a020 6d61 7374 6572 20e2 978f 201b 2842
1b5b 6d1b 5b33 336d ee82 b020 1b28 421b
5b6d 1b5b 4b

Please can I have the Unicode in my prompt back.
@meganrogge meganrogge added bug Issue identified by VS Code Team member as probable bug terminal Integrated terminal issues labels Nov 12, 2020
@meganrogge meganrogge added this to the Backlog milestone Nov 12, 2020
@roblourens
Copy link
Member

Are you using VS Code in a remote workspace? (for example, using the Remote-SSH, Remote-Containers, or Remote-WSL extensions)

@roblourens
Copy link
Member

roblourens commented Nov 12, 2020 •
edited

Also, can you try running

echo '\xee\x82\xa0'

I don't know a ton about this but it seems like your hex dumps would show that fish is just not sending those characters in the vscode terminal environment, right?

And a third question - your post shows that you are on version 1.50, we have released 1.51, is that because you downgraded after seeing this issue, or have you not yet updated to 1.51?

@Peilonrayz
Copy link
Author

I'm not using any of those extensions. AFAIK this is native.

Yeah I can run that. Tbh I'm not quite sure what's happening.

 /d/p/g/P/darkray ? ? master ● ? echo '\xee\x82\xa0'
\xee\x82\xa0
 /d/p/g/P/darkray ? ? master ● ? printf '\xee\x82\xa0' | xxd
00000000: ee82 a0                                  ...
 /d/p/g/P/darkray ? ? master ● ? printf '\xee\x82\xa0'
^J

I'm on 1.50 because Arch hasn't updated to 1.51 yet.

@roblourens
Copy link
Member

Does it work if you switch to bash or sh?

@Peilonrayz
Copy link
Author

Sorry for the delay. I was installing shellder on zsh. And it displays the glyphs fine. (Note the glyphs aren't displaying correctly on GitHub.)

 /d/p/g/P/darkray ? ? master ● ? printf '\xee\x82\xa0\n'

 /d/p/g/P/darkray ? ? master ● ? zsh
 d/p/g/P/darkray   master ●  printf '\xee\x82\xa0\n'

 d/p/g/P/darkray   master ●  exit
 /d/p/g/P/darkray ? ? master ● ?

console

Sorry in light of this it may not be a VS Code issue. Since fish didn't update it seems like a bug's came out of no-where.

@Peilonrayz
Copy link
Author

Peilonrayz commented Nov 12, 2020 •
edited

Found the problem. If I use set LC_ALL the glyphs, mostly, come back. Because of #110322 it looks like this is an electron issue.

locale

@github-actions github-actions bot locked and limited conversation to collaborators Dec 27, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@meganrogge meganrogge

Labels
bug Issue identified by VS Code Team member as probable bug terminal Integrated terminal issues
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
3 participants
@roblourens @Peilonrayz @meganrogge