Skip to content
This repository has been archived by the owner on Mar 5, 2022. It is now read-only.

Smarter colorization and better support for native terminals on Windows #277

Merged
merged 2 commits into from Apr 10, 2019
Merged

Smarter colorization and better support for native terminals on Windows #277

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c9eb8c8
Introduce smarter coloring and --colorize option
zmwangx Apr 10, 2019
d2b579f
Enable ANSI color in cmd and PowerShell on Windows 10
zmwangx Apr 10, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
50 changes: 46 additions & 4 deletions googler
View file
Open in desktop
Expand Up @@ -30,6 +30,7 @@ from http.client import HTTPSConnection
import locale
import logging
import os
import platform
import shutil
import signal
import socket
Expand Down Expand Up @@ -3126,8 +3127,13 @@ def parse_args(args=None, namespace=None):
addarg('-l', '--lang', metavar='LANG', help='display in language LANG')
addarg('-x', '--exact', action='store_true',
help='disable automatic spelling correction')
addarg('-C', '--nocolor', dest='colorize', action='store_false',
help='disable color output')
addarg('--colorize', nargs='?', choices=['auto', 'always', 'never'],
const='always', default='auto',
help="""whether to colorize output; defaults to 'auto', which enables
color when stdout is a tty device; using --colorize without an argument
is equivalent to --colorize=always""")
addarg('-C', '--nocolor', action='store_true',
help='equivalent to --colorize=never')
addarg('--colors', dest='colorstr', type=argparser.is_colorstr,
default=colorstr_env if colorstr_env else 'GKlgxy', metavar='COLORS',
help='set output colors (see man page for details)')
Expand Down Expand Up @@ -3165,7 +3171,11 @@ def parse_args(args=None, namespace=None):
addarg('-D', '--debugger', action='store_true', help=argparse.SUPPRESS)
addarg('--complete', help=argparse.SUPPRESS)

return argparser.parse_args(args, namespace)
parsed = argparser.parse_args(args, namespace)
if parsed.nocolor:
parsed.colorize = 'never'

return parsed


def main():
Expand Down Expand Up @@ -3203,14 +3213,46 @@ def main():
pass

# Set colors
if opts.colorize:
if opts.colorize == 'always':
colorize = True
elif opts.colorize == 'auto':
colorize = sys.stdout.isatty()
elif opts.colorize == 'never':
colorize = False
else:
raise ValueError("invalid --colorize value '%s'" % opts.colorize)

if colorize:
colors = Colors(*[COLORMAP[c] for c in opts.colorstr], reset=COLORMAP['x'])
else:
colors = None
Result.colors = colors
Result.urlexpand = True if os.getenv('DISABLE_URL_EXPANSION') is None else False
GooglerCmd.colors = colors

# Try to enable ANSI color support in cmd or PowerShell on Windows 10
if sys.platform == 'win32' and sys.stdout.isatty() and colorize:
# VT100 control sequences are supported on Windows 10 Anniversary Update and later.
# https://docs.microsoft.com/en-us/windows/console/console-virtual-terminal-sequences
# https://docs.microsoft.com/en-us/windows/console/setconsolemode
if platform.release() == '10':
STD_OUTPUT_HANDLE = -11
ENABLE_VIRTUAL_TERMINAL_PROCESSING = 0x0004
try:
from ctypes import windll, wintypes, byref
kernel32 = windll.kernel32
stdout_handle = kernel32.GetStdHandle(STD_OUTPUT_HANDLE)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great to add STD_ERROR_HANDLE = -12 and repeat this process. LGTM.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about this and it turns out setting the mode flag for stdout makes stderr VT100-aware too. I know little about Windows, I guess it's because the two streams are sharing the same tty device (lacking Windows-specific terminology)? Like on *nix, tcsetattr(3) and friends from termios apply to the tty device and are not specific to any stream.

I can't think of case where someone would split stdout and stderr on two ttys, so we're probably good here.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The messages sent to stderr through printerr will contain bad sequences, just like the original problem was with stdout. I guess it should not make a difference on non-Windows, but it sure makes it usable on Windows.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zmwangx and @guilt, do you guys understand that all these low-level hacks we are pulling out of our sleeves is because of the fact that we are trying to support a terminal that doesn't understand escape sequences in 2019?

We had an option to disable colors completely and there are terminal emulators on Windows which understand color codes.

Here's my take - we have spent quality time on this one. We are still discussing what we need to do to make it work on a terminal the source of which is not available to us.

Can we revert this patch and take any of the following paths:

  • document clearly that people using native terminals on Windows should be using the -C option?
  • if that looks inadequate, if we detect the platform is Windows we show a constant message at the beginning to tell the user that he may have to use the option -C if he is seeing ANSI escape codes
  • set -C to true by default on Windows as the probability is high the users may be using those terminals which are provided to them. And we document that to have colors on Windows one has to set C to false.

Any of these would be much more elegant than stuff we are wasting our time on now.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@guilt

The messages sent to stderr through printerr will contain bad sequences, just like the original problem was with stdout.

Yeah, I’m saying after setting ENABLE_VIRTUAL_TERMINAL_PROCESSING on stdout, stderr understands ANSI color too, at least on my machine. The prompt use reverse video which works alright in both cmd and PowerShell. Do you have a problem there? (I’m just using tcsetattr, which I understand well, as a model to guess why it works. Both fd 1 and fd 2 eventually points to /dev/pts/something which tsetattr works upon, and I assume it works similarly on Windows.)

Can we revert this patch and take any of the following paths...

I made sure this patch is strictly improvements without breaking anything that was working previously, so I don’t see why it needs be reverted. No color by default or always on warning are too heavy-handed and I myself as a ConEmu/Cmder user (when I’m using Windows, that is) won’t appreciate — color was working fine the first time I ported googler to Windows, that was what, two to three years ago? Adding a note to README is fine, it’s already full of things like this.

Also, consider this: Windows 7 EOL is near, Windows 8 is universally hated and admitting to using it in public is a no no ;), Windows 10 auto updates so probably no one is using a pre-1607 version, so this patch really covers almost the entire supported base of Windows users. The rest are like Python 3.3 users and who cares.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Anyway, on second thought I’ll set the flag for stderr too since it doesn’t really hurt... At most we set 7 | 4 = 7 which is a noop.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I see what you mean.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually 7= ENABLE_PROCESSED_OUTPUT | ENABLE_WRAP_AT_EOL_OUTPUT | ENABLE_VIRTUAL_TERMINAL_PROCESSING

The reason I suggested stderr is because of lines like:

googler/googler

Line 2103 in 1d3f70d

printerr('\x1b[1mInspect the DOM through the \x1b[4mtree\x1b[24m variable.\x1b[0m')

I'm glad you folks are taking this forward. Really appreciate it :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ENABLE_PROCESSED_OUTPUT and ENABLE_WRAP_AT_EOL_OUTPUT are the default, so when you get stdout console mode you get 3, you set it to 3 | 4 = 7; then when you get stderr console mode, which is the same console, you get 7, so setting it to 7 | 4 = 7 is a noop. Again, it doesn't hurt.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#280.

old_mode = wintypes.DWORD()
if not kernel32.GetConsoleMode(stdout_handle, byref(old_mode)):
raise RuntimeError('GetConsoleMode failed')
new_mode = old_mode.value | ENABLE_VIRTUAL_TERMINAL_PROCESSING
if not kernel32.SetConsoleMode(stdout_handle, new_mode):
raise RuntimeError('SetConsoleMode failed')
# Note: No need to restore at exit. SetConsoleMode seems to
# be limited to the calling process.
except Exception:
pass

if opts.url_handler is not None:
open_url.url_handler = opts.url_handler
else:
Expand Down