ANSI support on Windows

[richardpaulhudson]

Older versions of Windows did not support ANSI control codes out-of-the-box, but all currently supported versions of Windows 10 and 11 now do. Our stack at Explosion currently checks the OS and offers a simpler, ANSI-free console if it is Windows. I am trying to change this so that the richer console is offered consistently across all OS.

We use tqdm, and I have run into the problem that when tqdm is imported on a Windows platform it imports colorama (https://github.com/tqdm/tqdm/blob/master/tqdm/utils.py#L38). In order to handle the lack of ANSI that was prevalent within older versions of Windows, colorama then wraps the stdout and stderr streams with versions that filter out some ANSI codes and alter others.

Because the replacement of the streams has a global impact on the Python environment, this means that any application that imports tqdm is unable to make full use of ANSI on Windows even though Windows itself actually supports it. There is a workaround calling colorama.deinit(), but we don't want to include this in our stack because it seems likely to confuse our users and because there is no way of knowing in which order people will import things.

All currently supported versions of Windows 10 and 11 offer full native support for ANSI just as POSIX does. As far as I could make out, the only remaining Windows version that doesn't offer ANSI support by default is Windows 8.1, for which extended support will be discontinued in January 2023 and where users are already receiving messages advising them to switch to Windows 11 every time they log on.

Would it therefore be possible to remove the colorama import from tqdm and to use ANSI on Windows in the same way that it is already used on other OS? As colorama does not appear to be used for anything else within tqdm, this would have the additional advantage of removing a dependency.

[njsmith]

Note that even on modern Windows versions, you can't just assume your terminal supports ANSI – you have to explicitly "turn it on" using SetConsoleMode.

I think the ideal resolution would be for colorama to gain support for handling this itself, and then tqdm won't have to worry about it: tartley/colorama#139
But otherwise, if tqdm wants to drop colorama or colorama doesn't pick up that change, then it's not enough to just remove colorama; you'd need to also add some ctypes wrappers.

[richardpaulhudson]

Note that even on modern Windows versions, you can't just assume your terminal supports ANSI – you have to explicitly "turn it on" using SetConsoleMode.

It's hard to find primary documentation (i.e. from Microsoft) but multiple secondary sources (e.g. https://ss64.com/nt/syntax-ansi.html) state that ANSI has been enabled by default since Windows 10 version 1909. All versions older than that except Windows 8.1 are no longer supported. ANSI working out of the box is also the behaviour I have observed on an up-to-date Windows 11 machine.

If anyone has any information or examples to the contrary (or if continuing to support outdated Windows versions is a requirement), I believe @njsmith has a code snippet that performs the "turning on" — perhaps you could share it here?

[njsmith]

On my Win10 machine, ANSI doesn't seem to be enabled by default. It's a bit hard to tell, because cmd.exe does enable ANSI by default, so if you just open up a terminal window and then run python, it'll look like it's always enabled. But if you open a powershell terminal, or run a python console app directly without going through any shell, then it seems to be disabled by default.

The magic dance to turn on ANSI support looks something like this: https://github.com/explosion/wasabi/pull/24/files