ANSI escape codes break tqdm.tqdm.write formatting

[volans-]

• I have visited the source website, and in particular
  read the known issues
• I have searched through the issue tracker for duplicates
• I have mentioned version numbers, operating system and
  environment, where applicable: can repro since tqdm release 4.25.0 ( ignore ANSI escape codes in the bar length calculation #592 ) up to the latest release (4.32.2 as of the opening of this issue).

I'm aware that there are various efforts to improve existing issues related to write() ( #737 ) and to multiple progress bars, but I'd like to report that since tqdm release 4.25.0 and in particular change #592 a previously working code is now visually broken when colors are used.

Pre-4.25.0 behaviour:

$ cumin --force -c /tmp/cumin-dEULJH/config.yaml 'cumin-dEULJH-1' 'ls -la /tmp'
1 hosts will be targeted:
cumin-dEULJH-1
FORCE mode enabled, continuing without confirmation
----- OUTPUT of 'ls -la /tmp' -----
total 8
drwxrwxrwt    2 root     root          4096 Jul  5  2018 .
drwxr-xr-x    1 root     root          4096 Jul 18 16:29 ..
================
PASS |██████████████████████████████████████████████████████████████████████████████████████████████████████| 100% (1/1) [00:00<00:00,  5.40hosts/s]
FAIL |                                                                                                              |   0% (0/1) [00:00<?, ?hosts/s]
100.0% (1/1) success ratio (>= 100.0% threshold) for command: 'ls -la /tmp'.
100.0% (1/1) success ratio (>= 100.0% threshold) of nodes successfully executed all commands.

Since 4.25.0 (scroll all to the right):

$ cumin --force -c /tmp/cumin-dEULJH/config.yaml 'cumin-dEULJH-1' 'ls -la /tmp'
1 hosts will be targeted:
cumin-dEULJH-1
FORCE mode enabled, continuing without confirmation
                                                                                                                                                              |                                                                                                                   |   0% (0/1) [00:00<?, ?hosts/s]
----- OUTPUT of 'ls -la /tmp' -----                                                                                                                                                                                                                                                                                    |                                                                                                                   |   0% (0/1) [00:00<?, ?hosts/s]
total 8                                                                                                                                                                                                                                                                                                                |                                                                                                                   |   0% (0/1) [00:00<?, ?hosts/s]
drwxrwxrwt    2 root     root          4096 Jul  5  2018 .                                                                                                                                                                                                                                                             |                                                                                                                   |   0% (0/1) [00:00<?, ?hosts/s]
drwxr-xr-x    1 root     root          4096 Jul 18 16:29 ..                                                                                                                                                                                                                                                            |                                                                                                                   |   0% (0/1) [00:00<?, ?hosts/s]
================                                                                                                                                         PASS |███████████████████████████████████████████████████████████████████████████████████████████████████████████| 100% (1/1) [00:00<00:00,  4.64hosts/s]
FAIL |                                                                                                                   |   0% (0/1) [00:00<?, ?hosts/s]
100.0% (1/1) success ratio (>= 100.0% threshold) for command: 'ls -la /tmp'.
100.0% (1/1) success ratio (>= 100.0% threshold) of nodes successfully executed all commands.

To see the colors and also the effect on a normally-sized shell see the attached screenshot:

As soon as I either remove any color from bar_format or revert

N_BARS = max(1, ncols - len(RE_ANSI.sub('', l_bar + r_bar)))

to

N_BARS = max(1, ncols - len(l_bar + r_bar)

everything works back again as before, with the small issue of the progress bars being shorter. CC @jnhansen as was involved in the related change.

If instead of removing the colors I remove the second progress bar this happens (again scroll all to the right):

$ cumin --force -c /tmp/cumin-dEULJH/config.yaml 'cumin-dEULJH-1' 'ls -la /tmp'
1 hosts will be targeted:
cumin-dEULJH-1
FORCE mode enabled, continuing without confirmation
                                                                                                                                                     ----- OUTPUT of 'ls -la /tmp' -----
                                                                                                                                                     total 8
                                                                                                                                                     drwxrwxrwt    2 root     root          4096 Jul  5  2018 .
                                                                                                                                                     drwxr-xr-x    1 root     root          4096 Jul 18 16:29 ..
                                                                                                                                                     ================
PASS |███████████████████████████████████████████████████████████████████████████████████████████████████████| 100% (1/1) [00:00<00:00,  4.47hosts/s]
100.0% (1/1) success ratio (>= 100.0% threshold) for command: 'ls -la /tmp'.
100.0% (1/1) success ratio (>= 100.0% threshold) of nodes successfully executed all commands.

Please advice if there is any workaround that I can use to have those two progress bars working with recent versions of tqdm.

[nanonano]

I'm experiencing the same issue in v4.45.0 when using a single progress bar.

Extra line feeds get inserted for writes if color codes are used in the progress bar, using the workaround above, specifying the older version fixes it.

from tqdm import tqdm
  
for i in tqdm(range(10), desc='\033[91m' + "With color"):
    tqdm.write("something")

for i in tqdm(range(10), desc="Without color"):
    tqdm.write("something")

output looks like this.

something

something

something

something

something

something

something

something

something

something
With color: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:00<00:00, 1412.22it/s]
something
something
something
something
something
something
something
something
something
something
Without color: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:00<00:00, 966.74it/s]

[casperdcl]

duplicate of #692