Do an automatic stream flush before rendering a progress bar

[nickion]

4.60.0 3.7.6 (default, Jan 8 2020, 20:23:39) [MSC v.1916 64 bit (AMD64)] win32
Running Python in Spyder

While tqdm is great for the negligible effort required to have a progress bar, I've found it necessary to add a sys.stdout.flush() call before usage to avoid possible interleaved output. This taints the code and reduces the ease of use. e.g. below, where although the 1 was printed before the bar was started, the output was interleaved.

1%|          | 14/2408 [00:00<00:17, 135.41it/s]1
11%|█         | 263/2408 [00:02<00:17, 123.80it/s]

An option to have tqdm automatically flush the output stream it's about to use would be a huge benefit.

[casperdcl]

interesting... is this consistently reproducible?

[nickion]

Yes, every time. A flush always resolves.

I found a similar observation about buffering from a few years back. spyder-ide/spyder#2860
Their conclusion was that it's related to Qt and so they closed as not a Spyder bug. So it would be a tqdm enhancement that perhaps shouldn't be necessary but would be very useful.

I notice there's callback support in tqdm, but wasn't sure if tqdm itself has callbacks for the start, end and during processing. Given a start callback then a workaround might be to attach a function to flush, but a native feature would be ideal.

[casperdcl]

does the current devel fix it for you? (pip install "git+https://github.com/tqdm/tqdm@devel#egg=tqdm")

[nickion]

Sorry for the delayed feedback, and unfortunately not, a sys.stdout.flush() is still required before a call to tqdm.tqdm()

[casperdcl]

You mean sys.stderr.flush() is not enough? Even though tqdm writes to stderr you need to flush stdout first?

[nickion]

You mean sys.stderr.flush() is not enough? Even though tqdm writes to stderr you need to flush stdout first?

I didn't realise that tqdm is using stderr, but would explain the issue as there are then two independently buffered streams both writing to the same destination. stderr is traditionally used for errors and if there's an option to use stdout instead that could be an alternative to flushing, but to be safe, I'd probably opt simply for tqdm to flush both stderr and stdout when starting a new progress tracker.

[casperdcl]

can you try devel again now?

[nickion]

Thanks, the current devel version now resolves this.