Pipelining text with ANSI color code to bat consumes huge CPU and memory.
#1481
Comments
cedric-sun
changed the title
bat consume huge CPU and memory.bat consumes huge CPU and memory.
|
I might have an idea on what's going on. The way that ANSI escape sequences are passed through by It's possible that bat isn't clearing the history of emitted sequences properly, and is just continously emitting obsolete sequences over and over again. It would look the same, but end up with tons of unnecessary bytes being sent to The theory would need to be tested of course, but it's my best guess as of right now. |
|
Related to #1147? |
|
I did some of my own testing with this on bat's repo using Before I get into details, I would just like to say that the following three lines made this very difficult to figure out 😛 Lines 164 to 166 in 7ada963 Time to dig into why there's such a drastic difference... $ I'm not entirely sure why it's necessary for it to be emitting $ The diff between these two shows that So far looks like my previous theory was wrong, but there still one other thing to consider... what if we remove the plain style, and bring the decorations back? $ There's a giant, ever-increasing string of redundant color codes... Turns out my theory was right. When we use decorations and wrapping, |
|
Great analysis!
Ouch. Too much magic 😄 |
I think this is exactly the same. Try or see the slow output directly: It also creates ever-increasing lists of ANSI sequences. |
|
Also see my |
|
Hmm, this is more of a widespread issue than I thought. I think it's time for me to pay off the accrued technical debt for my wrapping implementation from way back when. Here's my plan: At the moment, I'm working on writing a (hopefully dependency-free) crate for properly handling the ANSI escape sequences. The idea is that we'll feed the extracted sequences into a struct, which will interpret them and build an expected state for the foreground and background color (and other attributes). This state can be used to build ANSI sequences that we can emit, rather than storing a super long string of redundant escape sequences. I'll replace the current implementation of ANSI color passthrough with that. After that (or perhaps simultaneously), we could find a way to reduce the number of What do you think? |
Except for the reservations in #1499 (comment), that sounds great. |
|
Well I think what you can do is have a configurable for switch in line of ms per update for bat to output the changes that it builds in its "virtual screen" something like virtual DOM of React. |
|
|
@eth-p oh I didn't realize you pipe it to |
It doesn't even have to be a future update, actually! You will have to rely on your terminal's scrollback buffer if you do it, but you can configure bat to permanently disable the pager. Either put |
|
Closed by #1596 |
What version of
batare you using?bat 0.17.1
Describe the bug you encountered:
I was working on the nginx source tree: http://nginx.org/download/nginx-1.19.6.tar.gz
lesswill open but scrolling to end (G) takes 8 seconds for a 2500 linesgrepresult.topreport thatlessconsumed a whole CPU core and 960 MiB physical memory (RES) during that 8 seconds.grep -rniI 'u_char' --color=never | batworks fine.grep -rniI 'u_char' --color=always | less -RNworks fine.What did you expect to happen instead?
Performance should be equivalent to
grep -rniI 'u_char' --color=always | less -RN.How did you install
bat?Archlinux pacman.
Evironment
system
$ uname -srm
Linux 5.9.14-arch1-1 x86_64
bat
$ bat --version
bat 0.17.1
$ env
bat_config
bat_wrapper
No wrapper script for 'bat'.
bat_wrapper_function
No wrapper function for 'bat'.
No wrapper function for 'cat'.
tool
$ less --version
less 563 (PCRE regular expressions)
The text was updated successfully, but these errors were encountered: