max_line_len has very bad performance on big bundles

[albertogasparin]

Bug report or Feature request? Feature request
(even if the performance regression is > 25% more time so quite big)

Version master

minify() options used max_line_len: 120

terser input Very large js files, for instance > 5MB

terser output or error Build time increase by 25% or more. On our CI webpack builds went from 20min to 25min by changing max_line_len from 32000 to 120

Expected result
Performance impact should be negligible.
Wonder if there is some optimisation we can add to https://github.com/terser/terser/blob/master/lib/output.js#L363

[fabiosantoscode]

Out of curiosity, what is the reason for using this option?

I've always considered it only a "formatting" option and I want to remove it with others like it.

[fabiosantoscode]

There are some tokens that don't accept newlines after them, such as return (it would return undefined and the next line would be dead code). So while it's not possible to make it super fast, it might be possible to do some incremental improvement.

[albertogasparin]

The main reason is that both Chrome and Firefox devtools hang (and often even crash) when you try to debug such large bundles. Especially when investigating performance or bugs in production, we have to load the minified version first before even loading source maps or formatting them with the builtin tools.
Seems like the renderer/tokeniser on every browser struggles with really long lines (like dozens of thousands of chars long).

Just to give you an example, having a minified 5MB compressed file with max_line_len at 120 loads up in a couple of seconds, setting it to 1000 takes a 5 seconds or so, with unrestricted length sometimes it hangs (especially if the debugger breakpoint is in the second half of the line length) while other times succeeds in 15-20s.

Obviously, this is not necessarily a terser issue, as would be nicer if devtools would work regardless of line length, but it's the easiest option to solve a big dev pain. So we don't use it as "formatting" option (agree with you that terser should not care about that stuff) but to solve an actual browser limitation.
Kinda saying that it would be nice if terser would preserve "some" line breaks.

[fabiosantoscode]

Thanks for letting me know! That makes a lot of sense. Files with a very large first line are the only thing that lags vim for me.

If there isn't a proper max_line_length that's fast and precise, at least there should be a faster option that does the same, with less precision. Some lines might be larger, or smaller, but it would use a more basic algorithm.

However I don't think this has a high priority especially when compared with the obscene amount of RAM terser uses with large payloads.

[gdh1995]

Hello, I also expect better max_line_len, and the below is my intent:

• I maintain a web-extension named "Vimium C" for Firefox and Chromium (https://github.com/gdh1995/vimium-c)
• there's a report it crashes on Firefox "only once" after a system reboots,
  according to Since Latest firefox it doesn't activate automatically gdh1995/vimium-c#442 (comment)
• the error place is at line 1 and column 1000, and the message is Uncaught Exception { name: "NS_ERROR_FAILURE", message: "", result: 2147500037}
• but, the uglified code is correct when I review it: the second letter l in l.Le=a?(...)?b:c:0|(...),l.Ae=...

So I have a plan to use max_line_len to force 120 characters per line, and let my user take a try in days.

[jridgewell]

Fixed by #1109.

[mieszko4]

The main reason is that both Chrome and Firefox devtools hang (and often even crash) when you try to debug such large bundles. Especially when investigating performance or bugs in production, we have to load the minified version first before even loading source maps or formatting them with the builtin tools. Seems like the renderer/tokeniser on every browser struggles with really long lines (like dozens of thousands of chars long).

@albertogasparin Thnx! Your comment helped me with resolving issue we had on lambda node 14.x.
We used WebPack with default settings that we used for AWS lambda which resulted as one liner with almost 3MB minified code.
When error was thrown in lambda, lambda would just freeze and timeout even though we had try/catch in the handler.
After adding few new lines in the minified code just to split this one-liner, the error was properly caught by try/catch.
I could also reproduce this locally on node v16.18.1.
My guess the issue is related to node having trouble with constructing stack trace.