Tool for markdown-it to escape the necessary characters in markdown so I can try autorender

[universemaster]

Hi,

I usually use server-side rendering with vscode-markdown-katex.

I want to try in-browser rendering with autorender, but I still want to use my markdown processor for it's other work.

Do you know of a tool/plug-in I can use to do the escaping required? I use markdown-it.

I could of course do write the escaping code myself, but I suspect there are lots of edge-cases that I will be chasing for weeks.

I'm asking for my own reasons, but also think it would be good to detail this on the appropriate pages of www.katex.org.

[edemaine]

May I ask why you want to use autorender?

The general recommendation is to use a markdown plugin that interprets all the math-entering symbols and calls KaTeX while rendering the Markdown, instead of escaping them and then calling autorender. For markdown-it, I've used markdown-it-texmath with good results.

I agree that this should probably be added to https://katex.org/docs/libs.html

[universemaster]

Your questioning makes me think maybe it's not the way to go for better performance.

My reason for the interest is that when I enable content-visibility: auto (see other discussion #3357 ) for browsers that support it, my limiting performance factor is Parsing HTML and CSS. So I was trying to send the raw KaTeX source for my equations, rather than the pre-rendered DOM tree.

---

Let me use this discussion thread to give a little more background regarding where I am with optimising performance.

Notice (on the screenshot below) that on desktop (and sadly desktop only), I can get a decent lighthouse score despite the massive (understatement!) DOM.

Notice also, that the most main-thread work (relating to total blocking time) is parsing HTML and CSS.

Disappointingly, mobile browsers and those that don't support content-visibility give a very different story (also less good when I switch to a remote server - or use a cloudflare tunnel which frustratingly doesn't support brotli from origin to cloudflare! My gzipped html is double the size of my brotli html).

These scores are with all the optimisations in the other threads and are for {output: html}, desktop chrome browser, local server (with brotli max level 11) and content-visibility: auto enabled.

For the stats on my other threads, I commented out content-visibility from my stylesheet - since it doesn't work well on mobile, or safari on any device.

[universemaster]

And mobile with otherwise the same settings (including content-visibility: auto - so it will stutter when I scroll).

[edemaine]

OK, makes sense now; you want to render only when the equation becomes visible.

The timing numbers you give lack some context: could you summarize how big the DOM is, how many equations, average DOM size per equation, etc.? This would help put the parse times into perspective.

I think the way to do this would be to use a plugin like markdown-it-texmath, but give it a fake KaTeX engine like so (untested code):

const markdown = require('markdown-it')({html: true})
.use(require('markdown-it-texmath'), {
  engine: {
    renderToString: (tex, options) => `<div class="math-todo" data-display="${options.displayMode}">${tex}</div>`
  }
})

Then you could write some JavaScript to build an IntersectionObserver to detect when these become visible and call KaTeX to render them, and possibly reset them when they're no longer visible. This on-demand KaTeX rendering code would probably be an interesting package to write, either stand-alone or for contrib here.

[universemaster]

But if I comment out all content-visibility:auto and contain: from my stylesheet and keep everything else the same, all the katex rendering needs to happen at page load (hence the issues I raised) and we get an almost laughable difference - here is desktop, which lighthouse timed-out after it took too long (sorry for the wait for this next post) - notice it didn't load the entire DOM! So it's actually even worse than the 45 seconds of main-thread!

(I shall not bother giving you the mobile version!)

[universemaster]

Sorry, I should have highlighted that the true DOM size is 562,872 elements - the first item under diagnostics (and the test without content-visibility didn't even load all of that).

There are 16,000 "equations", which I put in quotes become some are just $x$. "Almost all" of those elements are from KaTeX.

[mbourne]

@universemaster Can you please provide a link to your 16,000-"equation" page so I can run some experiments? Thanks.

[universemaster]

I've added an un-minimised version of the main stylesheet preview.css with comments.

[edemaine]

I'd be curious to see it too.

[universemaster]

I was waiting for that - added!

[kelvindi]

@universemaster Can you please add me too? I also have same question and want to explore

[universemaster]

Sure, added.

[universemaster]

Relating to this: If you scramble the non-KaTeX text (and maybe the "text" bits inside the KaTeX too), you could this html file for testing in the official KaTeX repository.

By scramble I mean replace each character with a random other character.

Relates to #2338

[universemaster]

Oh, you'd need to scramble the original markdown - this is getting complex!