Does include parse included file as markdown?

[abitrolly]

Is there any processing that occurs when include directive inserts external file in markdown? Like trying to parse the file as markdown, expand links etc. I could not find it in docs.

{%
   include "diagram.html"
%}

https://github.com/mondeja/mkdocs-include-markdown-plugin

[mondeja]

Nope, the include directive just includes files and not any further processing is done.

[abitrolly]

So, if I include HTML file (as in pypa/cibuildwheel#1697) mkdocs will still try to parse the included content as markdown? Even if plugin itself doesn't do any processing. Is that how it works?

[mondeja]

Check these lines of Mkdocs:

https://github.com/mkdocs/mkdocs/blob/b3f278ae73d7ef1b04630cb3851807fbc440671c/mkdocs/commands/build.py#L164-L168C14

This plugin runs on on_page_markdown step. The content is returned as Markdown in the page.markdown variable. After that, Mkdocs uses Python-Markdown to process it and convert to HTML at page.render():

https://github.com/mkdocs/mkdocs/blob/b3f278ae73d7ef1b04630cb3851807fbc440671c/mkdocs/structure/pages.py#L262-L279

Python-Markdown does not transforms HTML. Quoting the documentation of this preprocessor:

https://github.com/Python-Markdown/markdown/blob/a2a9c53ea262ea0d39e76aa18430ce071fbe68d5/markdown/preprocessors.py#L78-L91

Remove html blocks from the text and store them for later retrieval.

And this postprocessor seems like is not doing strange things, just re-adding the HTML content using the placeholder \u0002wzxhzdk:%s\u0003:

https://github.com/Python-Markdown/markdown/blob/a2a9c53ea262ea0d39e76aa18430ce071fbe68d5/markdown/postprocessors.py#L74-L82

So not, if you include HTML the content will be added to the final HTML as is. Note that Markdown processors shouldn't transform HTML as Markdown is just a subset of HTML.

[abitrolly]

Thanks. That's a very good analysis. I probably expected the include to work on page_content event:

https://www.mkdocs.org/dev-guide/plugins/#on_page_content

The page_content event is called after the Markdown text is rendered to HTML (but before being passed to a template) and can be used to alter the HTML body of the page.

But at this point the markdown marker to know where to insert the content is most likely lost.

---

From the HTML parser I don't see if it allows markdown inside HTML tags. GitHub allows it, but it is inconsistent.

<blockquote> **GitHub** ~~definitely~~ allows it.</blockquote>

When this markup is pasted inline,

GitHub definitely allows it.

, but if specified on a separate like, it doesn't expand:

**GitHub** ~~definitely~~ allows it.

So I'd like to use include mechanism that avoids Markdown expansion in HTML, and HTMLParser code is too complicated to understand if it cuts everything from start tag to end tag, or if it just strips tags and still leaves the content in between for processing.

[mondeja]

Maybe you can just use a custom template for the file. But to be clear, the change you're trying to do in cibuildwheel is just a speedup improvement, is not changing anything relative to the processing.

Have you experienced the removal by Mkdocs of some security-related tags like <script>s?

[abitrolly]

I wanted to understand what change I was trying to do. :D And now everything is clear. So it was just an attempt to simplify things.

I don't have much experience with Mkdocs, so I didn't experience any bugs with stripped scripts (yet). But if I'd met some, the first thing I would probably do for debugging, is to replace include directive in Markdown with SOMEGIBBERISHCONTENTTAGLIKETHIS then use a tool like https://github.com/facebookincubator/fastmod to change it to HTML that needs to be pasted. Then compare the result.