Build fails when using some emojis and privacy plugin

[jonaharagon]

Context

No response

Bug description

The site is failing to build when using some external emojis (like 📧 and country flags) and the privacy plugin. I just tested this more and some emojis seem to work, like :gear: and based on limited testing I think it is just ones with special characters in the name?

$ mkdocs serve
INFO     -  Building documentation...
INFO     -  Cleaning site directory
ERROR    -  Error building page 'index.md': Document is empty
Traceback (most recent call last):
  File "/opt/homebrew/bin/mkdocs", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/mkdocs/__main__.py", line 234, in serve_command
    serve.serve(dev_addr=dev_addr, livereload=livereload, watch=watch, **kwargs)
  File "/opt/homebrew/lib/python3.11/site-packages/mkdocs/commands/serve.py", line 83, in serve
    builder(config)
  File "/opt/homebrew/lib/python3.11/site-packages/mkdocs/commands/serve.py", line 76, in builder
    build(config, live_server=live_server, dirty=dirty)
  File "/opt/homebrew/lib/python3.11/site-packages/mkdocs/commands/build.py", line 329, in build
    _build_page(file.page, config, doc_files, nav, env, dirty)
  File "/opt/homebrew/lib/python3.11/site-packages/mkdocs/commands/build.py", line 234, in _build_page
    output = config.plugins.run_event('post_page', output, page=page, config=config)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/mkdocs/plugins.py", line 520, in run_event
    result = method(item, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/material/plugins/privacy/plugin.py", line 198, in on_post_page
    return self._parse_html(output, page.file, config)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/material/plugins/privacy/plugin.py", line 352, in _parse_html
    return re.sub(
           ^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/re/__init__.py", line 185, in sub
    return _compile(pattern, flags).sub(repl, string, count)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/material/plugins/privacy/plugin.py", line 305, in replace
    el: HtmlElement = fragment_fromstring(match.group())
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/lxml/html/__init__.py", line 829, in fragment_fromstring
    elements = fragments_fromstring(
               ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/lxml/html/__init__.py", line 792, in fragments_fromstring
    doc = document_fromstring(html, parser=parser, base_url=base_url, **kw)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/lxml/html/__init__.py", line 761, in document_fromstring
    raise etree.ParserError(
lxml.etree.ParserError: Document is empty

Related links

• Privacy plugin does not download Twemoji SVGs #3851 (comment)
• Emojis are not working w/ privacy plugin privacyguides/privacyguides.org#2063

Reproduction

Archive.zip

Steps to reproduce

1. Enable the Twemoji emoji index (I have not tested this with any other emoji sets)
2. Enable the privacy plugin
3. Add :flag_us: to Markdown document

Browser

No response

Before submitting

• I have read and followed the bug reporting guidelines.
• I have attached links to the documentation, and possibly related issues and discussions.
• I assure that I have removed all customizations before submitting this bug report.
• I have attached a .zip file with a minimal reproduction.

[jonaharagon]

Does it also happen if you use 4.32.0?

@squidfunk if it helps, no it does not happen on 4.32.0.

[squidfunk]

Yes that helps. Please try master again. Potential fix in 01bd4b8. Using unicode_escapes seems to do the trick for emojis as well as diacritics. In case you find further issues, please attach them here as reproductions.

[jonaharagon]

Thanks! This fix seems to work 👍

I'm still having the other issue I mentioned at privacyguides/privacyguides.org#2063 (I think that is specifically with emojis in admonitions)... but that seems to be a different problem, if I can figure out how to reproduce it in a minimal install I can open a separate bug report (because I'm not sure if it's really an issue here at all).

[squidfunk]

Released as part of 9.1.0+insiders-4.32.2.