0.4.0: pytest is failing

[kloczek]

Describe the bug

Looks like with new version pytest is failing in few units.

Reproduce the bug

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

• python3 -sBm build -w --no-isolation
• because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
• install .whl file in </install/prefix> using 'installer` module
• run pytest with $PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>
• build is performed in env which is cut off from access to the public network (pytest is executed with -m "not network")

Here is pytest output:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-mdit-py-plugins-0.4.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-mdit-py-plugins-0.4.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -m 'not network'
==================================================================================== test session starts ====================================================================================
platform linux -- Python 3.8.16, pytest-7.3.1, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/mdit-py-plugins-0.4.0
plugins: datadir-1.4.1, regressions-2.4.2
collected 410 items

tests/test_admon.py .....................F...                                                                                                                                         [  6%]
tests/test_amsmath.py ...................F                                                                                                                                            [ 10%]
tests/test_anchors.py ........                                                                                                                                                        [ 12%]
tests/test_attrs.py ...............................F                                                                                                                                  [ 20%]
tests/test_colon_fence.py ................................F.                                                                                                                          [ 29%]
tests/test_container.py ......................F                                                                                                                                       [ 34%]
tests/test_deflist.py ...............F                                                                                                                                                [ 38%]
tests/test_dollarmath.py ...............................................................F                                                                                             [ 54%]
tests/test_field_list.py ................F                                                                                                                                            [ 58%]
tests/test_footnote.py ......................F                                                                                                                                        [ 63%]
tests/test_front_matter.py .........                                                                                                                                                  [ 66%]
tests/test_myst_block.py ..............F..                                                                                                                                            [ 70%]
tests/test_myst_role.py ...............                                                                                                                                               [ 73%]
tests/test_substitution.py ..........                                                                                                                                                 [ 76%]
tests/test_tasklists.py .......                                                                                                                                                       [ 78%]
tests/test_texmath.py .......................................................................................                                                                         [ 99%]
tests/test_wordcount.py ...                                                                                                                                                           [100%]

========================================================================================= FAILURES ==========================================================================================
_ test_all[306-Indented by 4 spaces, DISABLE-CODEBLOCKS-    ??? note\n        content\n-<div class="admonition note is-collapsible collapsible-closed">\n<p class="admonition-title">Note</p>\n<p>content</p>\n</div>\n] _

line = 306, title = 'Indented by 4 spaces, DISABLE-CODEBLOCKS', input = '    ??? note\n        content\n'
expected = '<div class="admonition note is-collapsible collapsible-closed">\n<p class="admonition-title">Note</p>\n<p>content</p>\n</div>\n'

    @pytest.mark.parametrize(
        "line,title,input,expected",
        read_fixture_file(FIXTURE_PATH.joinpath("fixtures", "admon.md")),
    )
    def test_all(line, title, input, expected):
        md = MarkdownIt("commonmark").use(admon_plugin)
        if "DISABLE-CODEBLOCKS" in title:
            md.disable("code")
        md.options["xhtmlOut"] = False
        text = md.render(input)
        print(text)
>       assert text.rstrip() == expected.rstrip()
E       assert '<p>??? note\ncontent</p>' == '<div class="...t</p>\n</div>'
E         + <p>??? note
E         - <div class="admonition note is-collapsible collapsible-closed">
E         - <p class="admonition-title">Note</p>
E         - <p>content</p>
E         ? ---           -
E         + content</p>
E         - </div>

tests/test_admon.py:24: AssertionError
----------------------------------------------------------------------------------- Captured stdout call ------------------------------------------------------------------------------------
<p>??? note
content</p>

_ test_fixtures[237-Indented by 4 spaces, DISABLE-CODEBLOCKS-    \\begin{equation}\n    a = 1\n    \\end{equation}\n-<div class="math amsmath">\n\\begin{equation}\n    a = 1\n    \\end{equation}\n</div>\n] _

line = 237, title = 'Indented by 4 spaces, DISABLE-CODEBLOCKS', input = '    \\begin{equation}\n    a = 1\n    \\end{equation}\n'
expected = '<div class="math amsmath">\n\\begin{equation}\n    a = 1\n    \\end{equation}\n</div>\n'

    @pytest.mark.parametrize(
        "line,title,input,expected",
        read_fixture_file(FIXTURE_PATH.joinpath("fixtures", "amsmath.md")),
    )
    def test_fixtures(line, title, input, expected):
        md = MarkdownIt("commonmark").use(amsmath_plugin)
        if "DISABLE-CODEBLOCKS" in title:
            md.disable("code")
        md.options["xhtmlOut"] = False
        text = md.render(input)
        print(text)
>       assert text.rstrip() == expected.rstrip()
E       assert '<p>\\begin{e...equation}</p>' == '<div class="...tion}\n</div>'
E         - <div class="math amsmath">
E         - \begin{equation}
E         + <p>\begin{equation}
E         ? +++
E         -     a = 1
E         ? ----
E         + a = 1...
E
E         ...Full output truncated (5 lines hidden), use '-vv' to show

tests/test_amsmath.py:58: AssertionError
----------------------------------------------------------------------------------- Captured stdout call ------------------------------------------------------------------------------------
<p>\begin{equation}
a = 1
\end{equation}</p>

_______________________________ test_attrs[254-Indented by 4 spaces, DISABLE-CODEBLOCKS-    {#a .a b=c}\n    # head\n-<h1 id="a" b="c" class="a">head</h1>\n] _______________________________

line = 254, title = 'Indented by 4 spaces, DISABLE-CODEBLOCKS', input = '    {#a .a b=c}\n    # head\n', expected = '<h1 id="a" b="c" class="a">head</h1>\n'

    @pytest.mark.parametrize(
        "line,title,input,expected", read_fixture_file(FIXTURE_PATH / "attrs.md")
    )
    def test_attrs(line, title, input, expected):
        md = MarkdownIt("commonmark").use(attrs_plugin, spans=True).use(attrs_block_plugin)
        if "DISABLE-CODEBLOCKS" in title:
            md.disable("code")
        md.options["xhtmlOut"] = False
        text = md.render(input)
        print(text)
>       assert text.rstrip() == expected.rstrip()
E       assert '<p>{#a .a b=c}\n# head</p>' == '<h1 id="a" b..."a">head</h1>'
E         - <h1 id="a" b="c" class="a">head</h1>
E         + <p>{#a .a b=c}
E         + # head</p>

tests/test_attrs.py:22: AssertionError
----------------------------------------------------------------------------------- Captured stdout call ------------------------------------------------------------------------------------
<p>{#a .a b=c}
# head</p>

______________________ test_fixtures[438-Indented by 4 spaces, DISABLE-CODEBLOCKS-    :::name\n    foo\n    :::\n-<pre><code class="block-name" >foo\n</code></pre>\n] ______________________

line = 438, title = 'Indented by 4 spaces, DISABLE-CODEBLOCKS', input = '    :::name\n    foo\n    :::\n', expected = '<pre><code class="block-name" >foo\n</code></pre>\n'

    @pytest.mark.parametrize("line,title,input,expected", read_fixture_file(FIXTURE_PATH))
    def test_fixtures(line, title, input, expected):
        md = MarkdownIt("commonmark").use(colon_fence_plugin)
        if "DISABLE-CODEBLOCKS" in title:
            md.disable("code")
        md.options["xhtmlOut"] = False
        text = md.render(input)
        try:
>           assert text.rstrip() == expected.rstrip()
E           assert '<p>:::name\nfoo\n:::</p>' == '<pre><code c...</code></pre>'
E             - <pre><code class="block-name" >foo
E             - </code></pre>
E             + <p>:::name
E             + foo
E             + :::</p>

tests/test_colon_fence.py:21: AssertionError
----------------------------------------------------------------------------------- Captured stdout call ------------------------------------------------------------------------------------
<p>:::name
foo
:::</p>

_________________________ test_all[301-Indented by 4 spaces, DISABLE-CODEBLOCKS-    ::: name\n    content\n    :::\n-<div class="name">\n<p>content</p>\n</div>\n] __________________________

line = 301, title = 'Indented by 4 spaces, DISABLE-CODEBLOCKS', input = '    ::: name\n    content\n    :::\n', expected = '<div class="name">\n<p>content</p>\n</div>\n'

    @pytest.mark.parametrize("line,title,input,expected", read_fixture_file(FIXTURE_PATH))
    def test_all(line, title, input, expected):
        md = MarkdownIt("commonmark").use(container_plugin, "name")
        if "DISABLE-CODEBLOCKS" in title:
            md.disable("code")
        md.options["xhtmlOut"] = False
        text = md.render(input)
        print(text)
>       assert text.rstrip() == expected.rstrip()
E       assert '<p>::: name\...tent\n:::</p>' == '<div class="...t</p>\n</div>'
E         - <div class="name">
E         - <p>content</p>
E         - </div>
E         + <p>::: name
E         + content
E         + :::</p>

tests/test_container.py:50: AssertionError
----------------------------------------------------------------------------------- Captured stdout call ------------------------------------------------------------------------------------
<p>::: name
content
:::</p>

______________________ test_all[261-Indented by 4 spaces, DISABLE-CODEBLOCKS-    Term 1\n\n    : Definition 1\n-<dl>\n<dt>Term 1</dt>\n<dd>Definition 1</dd>\n</dl>\n] ______________________

line = 261, title = 'Indented by 4 spaces, DISABLE-CODEBLOCKS', input = '    Term 1\n\n    : Definition 1\n', expected = '<dl>\n<dt>Term 1</dt>\n<dd>Definition 1</dd>\n</dl>\n'

    @pytest.mark.parametrize("line,title,input,expected", read_fixture_file(FIXTURE_PATH))
    def test_all(line, title, input, expected):
        md = MarkdownIt("commonmark").use(deflist_plugin)
        if "DISABLE-CODEBLOCKS" in title:
            md.disable("code")
        md.options["xhtmlOut"] = False
        text = md.render(input)
        print(text)
>       assert text.rstrip() == expected.rstrip()
E       AssertionError: assert '<p>Term 1</p...inition 1</p>' == '<dl>\n<dt>Te...1</dd>\n</dl>'
E         - <dl>
E         - <dt>Term 1</dt>
E         ?  ^^         ^^
E         + <p>Term 1</p>
E         ?  ^         ^
E         - <dd>Definition 1</dd>
E         ?  ^^               ^^ -...
E
E         ...Full output truncated (3 lines hidden), use '-vv' to show

tests/test_deflist.py:39: AssertionError
----------------------------------------------------------------------------------- Captured stdout call ------------------------------------------------------------------------------------
<p>Term 1</p>
<p>: Definition 1</p>

_________________________________ test_dollarmath_fixtures[575-Indented by 4 spaces, DISABLE-CODEBLOCKS-    $$a$$\n-<div class="math block">\na\n</div>\n] __________________________________

line = 575, title = 'Indented by 4 spaces, DISABLE-CODEBLOCKS', input = '    $$a$$\n', expected = '<div class="math block">\na\n</div>\n'

    @pytest.mark.parametrize(
        "line,title,input,expected",
        read_fixture_file(FIXTURE_PATH.joinpath("dollar_math.md")),
    )
    def test_dollarmath_fixtures(line, title, input, expected):
        md = MarkdownIt("commonmark").use(
            dollarmath_plugin,
            allow_space=False,
            allow_digits=False,
            double_inline=True,
            allow_blank_lines=False,
        )
        if "DISABLE-CODEBLOCKS" in title:
            md.disable("code")
        md.options.xhtmlOut = False
        text = md.render(input)
        print(text)
>       assert text.rstrip() == expected.rstrip()
E       assert '<p><div clas...">a</div></p>' == '<div class="...">\na\n</div>'
E         + <p><div class="math inline">a</div></p>
E         - <div class="math block">
E         - a
E         - </div>

tests/test_dollarmath.py:106: AssertionError
----------------------------------------------------------------------------------- Captured stdout call ------------------------------------------------------------------------------------
<p><div class="math inline">a</div></p>

__________________________________________________________________ test_all[378-Indented_by_4_spaces,_DISABLE-CODEBLOCKS] ___________________________________________________________________

line = 378, title = 'Indented by 4 spaces, DISABLE-CODEBLOCKS', input = '    :name: text\n        indented\n'
expected = '<dl class="field-list">\n<dt>name</dt>\n<dd>\n<p>text\nindented</p>\n</dd>\n</dl>\n'

    @pytest.mark.parametrize(
        "line,title,input,expected",
        fixtures,
        ids=[f"{f[0]}-{f[1].replace(' ', '_')}" for f in fixtures],
    )
    def test_all(line, title, input, expected):
        md = MarkdownIt("commonmark").use(fieldlist_plugin)
        if "DISABLE-CODEBLOCKS" in title:
            md.disable("code")
        md.options["xhtmlOut"] = False
        text = md.render(input)
        print(text)
>       assert text.rstrip() == expected.rstrip()
E       assert '<p>:name: text\nindented</p>' == '<dl class="f...n</dd>\n</dl>'
E         + <p>:name: text
E         - <dl class="field-list">
E         - <dt>name</dt>
E         - <dd>
E         - <p>text
E         - indented</p>
E         ?             -...
E
E         ...Full output truncated (3 lines hidden), use '-vv' to show

tests/test_field_list.py:41: AssertionError
----------------------------------------------------------------------------------- Captured stdout call ------------------------------------------------------------------------------------
<p>:name: text
indented</p>

_ test_all[360-Indented by 4 spaces, DISABLE-CODEBLOCKS-    [^1]\n\n    [^1]: footnote\n-<p><sup class="footnote-ref"><a href="#fn1" id="fnref1">[1]</a></sup></p>\n<hr class="footnotes-sep">\n<section class="footnotes">\n<ol class="footnotes-list">\n<li id="fn1" class="footnote-item"><p>footnote <a href="#fnref1" class="footnote-backref">\u21a9\ufe0e</a></p>\n</li>\n</ol>\n</section>\n] _

line = 360, title = 'Indented by 4 spaces, DISABLE-CODEBLOCKS', input = '    [^1]\n\n    [^1]: footnote\n'
expected = '<p><sup class="footnote-ref"><a href="#fn1" id="fnref1">[1]</a></sup></p>\n<hr class="footnotes-sep">\n<section class...1" class="footnote-item"><p>footnote <a href="#fnref1" class="footnote-backref">↩︎</a></p>\n</li>\n</ol>\n</section>\n'

    @pytest.mark.parametrize("line,title,input,expected", read_fixture_file(FIXTURE_PATH))
    def test_all(line, title, input, expected):
        md = MarkdownIt("commonmark").use(footnote_plugin)
        if "DISABLE-CODEBLOCKS" in title:
            md.disable("code")
        md.options["xhtmlOut"] = False
        text = md.render(input)
        print(text)
>       assert text.rstrip().replace("↩︎", "<-").replace(
            "↩", "<-"
        ) == expected.rstrip().replace("↩︎", "<-").replace("↩", "<-")
E       assert '<p>[^1]</p>\... footnote</p>' == '<p><sup clas...>\n</section>'
E         + <p>[^1]</p>
E         + <p>[^1]: footnote</p>
E         - <p><sup class="footnote-ref"><a href="#fn1" id="fnref1">[1]</a></sup></p>
E         - <hr class="footnotes-sep">
E         - <section class="footnotes">
E         - <ol class="footnotes-list">
E         - <li id="fn1" class="footnote-item"><p>footnote <a href="#fnref1" class="footnote-backref"><-</a></p>...
E
E         ...Full output truncated (3 lines hidden), use '-vv' to show

tests/test_footnote.py:459: AssertionError
----------------------------------------------------------------------------------- Captured stdout call ------------------------------------------------------------------------------------
<p>[^1]</p>
<p>[^1]: footnote</p>

___ test_all[168-Indented by 4 spaces, DISABLE-CODEBLOCKS-    +++\n\n    % abc\n\n    (a)=\n-<hr class="myst-block">\n<!-- abc --><div class="myst-target"><a href="#a">(a)=</a></div>\n] ___

line = 168, title = 'Indented by 4 spaces, DISABLE-CODEBLOCKS', input = '    +++\n\n    % abc\n\n    (a)=\n'
expected = '<hr class="myst-block">\n<!-- abc --><div class="myst-target"><a href="#a">(a)=</a></div>\n'

    @pytest.mark.parametrize("line,title,input,expected", read_fixture_file(FIXTURE_PATH))
    def test_all(line, title, input, expected):
        md = MarkdownIt("commonmark").use(myst_block_plugin)
        if "DISABLE-CODEBLOCKS" in title:
            md.disable("code")
        md.options["xhtmlOut"] = False
        text = md.render(input)
        print(text)
>       assert text.rstrip() == expected.rstrip()
E       assert '<p>+++</p>\n...\n<p>(a)=</p>' == '<hr class="m...a)=</a></div>'
E         - <hr class="myst-block">
E         - <!-- abc --><div class="myst-target"><a href="#a">(a)=</a></div>
E         + <p>+++</p>
E         + <p>% abc</p>
E         + <p>(a)=</p>

tests/test_myst_block.py:21: AssertionError
----------------------------------------------------------------------------------- Captured stdout call ------------------------------------------------------------------------------------
<p>+++</p>
<p>% abc</p>
<p>(a)=</p>

================================================================================== short test summary info ==================================================================================
FAILED tests/test_admon.py::test_all[306-Indented by 4 spaces, DISABLE-CODEBLOCKS-    ??? note\n        content\n-<div class="admonition note is-collapsible collapsible-closed">\n<p class="admonition-title">Note</p>\n<p>content</p>\n</div>\n] - assert '<p>??? note\ncontent</p>' == '<div class="...t</p>\n</div>'
FAILED tests/test_amsmath.py::test_fixtures[237-Indented by 4 spaces, DISABLE-CODEBLOCKS-    \\begin{equation}\n    a = 1\n    \\end{equation}\n-<div class="math amsmath">\n\\begin{equation}\n    a = 1\n    \\end{equation}\n</div>\n] - assert '<p>\\begin{e...equation}</p>' == '<div class="...tion}\n</div>'
FAILED tests/test_attrs.py::test_attrs[254-Indented by 4 spaces, DISABLE-CODEBLOCKS-    {#a .a b=c}\n    # head\n-<h1 id="a" b="c" class="a">head</h1>\n] - assert '<p>{#a .a b=c}\n# head</p>' == '<h1 id="a" b..."a">head</h1>'
FAILED tests/test_colon_fence.py::test_fixtures[438-Indented by 4 spaces, DISABLE-CODEBLOCKS-    :::name\n    foo\n    :::\n-<pre><code class="block-name" >foo\n</code></pre>\n] - assert '<p>:::name\nfoo\n:::</p>' == '<pre><code c...</code></pre>'
FAILED tests/test_container.py::test_all[301-Indented by 4 spaces, DISABLE-CODEBLOCKS-    ::: name\n    content\n    :::\n-<div class="name">\n<p>content</p>\n</div>\n] - assert '<p>::: name\...tent\n:::</p>' == '<div class="...t</p>\n</div>'
FAILED tests/test_deflist.py::test_all[261-Indented by 4 spaces, DISABLE-CODEBLOCKS-    Term 1\n\n    : Definition 1\n-<dl>\n<dt>Term 1</dt>\n<dd>Definition 1</dd>\n</dl>\n] - AssertionError: assert '<p>Term 1</p...inition 1</p>' == '<dl>\n<dt>Te...1</dd>\n</dl>'
FAILED tests/test_dollarmath.py::test_dollarmath_fixtures[575-Indented by 4 spaces, DISABLE-CODEBLOCKS-    $$a$$\n-<div class="math block">\na\n</div>\n] - assert '<p><div clas...">a</div></p>' == '<div class="...">\na\n</div>'
FAILED tests/test_field_list.py::test_all[378-Indented_by_4_spaces,_DISABLE-CODEBLOCKS] - assert '<p>:name: text\nindented</p>' == '<dl class="f...n</dd>\n</dl>'
FAILED tests/test_footnote.py::test_all[360-Indented by 4 spaces, DISABLE-CODEBLOCKS-    [^1]\n\n    [^1]: footnote\n-<p><sup class="footnote-ref"><a href="#fn1" id="fnref1">[1]</a></sup></p>\n<hr class="footnotes-sep">\n<section class="footnotes">\n<ol class="footnotes-list">\n<li id="fn1" class="footnote-item"><p>footnote <a href="#fnref1" class="footnote-backref">\u21a9\ufe0e</a></p>\n</li>\n</ol>\n</section>\n] - assert '<p>[^1]</p>\... footnote</p>' == '<p><sup clas...>\n</section>'
FAILED tests/test_myst_block.py::test_all[168-Indented by 4 spaces, DISABLE-CODEBLOCKS-    +++\n\n    % abc\n\n    (a)=\n-<hr class="myst-block">\n<!-- abc --><div class="myst-target"><a href="#a">(a)=</a></div>\n] - assert '<p>+++</p>\n...\n<p>(a)=</p>' == '<hr class="m...a)=</a></div>'
============================================================================== 10 failed, 400 passed in 1.22s ===============================================================================

List your environment

Here is list of installed modules in build env

Package                       Version
----------------------------- -------
alabaster                     0.7.13
asttokens                     2.2.1
Babel                         2.12.1
backcall                      0.2.0
build                         0.10.0
charset-normalizer            3.1.0
decorator                     5.1.1
distro                        1.8.0
docutils                      0.19
exceptiongroup                1.1.1
executing                     1.2.0
gpg                           1.20.0
idna                          3.4
imagesize                     1.4.1
importlib-metadata            6.6.0
iniconfig                     2.0.0
installer                     0.7.0
ipython                       8.12.0
jedi                          0.18.2
Jinja2                        3.1.2
libcomps                      0.1.19
markdown-it-py                2.2.0
MarkupSafe                    2.1.2
matplotlib-inline             0.1.6
mdit-py-plugins               0.3.5
mdurl                         0.1.2
myst-parser                   1.0.0
packaging                     23.1
parso                         0.8.3
pexpect                       4.8.0
pickleshare                   0.7.5
pluggy                        1.0.0
prompt-toolkit                3.0.38
ptyprocess                    0.7.0
pure-eval                     0.2.2
Pygments                      2.15.1
pyproject_hooks               1.0.0
pytest                        7.3.1
pytest-datadir                1.4.1
pytest-regressions            2.4.2
python-dateutil               2.8.2
pytz                          2023.2
PyYAML                        6.0
requests                      2.30.0
setuptools                    67.7.2
six                           1.16.0
snowballstemmer               2.2.0
Sphinx                        6.2.1
sphinxcontrib-applehelp       1.0.4
sphinxcontrib-devhelp         1.0.2
sphinxcontrib-htmlhelp        2.0.0
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.3
sphinxcontrib-serializinghtml 1.1.5
stack-data                    0.6.2
tomli                         2.0.1
traitlets                     5.9.0
typing_extensions             4.5.0
urllib3                       1.26.15
wcwidth                       0.2.6
wheel                         0.40.0
zipp                          3.15.0

[eamanu]

Hi @kloczek, same issue in Debian. Please see #92