Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pytest-html 3.2.0 error on report generation on Gitlab-CI #578

Open
musjaster opened this issue Jan 2, 2023 · 40 comments
Open

pytest-html 3.2.0 error on report generation on Gitlab-CI #578

musjaster opened this issue Jan 2, 2023 · 40 comments

Comments

@musjaster
Copy link

pytest-html 3.2.0 error on report generation on Gitlab-CI
Observed: error on report generation
NB: we have downgraded to version 3.1.1 and everything works well

I think the problem is in: Explicitly add py.xml dependency.

And the issue is the same as: microsoft/pylance-release#2357

Log:
File "/Users/mobile-ci/.pyenv/versions/3.9.16/bin/pytest", line 8, in
sys.exit(console_main())
File "/Users/mobile-ci/.pyenv/versions/3.9.16/lib/python3.9/site-packages/_pytest/config/init.py", line 190, in console_main
code = main()
File "/Users/mobile-ci/.pyenv/versions/3.9.16/lib/python3.9/site-packages/_pytest/config/init.py", line 167, in main
ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(
File "/Users/mobile-ci/.pyenv/versions/3.9.16/lib/python3.9/site-packages/pluggy/_hooks.py", line 265, in call
return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
File "/Users/mobile-ci/.pyenv/versions/3.9.16/lib/python3.9/site-packages/pluggy/_manager.py", line 80, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
File "/Users/mobile-ci/.pyenv/versions/3.9.16/lib/python3.9/site-packages/pluggy/_callers.py", line 60, in _multicall
return outcome.get_result()
File "/Users/mobile-ci/.pyenv/versions/3.9.16/lib/python3.9/site-packages/pluggy/_result.py", line 60, in get_result
raise ex[1].with_traceback(ex[2])
File "/Users/mobile-ci/.pyenv/versions/3.9.16/lib/python3.9/site-packages/pluggy/_callers.py", line 39, in _multicall
res = hook_impl.function(*args)
File "/Users/mobile-ci/.pyenv/versions/3.9.16/lib/python3.9/site-packages/_pytest/main.py", line 317, in pytest_cmdline_main
return wrap_session(config, _main)
File "/Users/mobile-ci/.pyenv/versions/3.9.16/lib/python3.9/site-packages/_pytest/main.py", line 305, in wrap_session
config.hook.pytest_sessionfinish(
File "/Users/mobile-ci/.pyenv/versions/3.9.16/lib/python3.9/site-packages/pluggy/_hooks.py", line 265, in call
return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
File "/Users/mobile-ci/.pyenv/versions/3.9.16/lib/python3.9/site-packages/pluggy/_manager.py", line 80, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
File "/Users/mobile-ci/.pyenv/versions/3.9.16/lib/python3.9/site-packages/pluggy/_callers.py", line 55, in _multicall
gen.send(outcome)
File "/Users/mobile-ci/.pyenv/versions/3.9.16/lib/python3.9/site-packages/_pytest/terminal.py", line 808, in pytest_sessionfinish
outcome.get_result()
File "/Users/mobile-ci/.pyenv/versions/3.9.16/lib/python3.9/site-packages/pluggy/_result.py", line 60, in get_result
raise ex[1].with_traceback(ex[2])
File "/Users/mobile-ci/.pyenv/versions/3.9.16/lib/python3.9/site-packages/pluggy/_callers.py", line 39, in _multicall
res = hook_impl.function(*args)
File "/Users/mobile-ci/.pyenv/versions/3.9.16/lib/python3.9/site-packages/pytest_html/html_report.py", line 332, in pytest_sessionfinish
report_content = self._generate_report(session)
File "/Users/mobile-ci/.pyenv/versions/3.9.16/lib/python3.9/site-packages/pytest_html/html_report.py", line 206, in _generate_report
unicode_doc = "\n{}".format(doc.unicode(indent=2))
File "/Users/mobile-ci/.pyenv/versions/3.9.16/lib/python3.9/site-packages/py/_xmlgen.py", line 69, in unicode
HtmlVisitor(l.append, indent, shortempty=False).visit(self)
File "/Users/mobile-ci/.pyenv/versions/3.9.16/lib/python3.9/site-packages/py/_xmlgen.py", line 128, in visit
visitmethod(node)
File "/Users/mobile-ci/.pyenv/versions/3.9.16/lib/python3.9/site-packages/py/_xmlgen.py", line 160, in Tag
self.visit(x)
File "/Users/mobile-ci/.pyenv/versions/3.9.16/lib/python3.9/site-packages/py/_xmlgen.py", line 128, in visit
visitmethod(node)
File "/Users/mobile-ci/.pyenv/versions/3.9.16/lib/python3.9/site-packages/py/_xmlgen.py", line 160, in Tag
self.visit(x)
File "/Users/mobile-ci/.pyenv/versions/3.9.16/lib/python3.9/site-packages/py/_xmlgen.py", line 128, in visit
visitmethod(node)
File "/Users/mobile-ci/.pyenv/versions/3.9.16/lib/python3.9/site-packages/py/_xmlgen.py", line 160, in Tag
self.visit(x)
File "/Users/mobile-ci/.pyenv/versions/3.9.16/lib/python3.9/site-packages/py/_xmlgen.py", line 128, in visit
visitmethod(node)
File "/Users/mobile-ci/.pyenv/versions/3.9.16/lib/python3.9/site-packages/py/_xmlgen.py", line 140, in list
assert id(obj) not in self.visited
AssertionError: assert 4500331456 not in {4500220800: 1, 4500222816: 1, 4500259040: 1, 4500259280: 1, ...}

  • where 4500331456 = id([<'span' tag object 4500315888>, <'span' tag object 4500315728>, <'span' tag object 4500315408>, <'span' tag object 4500313728>])
  • and {4500220800: 1, 4500222816: 1, 4500259040: 1, 4500259280: 1, ...} = <py._xmlgen.HtmlVisitor object at 0x10c3bbb20>.visited
@BeyondEvil
Copy link
Contributor

Please try 4.0.0rc0

@musjaster
Copy link
Author

musjaster commented Mar 8, 2023 via email

@BeyondEvil
Copy link
Contributor

Thank you, I'll look into it.

@BeyondEvil
Copy link
Contributor

Can you try 4.0.0rc1 🙇

@musjaster
Copy link
Author

musjaster commented Apr 5, 2023 via email

@BeyondEvil
Copy link
Contributor

BeyondEvil commented Apr 5, 2023 via email

@musjaster
Copy link
Author

musjaster commented Apr 5, 2023 via email

@BeyondEvil
Copy link
Contributor

What’s the problem with title?

@musjaster
Copy link
Author

musjaster commented Apr 5, 2023 via email

@BeyondEvil
Copy link
Contributor

Has this worked previously?

Because title is just a text-field. It doesn’t parse HTML.

@musjaster
Copy link
Author

musjaster commented Apr 5, 2023 via email

@BeyondEvil
Copy link
Contributor

Can you share a screenshot of how it used to look?

Was the title clickable?

@musjaster
Copy link
Author

musjaster commented Apr 5, 2023 via email

@BeyondEvil
Copy link
Contributor

In #578 (comment) you write "and I have attached the file where the error occurs", but the comment is sent via email, and I don't think attachments are visible then.

You have to actually visit this issue and attach them manually.

@BeyondEvil
Copy link
Contributor

So I went through the history, specifically this PR (and this one), and the intention (which the contributor actually noted) was never for the title to be anything other than text.

I would suggest logging a new issue with a feature request of making title clickable.

In the meantime my suggestion is to use the summary (prefix) hook to achieve what you're looking for.

@musjaster
Copy link
Author

musjaster commented Apr 6, 2023 •
edited

i'm attaching the reports here
Archive.zip

@musjaster
Copy link
Author

Screenshot 2023-04-05 at 20 11 30

@musjaster
Copy link
Author

musjaster commented Apr 6, 2023 •
edited by BeyondEvil

In the meantime my suggestion is to use the summary (prefix) hook to achieve what you're looking for.

what i see in your link is:

from py.xml import html

but with new version pytest-html
i have the warning DeprecationWarning: The 'py' module is deprecated and support will be removed in a future release.
so another question are you going to stop using deprecated modules? ;)

@musjaster
Copy link
Author

musjaster commented Apr 6, 2023 via email

@BeyondEvil
Copy link
Contributor

In the meantime my suggestion is to use the summary (prefix) hook to achieve what you're looking for.

what i see in your link is:

from py.xml import html

but with new version pytest-html i have the warning DeprecationWarning: The 'py' module is deprecated and support will be removed in a future release. so another question are you going to stop using deprecated modules? ;)

Thanks for pointing that out. I just need to update the docs. As you can see in the same box, we're actually not using the imported module.

And we're not using the py-module in code.

We transform your code if you're using it. 😊

@musjaster
Copy link
Author

musjaster commented Apr 6, 2023 via email

@BeyondEvil
Copy link
Contributor

BeyondEvil commented Apr 6, 2023 •
edited

yes, I have seen but what I need to do to avoid the warning? stop using it too? or i can use another module on its place you see the report was done long ago on our project and it is working well thus i don't want to see warning, but from the other hand if i need to rewrite everything to get rid of them .... it would be not very productive
…
On Thu, 6 Apr 2023 at 10:08, Jim Brännlund @.> wrote: In the meantime my suggestion is to use the summary (prefix) hook https://pytest-html.readthedocs.io/en/latest/user_guide.html#additional-summary-information to achieve what you're looking for. what i see in your link is: from py.xml import html but with new version pytest-html i have the warning DeprecationWarning: The 'py' module is deprecated and support will be removed in a future release. so another question are you going to stop using deprecated modules? ;) Thanks for pointing that out. I just need to update the docs. As you can see in the same box, we're actually not using the imported module. And we're not using the py-module in code. We transform https://github.com/pytest-dev/pytest-html/blob/master/src/pytest_html/table.py#L44 your code if you're using it. 😊 — Reply to this email directly, view it on GitHub <#578 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/A5BGMCC7G7URQXMY3OIEWZLW7Z2XPANCNFSM6AAAAAATOUFFEE . You are receiving this because you authored the thread.Message ID: @.>

To avoid the warning you can add filterwarnings: https://docs.pytest.org/en/6.2.x/warnings.html#deprecationwarning-and-pendingdeprecationwarning

I would however recommend stop using it, since the py lib itself is deprecated AND has a security vuln: pytest-dev/py#287

Also note that the filterwarning will only work until we remove support, then it will start failing hard.

Depending on how much usage you have you can write a migration script to replace it.

@musjaster
Copy link
Author

musjaster commented Apr 6, 2023 via email

@musjaster
Copy link
Author

musjaster commented Apr 7, 2023 via email

@BeyondEvil
Copy link
Contributor

  1. I can’t repro this. Can you provide how to repro?
  2. The ini-config or query param?

@musjaster
Copy link
Author

musjaster commented Apr 7, 2023 via email

@BeyondEvil
Copy link
Contributor

  1. no sorry, I was wrong, I was looking at the old report 2. ini config render_collapsed = True i have not understood where to put query param frankly speaking br
    …
  1. Super ☺️
  2. Try ”all” instead of true (you can now pass the same in the ini as in query, check docs)

@musjaster
Copy link
Author

musjaster commented Apr 7, 2023 via email

@musjaster
Copy link
Author

maybe it happens because SKIP is imported not directly from pytest, but is redefined in test.py ?

400rc1_skipped

@BeyondEvil
Copy link
Contributor

BeyondEvil commented Apr 7, 2023 •
edited

Is it just the summary or are they missing in the table (as rows) as well?

If pytest reports it, so should the html report

@musjaster
Copy link
Author

musjaster commented Apr 7, 2023 via email

@BeyondEvil
Copy link
Contributor

BeyondEvil commented Apr 7, 2023 •
edited

Well, the screenshot is cut off. That’s why I asked.

@musjaster
Copy link
Author

musjaster commented Apr 7, 2023 via email

@BeyondEvil
Copy link
Contributor

no, it is not cut off at all the 1st one is FAILED then i usually have SKIPPED and PASSED after all of them
…
On Fri, 7 Apr 2023 at 12:28, Jim Brännlund @.> wrote: Well, the screenshot is cut off and is only showing ”Passed” (no the Failed ones either). That’s why I asked. — Reply to this email directly, view it on GitHub <#578 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/A5BGMCBGEFFUV75U2DM6ED3W77T47ANCNFSM6AAAAAATOUFFEE . You are receiving this because you authored the thread.Message ID: @.>

Regardless, I can't repro with 4.0.0rc1. Are there any errors in the browser console?

Screenshot 2023-04-07 at 12 35 26

Screenshot 2023-04-07 at 12 36 11

Screenshot 2023-04-07 at 12 36 21

If you can create a minimal example repo that reproduces the issue, I can fix it.

@BeyondEvil
Copy link
Contributor

BeyondEvil commented Apr 7, 2023 •
edited

Also note that, with the way the green arrow is pointing, the sort order would be:

Error
Failed
Passed
Rerun
Skipped
XFailed
XPassed

So "Skipped" would come after "Passed" in the table.

@musjaster
Copy link
Author

this is the console log

=============================== warnings summary ===============================
backend/scripts/company_v3.py: 12 warnings
backend/scripts/trip_v3.py: 18 warnings
backend/scripts/trip_filters_v3.py: 6 warnings
backend/scripts/obt_information.py: 21 warnings
backend/scripts/user_v3.py: 21 warnings
backend/scripts/event_v3.py: 24 warnings
backend/scripts/messages_v3.py: 9 warnings
/Users/mobile-ci/builds/8CBqSu93/0/mobility/qa-automation/.pyenv/lib/python3.9/site-packages/pytest_html/table.py:36: DeprecationWarning: The 'py' module is deprecated and support will be removed in a future release.
warnings.warn(
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

  • generated xml file: /Users/mobile-ci/builds/8CBqSu93/0/mobility/qa-automation/results.xml -
  • Generated html report: file:///Users/mobile-ci/builds/8CBqSu93/0/mobility/qa-automation/report.html -
    =========================== short test summary info ============================
    SKIPPED [4] core/test.py:109: impl in MOB-3503
    FAILED backend/scripts/trip_v3.py::TestTripV3::test_trip_by_ids_v3 - KeyError: 'count'
    ====== 1 failed, 36 passed, 4 skipped, 111 warnings in 144.02s (0:02:24) =======
    Uploading artifacts for failed job

you see that SKIPPED is marked as happening not in the test file, but in the main test class

i attach you report, that you can see that there are no SKIPPED in report (they are skipped, lol )

report.html.zip

@BeyondEvil
Copy link
Contributor

this is the console log

Well yes, but I wanted the browser console log (if any): https://stackoverflow.com/questions/13490302/how-can-i-see-the-output-of-console-log

=========================== short test summary info ============================
SKIPPED [4] core/test.py:109: impl in MOB-3503
FAILED backend/scripts/trip_v3.py::TestTripV3::test_trip_by_ids_v3 - KeyError: 'count'
====== 1 failed, 36 passed, 4 skipped, 111 warnings in 144.02s (0:02:24) =======
Uploading artifacts for failed job

you see that SKIPPED is marked as happening not in the test file, but in the main test class

Oh, interesting. So the skip marker is put on the entire class? Like so: https://docs.pytest.org/en/6.2.x/skipping.html#skip-all-test-functions-of-a-class-or-module

@BeyondEvil
Copy link
Contributor

I can repro now! 😊

Essentially, if the test is skipped using a marker, it's not picked up by the report.

Great find, thank you! 🙏

@musjaster
Copy link
Author

great
because i was going to reproduce the whole test Class hierachy here, lol

@BeyondEvil
Copy link
Contributor

great because i was going to reproduce the whole test Class hierachy here, lol

Haha 😅

No worries, I've identified the root-cause as well. So a fix will be included in RC2. 🥳

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@BeyondEvil @musjaster