Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

'module' object has no attribute 'GtkWarning' #404

Open
ThrashAbaddon opened this issue Jan 17, 2019 · 11 comments
Open

'module' object has no attribute 'GtkWarning' #404

ThrashAbaddon opened this issue Jan 17, 2019 · 11 comments
Labels
question

Comments

@ThrashAbaddon
Copy link

ThrashAbaddon commented Jan 17, 2019 •
edited

Hello. First of all I would like to thank everyone involved in making and maintaining pytest and xdist, superb job. :) 馃憤

Now the situation that is manifesting on my windows7 machine.
I'm trying to assemble CI infrastructure using jenkins for running gui tests via pytest and xdist.
I decided to use xdist because python process sometimes crashes and I want to continue running the remaining number of tests. I've managed to reproduce situation bellow also in the console (cmd.exe) so I don't think this is jenkins related.

Bellow we can see one of the commands I'm using to run pytest, I've moved some args onto a new line so it is easier on the eyes.

C:\Users\user\.jenkins\workspace\Rmaster_GUI>(D: &&
D:/python_folder/python -m pytest --disable-pytest-warnings --showlocals -n1
--cov=D:/installation_folder/python/client/project_name
--cov-report xml:C:\Users\user\.jenkins\workspace\Rmaster_GUI/coverage-reports/coverage.xml
--junit-xml=C:\Users\user\.jenkins\workspace\Rmaster_GUI/test-reports/results.xml
-v
C:\Users\user\.jenkins\workspace\Rmaster_GUI/test/project_name_test )

17:54:47 ============================= test session starts =============================
17:54:47 platform win32 -- Python 2.7.10, pytest-4.1.1, py-1.5.4, pluggy-0.8.1 -- D:\python_folder\python.exe
17:54:47 cachedir: .pytest_cache
17:54:47 rootdir: D:\, inifile:
17:54:47 plugins: xdist-1.26.0, forked-1.0.1, cov-2.5.1
17:54:47 gw0 I
17:54:47 
[gw0] win32 Python 2.7.10 cwd: D:\
17:54:48 
[gw0] Python 2.7.10 (default, Nov 29 2018, 13:23:51) [MSC v.1900 64 bit (AMD64)]
17:55:45 gw0 [1096]
17:55:45 
17:55:45 scheduling tests via LoadScheduling

## After some time and a couple tests ...

17:59:22 [gw0] [  1%] PASSED integration/gui/test_some_component.py::TestSomeComponentEditor::test_component !!!!!!!!!!!!!!!!!!!! D:\python_folder\Lib\site-packages\xdist\workermanage.py:436: AttributeError: 'module' object has no attribute 'GtkWarning'
17:59:22 
17:59:22 INTERNALERROR> Traceback (most recent call last):
17:59:22 INTERNALERROR>   File "D:\python_folder\Lib\site-packages\xdist\workermanage.py", line 336, in process_from_remote
17:59:22 INTERNALERROR>     kwargs["warning_message_data"]
17:59:22 INTERNALERROR>   File "D:\python_folder\Lib\site-packages\xdist\workermanage.py", line 436, in unserialize_warning_message
17:59:22 INTERNALERROR>     cls = getattr(mod, data["message_class_name"])
17:59:22 INTERNALERROR> AttributeError: 'module' object has no attribute 'GtkWarning'
17:59:22 [gw0] node down: D:\python_folder\Lib\site-packages\xdist\workermanage.py:436: AttributeError: 'module' object has no attribute 'GtkWarning'
17:59:22 [gw0] [  1%] FAILED integration/gui/test_some_component.py::TestSomeComponentEditor::test_component
17:59:22 Replacing crashed worker gw0
17:59:22 
[gw1] win32 Python 2.7.10 cwd: D:\
17:59:22 Coverage.py warning: No data was collected. (no-data-collected)
17:59:37 INTERNALERROR> Traceback (most recent call last):
17:59:37 INTERNALERROR>   File "D:\python_folder\Lib\site-packages\_pytest\main.py", line 203, in wrap_session
17:59:37 INTERNALERROR>     session.exitstatus = doit(config, session) or 0
17:59:37 INTERNALERROR>   File "D:\python_folder\Lib\site-packages\_pytest\main.py", line 243, in _main
17:59:37 INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
17:59:37 INTERNALERROR>   File "D:\python_folder\Lib\site-packages\pluggy\hooks.py", line 284, in __call__
17:59:37 INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
17:59:37 INTERNALERROR>   File "D:\python_folder\Lib\site-packages\pluggy\manager.py", line 68, in _hookexec
17:59:37 INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
17:59:37 INTERNALERROR>   File "D:\python_folder\Lib\site-packages\pluggy\manager.py", line 62, in <lambda>
17:59:37 INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
17:59:37 INTERNALERROR>   File "D:\python_folder\Lib\site-packages\pluggy\callers.py", line 203, in _multicall
17:59:37 INTERNALERROR>     gen.send(outcome)
17:59:37 INTERNALERROR>   File "D:\python_folder\Lib\site-packages\pluggy\callers.py", line 81, in get_result
17:59:37 INTERNALERROR>     _reraise(*ex)  # noqa
17:59:37 INTERNALERROR>   File "D:\python_folder\Lib\site-packages\pluggy\callers.py", line 187, in _multicall
17:59:37 INTERNALERROR>     res = hook_impl.function(*args)
17:59:37 INTERNALERROR>   File "D:\python_folder\Lib\site-packages\xdist\dsession.py", line 115, in pytest_runtestloop
17:59:37 INTERNALERROR>     self.loop_once()
17:59:37 INTERNALERROR>   File "D:\python_folder\Lib\site-packages\xdist\dsession.py", line 138, in loop_once
17:59:37 INTERNALERROR>     call(**kwargs)
17:59:37 INTERNALERROR>   File "D:\python_folder\Lib\site-packages\xdist\dsession.py", line 258, in worker_runtest_protocol_complete
17:59:37 INTERNALERROR>     self.sched.mark_test_complete(node, item_index, duration)
17:59:37 INTERNALERROR>   File "D:\python_folder\Lib\site-packages\xdist\scheduler\load.py", line 151, in mark_test_complete
17:59:37 INTERNALERROR>     self.node2pending[node].remove(item_index)
17:59:37 INTERNALERROR> KeyError: <WorkerController gw0>
17:59:47 
17:59:47 ============= 18 failed, 1 passed, 312 warnings in 298.89 seconds =============

Above code was executed on jenkins and the one bellow in the console on the same computer.

D:\>my_python -m pytest --disable-pytest-warnings -v -l -n1 C:\Users\user\.jenkins\workspace\master_GUI\test\client_test\integration\gui
============================= test session starts =============================
platform win32 -- Python 2.7.10, pytest-4.1.1, py-1.5.4, pluggy-0.8.1 -- D:\python_folder\python.exe
cachedir: .pytest_cache
rootdir: D:\, inifile:
plugins: xdist-1.26.0, forked-1.0.1, cov-2.5.1
[gw0] win32 Python 2.7.10 cwd: D:\
[gw0] Python 2.7.10 (default, Nov 29 2018, 13:23:51) [MSC v.1900 64 bit (AMD64)]
gw0 [27]
scheduling tests via LoadScheduling

test_component.py::TestComponentEditor::test_component
[gw0] [ 37%] PASSED test_some_other_component.py::test_some_other_component !!!!!!!!!!!!!!!!!!!! D:\python_folder\Lib\site-packages\xdist\workermanage.py:436: Attr
ibuteError: 'module' object has no attribute 'GtkWarning'

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\xdist\workermanage.py", line 336, in process_from_remote
INTERNALERROR>     kwargs["warning_message_data"]
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\xdist\workermanage.py", line 436, in unserialize_warning_message
INTERNALERROR>     cls = getattr(mod, data["message_class_name"])
INTERNALERROR> AttributeError: 'module' object has no attribute 'GtkWarning'
[gw0] node down: D:\python_folder\Lib\site-packages\xdist\workermanage.py:436: AttributeError: 'module' object has no attribute 'GtkWarning'
[gw0] [ 37%] FAILED test_some_other_component.py::test_some_other_component
Replacing crashed worker gw0
!!!!!!!!!!!!!!!!!!!! D:\python_folder\Lib\site-packages\xdist\workermanage.py:436: AttributeError: 'module' object has no attribute 'GtkWarning'
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\xdist\workermanage.py", line 336, in process_from_remote
INTERNALERROR>     kwargs["warning_message_data"]
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\xdist\workermanage.py", line 436, in unserialize_warning_message
INTERNALERROR>     cls = getattr(mod, data["message_class_name"])
INTERNALERROR> AttributeError: 'module' object has no attribute 'GtkWarning'
[gw1] win32 Python 2.7.10 cwd: D:\
gw1 C[gw0] node down: D:\python_folder\Lib\site-packages\xdist\workermanage.py:436: AttributeError: 'module' object has no attribute 'GtkWarning'
Replacing crashed worker gw0
Exception KeyError: (1,) in <function remove at 0x0000000004162208> ignored
[gw2] win32 Python 2.7.10 cwd: D:\
gw2 CINTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\_pytest\main.py", line 203, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\_pytest\main.py", line 243, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\pluggy\hooks.py", line 284, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\pluggy\manager.py", line 68, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\pluggy\manager.py", line 62, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\pluggy\callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\pluggy\callers.py", line 81, in get_result
INTERNALERROR>     _reraise(*ex)  # noqa
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\pluggy\callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\xdist\dsession.py", line 115, in pytest_runtestloop
INTERNALERROR>     self.loop_once()
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\xdist\dsession.py", line 138, in loop_once
INTERNALERROR>     call(**kwargs)
INTERNALERROR>   File "D:\python_folder\Lib\site-packages\xdist\dsession.py", line 208, in worker_errordown
INTERNALERROR>     self._active_nodes.remove(node)
INTERNALERROR> KeyError: <WorkerController gw0>

============ 4 failed, 7 passed, 68736 warnings in 784.27 seconds =============

As far as I can see not all tests are being executed because all nodes crash somewhere along the path and don't restart.
Traceback in jenkins and cmd are equal up to the line 138, in loop_once where after they start to diverge but end up with the same KeyError.

I'm also using pytest and xdist for running non gui tests and they work perfectly ok.
@RonnyPfannschmidt
Copy link
Member

im under the impression the lookup for warnings class of gtk desnt work as normal warning lookup - im not sure how to debug this one

@ThrashAbaddon
Copy link
Author

Maybe I can do it for you. How do you do it for other classes?

@RonnyPfannschmidt
Copy link
Member

off hand no idea, @nicoddemus took charge of warning serialization, and that bit had a fair share of issues due to strangeness in the warning system ^^, also im basically just on the computer to call in sick, i'll hit the bed soon

@ThrashAbaddon
Copy link
Author

ThrashAbaddon commented Jan 17, 2019 •
edited

Uh, hope you get better soon.

I don't care about the gtk warnings so I'll try my luck with warnings module but I'm not sure what hook to use in conftest.py. Maybe

import warnings
warnings.filterwarnings("ignore")

inside pytest_sessionstart?

@nicoddemus
Copy link
Member

@ThrashAbaddon ignoring the warning like that should work. Please report back with any findings. 馃憤

@ThrashAbaddon
Copy link
Author

This works so @RonnyPfannschmidt thanks for the hint. :)
The solution was to add in my conftest.py following code:

import warnings

def pytest_sessionstart(session):
    warnings.filterwarnings("ignore")

Also, as GTK is popular gui framework I would like to have pytest working out of the box with GTKWarnings, or have a detailed explanation and workaround in official documentation. What do you guys think?

@nicoddemus
Copy link
Member

What do you guys think?

It sounds good. The first step, I think, would be to understand if there's any "magic" being done by GTK that we can't import the warning classes in xdist.

@RonnyPfannschmidt
Copy link
Member

I believe we need some kind of fix

@smarie
Copy link
Contributor

smarie commented Jan 25, 2022 •
edited

We seeem to have met the same issue here : smarie/python-pytest-cases#249

The problem has the same cause: the unserialize_warning_message function of xdist seems to assume that the warning class will be an attribute of the top-level package (not of a sub-package).

INTERNALERROR>   File "D:\python_folder\Lib\site-packages\xdist\workermanage.py", line 436, in unserialize_warning_message
INTERNALERROR>     cls = getattr(mod, data["message_class_name"])

Of course a workaround is to make our libs expose all warning classes at the top package level. However, there is probably an easier fix to be done in unserialize_warning_message ? Maybe the full qualified name of the warnings class could be serialized so that it can be deserialized ok ?

@smarie smarie mentioned this issue Jan 25, 2022
Closed
@smarie
Copy link
Contributor

smarie commented Mar 21, 2022

I found the culprit: https://github.com/smarie/python-pytest-cases/blob/8d11b7de97bd8c18891aa7d5f21cd45539486843/src/pytest_cases/case_parametrizer_new.py#L682-L686

The issue is that I set __module__ myself so that the warning message looks nice... However then the __init__ should contain the class for consistency.

@smarie
Copy link
Contributor

smarie commented Mar 21, 2022

Still, while investigating this issue, I created a few handy tests concerning warning serialization/deserialization. So I'll propose them in a PR

smarie pushed a commit to smarie/pytest-xdist that referenced this issue Mar 21, 2022
Added a couple tests for pytest-dev#404
f21536e
@smarie smarie mentioned this issue Mar 21, 2022
Merged
ssbarnea pushed a commit that referenced this issue Aug 23, 2022
@smarie
Added a couple tests for #404 (#766)
9236f11 
* Added a couple tests for #404
Co-authored-by: Sylvain MARIE <sylvain.marie@se.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@RonnyPfannschmidt @nicoddemus @smarie @ThrashAbaddon