Travis tests do not quit cleanly under linux.

[BoPeng]

I have spent long hours trying to figure out why travis tests do not quit cleanly under travis. It seems to boils down to test_execute.py and removing one test seemed to work but there is nothing suspicious about this test.

I suspected zombie processes and wraps the tests under check_zombie.py but this does not seems to be a problem either. Perhaps we will have to log in to travis or get a test docker image to figure out the reason.

[gaow]

Indeed, what's special about this workflow in question? Is there another simpler test for executing ipynb files? Can it be some dependency issues parsing ipynb files?

Also current build still seems to fail even though all tests pass? Is that the warning messages causing it? For the warning message,

    from collections import Sequence, Mapping, Set

should now be

    from collections.abc import Sequence, Mapping, Set

[BoPeng]

I created a branch and add remove tests. Removing that test fixed the hanging issue but removing some other tests could also stop hanging. The warnings you mentioned appear also in successful test runs so are unlikely to be the culprit.

The only sensible way to debug is to login to Travis, which we will have to contact Travis support ...

[BoPeng]

Per https://docs.travis-ci.com/user/running-build-in-debug-mode/, I just emailed travis tech support.

[BoPeng]

testSetVariablesTo_Output is confirmed to single-handedly trigger the hang, but the test hangs after removing this test, so there are more.

[BoPeng]

This test does not fail under windows using native python 3.6, but fails under anaconda python 3.7.

[BoPeng]

OK, it is confirmed. The failure comes from the python module pytest-cov, which is a module for pytest that check the coverage of source code. The minimal test case is

import unittest

from sos.parser import SoS_Script
from sos.workflow_executor import Base_Executor

class TestExecute(unittest.TestCase):
    def testSetVariablesTo_Output(self):
        '''Test assigning variables to _output'''
        # if there are substeps
        script = SoS_Script('''\
[10]
input: for_each=dict(i=range(2))
output: f'a_{i}.txt'
_output.touch()
''')
        wf = script.workflow()
        Base_Executor(wf).run()

if __name__ == '__main__':
    unittest.main()

Because this test uses multiprocessing, it seems that the pytest-cov adds some hooks to multiprocesses and makes sos hang.

[BoPeng]

Maybe this is related to https://pytest-cov.readthedocs.io/en/latest/mp.html.

[BoPeng]

Good, after installing pytest-cov locally, I can reproduce the hang....

[BoPeng]

Will fix after the upstream fix (pytest-dev/pytest-cov#262).