LocalPath has no attribute startswith in pytest_load_initial_conftests

[Erotemic]

Summary

In my latest version of xdoctest my dashboards are failing in pytest-cov with the error:

    @pytest.mark.tryfirst
    def pytest_load_initial_conftests(early_config, parser, args):
        options = early_config.known_args_namespace
        no_cov = options.no_cov_should_warn = False
        for arg in args:
            # arg = str(arg)
            if arg == '--no-cov':
                no_cov = True
>           elif arg.startswith('--cov') and no_cov:
E           AttributeError: 'LocalPath' object has no attribute 'startswith'

/home/joncrall/.local/conda/envs/py38/lib/python3.8/site-packages/pytest_cov/plugin.py:121: AttributeError

The traceback can be seen here:
https://app.circleci.com/pipelines/github/Erotemic/xdoctest/360/workflows/81333393-b945-4d01-9714-4039305e7dce/jobs/1856/steps

Expected vs actual result

I'm not sure if this is a pytest-cov error or not, pytest figures are incredibly hard to debug as they don't really allow for IPython embedding, and you can't make instances of them without running inside the pytest entry point. For whatever reason the args passed to the pytest-cov function:

@pytest.mark.tryfirst
def pytest_load_initial_conftests(early_config, parser, args):
    options = early_config.known_args_namespace
    no_cov = options.no_cov_should_warn = False
    for arg in args:
        if arg == '--no-cov':
            no_cov = True
        elif arg.startswith('--cov') and no_cov:
            options.no_cov_should_warn = True
            break

    if early_config.known_args_namespace.cov_source:
        plugin = CovPlugin(options, early_config.pluginmanager)
        early_config.pluginmanager.register(plugin, '_cov')

includes a non-string LocalPath object, which does not have the startswith method. If we simply force the arg to be a string by adding: arg = str(arg) as the first line of the loop everything works as expected.

Any advice on if this is a pytest-cov bug, a pytest-bug, or a bug in the way I'm using them in xdoctest would be greatly appreciated. I can't seem to find anything in xdoctest that would trigger it, but I also don't see this error report anywhere else, so my suspicion is that its something on my end. But I guess it could also be that I'm the first one to find this, so I figured it wouldn't hurt to submit a bug report.

Reproducer

This can be reproduced by

mkdir -p $HOME/tmp
cd $HOME/tmp
git clone git@github.com:Erotemic/xdoctest.git -b dev/0.13.0 $HOME/tmp/xdoctest_0_13_0
cd $HOME/tmp/xdoctest_0_13_0
pip install -r requirements.txt -U
pytest testing/test_plugin.py::TestXDoctestModuleLevel::test_collect_module_two_doctest_no_modulelevel

Versions

(py38) joncrall@Ooo:~/tmp/xdoctest_0_13_0$ pytest --version
This is pytest version 5.4.3, imported from /home/joncrall/.local/conda/envs/py38/lib/python3.8/site-packages/pytest/__init__.py
setuptools registered plugins:
  pytest-cov-2.10.0 at /home/joncrall/.local/conda/envs/py38/lib/python3.8/site-packages/pytest_cov/plugin.py
  pytest-timeout-1.3.4 at /home/joncrall/.local/conda/envs/py38/lib/python3.8/site-packages/pytest_timeout.py

[ionelmc]

Not sure whose bug is this but I'll implement a workaround since this is also a regression.

[Erotemic]

I did verify that this bug does not occur with pytest-cov 2.8.1 or 2.9.0, but does occur in 2.10.0.

I also found a MWE that reproduces the issue. I created the file foo.py in an empty directory.

class TestMWE(object):

    def test_mwe(self, testdir):
        """
        CommandLine:
            pytest -p pytester foo.py
        """
        w = testdir.maketxtfile(whatever="")
        items, reprec = testdir.inline_genitems(w)

Produces the following result:

(py38) joncrall@Ooo:~/code/tmp$ pytest -p pytester foo.py 
======================================================== test session starts =========================================================
platform linux -- Python 3.8.3, pytest-5.4.3, py-1.9.0, pluggy-0.13.1
rootdir: /home/joncrall/code/tmp
plugins: cov-2.10.0, timeout-1.4.1
collected 1 item                                                                                                                     

foo.py F                                                                                                                       [100%]

============================================================== FAILURES ==============================================================
__________________________________________________________ TestMWE.test_mwe __________________________________________________________

self = <foo.TestMWE object at 0x7fe304250400>, testdir = <Testdir local('/tmp/pytest-of-joncrall/pytest-36/test_mwe0')>

    def test_mwe(self, testdir):
        """
        CommandLine:
            pytest -p pytester foo.py
        """
        w = testdir.maketxtfile(whatever="")
>       items, reprec = testdir.inline_genitems(w)

/home/joncrall/code/tmp/foo.py:9: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/home/joncrall/.local/conda/envs/py38/lib/python3.8/site-packages/_pytest/pytester.py:848: in inline_genitems
    rec = self.inline_run("--collect-only", *args)
/home/joncrall/.local/conda/envs/py38/lib/python3.8/site-packages/_pytest/pytester.py:898: in inline_run
    ret = pytest.main(list(args), plugins=plugins)
/home/joncrall/.local/conda/envs/py38/lib/python3.8/site-packages/_pytest/config/__init__.py:105: in main
    config = _prepareconfig(args, plugins)
/home/joncrall/.local/conda/envs/py38/lib/python3.8/site-packages/_pytest/config/__init__.py:257: in _prepareconfig
    return pluginmanager.hook.pytest_cmdline_parse(
/home/joncrall/.local/conda/envs/py38/lib/python3.8/site-packages/pluggy/hooks.py:286: in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
/home/joncrall/.local/conda/envs/py38/lib/python3.8/site-packages/pluggy/manager.py:93: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
/home/joncrall/.local/conda/envs/py38/lib/python3.8/site-packages/pluggy/manager.py:84: in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
/home/joncrall/.local/conda/envs/py38/lib/python3.8/site-packages/_pytest/helpconfig.py:90: in pytest_cmdline_parse
    config = outcome.get_result()
/home/joncrall/.local/conda/envs/py38/lib/python3.8/site-packages/_pytest/config/__init__.py:836: in pytest_cmdline_parse
    self.parse(args)
/home/joncrall/.local/conda/envs/py38/lib/python3.8/site-packages/_pytest/config/__init__.py:1044: in parse
    self._preparse(args, addopts=addopts)
/home/joncrall/.local/conda/envs/py38/lib/python3.8/site-packages/_pytest/config/__init__.py:1001: in _preparse
    self.hook.pytest_load_initial_conftests(
/home/joncrall/.local/conda/envs/py38/lib/python3.8/site-packages/pluggy/hooks.py:286: in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
/home/joncrall/.local/conda/envs/py38/lib/python3.8/site-packages/pluggy/manager.py:93: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
/home/joncrall/.local/conda/envs/py38/lib/python3.8/site-packages/pluggy/manager.py:84: in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

early_config = <_pytest.config.Config object at 0x7fe30420a400>, parser = <_pytest.config.argparsing.Parser object at 0x7fe30420a460>
args = ['--collect-only', local('/tmp/pytest-of-joncrall/pytest-36/test_mwe0/whatever.txt')]

    @pytest.mark.tryfirst
    def pytest_load_initial_conftests(early_config, parser, args):
        options = early_config.known_args_namespace
        no_cov = options.no_cov_should_warn = False
        for arg in args:
            if arg == '--no-cov':
                no_cov = True
>           elif arg.startswith('--cov') and no_cov:
E           AttributeError: 'LocalPath' object has no attribute 'startswith'

/home/joncrall/.local/conda/envs/py38/lib/python3.8/site-packages/pytest_cov/plugin.py:120: AttributeError
====================================================== short test summary info =======================================================
FAILED foo.py::TestMWE::test_mwe - AttributeError: 'LocalPath' object has no attribute 'startswith'
========================================================= 1 failed in 0.27s ==========================================================

Note this does seem to be caused by an interaction with the "testdir" fixture (god, I hate fixtures) from the builtin pytester plugin. It seems like they switched to using pathlib.Path objects, which is not handled correctly by pytest-cov, so it is a pytest-cov bug.