collect doctests from C/Cython implemented functions

[ev-br]

I'm trying to use pytest doctesting of functions/modules implemented in C (or Cython if that matters). Collection fails with

$ pytest --pyargs scipy.interpolate.interpnd -v --doctest-modules --collect-only
================================= test session starts ==================================
platform linux -- Python 3.10.10, pytest-8.0.0.dev257+g8062743f6, pluggy-1.3.0 -- /home/br/mambaforge/envs/scipy-dev/bin/python3.10
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/br/sweethome/.hypothesis/examples')
rootdir: /home/br/sweethome
plugins: cov-4.0.0, doctestplus-1.0.0, timeout-2.1.0, hypothesis-6.84.3, xdist-3.2.1
collecting ... 
collected 0 items                                                              

<snip>

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================= no tests collected in 2.86s ==========================
ERROR: found no collectors for /home/br/repos/scipy/scipy/build-install/lib/python3.10/site-packages/scipy/interpolate/interpnd.cpython-310-x86_64-linux-gnu.so

So the module itself is resolved, but the doctests are not found. The errors seems to come from https://github.com/pytest-dev/pytest/blob/main/src/_pytest/main.py#L672

Doctests exist and are found by the standard library doctest:

In [1]: import scipy.interpolate.interpnd as interpnd

In [2]: import doctest

In [3]: doctest.testmod(interpnd)

<snip a bunch of expected doctest failures>

2 items had failures:
   5 of  18 in scipy.interpolate.interpnd.CloughTocher2DInterpolator
   5 of  18 in scipy.interpolate.interpnd.LinearNDInterpolator
***Test Failed*** 10 failures.
Out[3]: TestResults(failed=10, attempted=36)

So I'd appreciate any pointers to how to fix this; if there's some magic config, great; if not we can try implementing what needs implementing.

[nicoddemus]

This is being discussed in #11388.

[ev-br]

Thanks, my google-fu failed me on this one. Will test and comment over the original issue then.

[ev-br]

Actually, there is one other part of the issue, which makes it not a complete duplicate.

My use case is collecting doctests. The direct issue in the OP, ERROR: found no collectors for ....so --- notice the .so suffix --- is due to the compiled modules being skipped in pytest_collect_file:

https://github.com/pytest-dev/pytest/blob/main/src/_pytest/doctest.py#L131

Changing if file_path.suffix == ".py" into (an exploratory hack, I know) if file_path.suffix in (".py", ".so)" fails in the DoctestModule invoking the importlib machinery:

_________ ERROR collecting build-install/lib/python3.10/site-packages/scipy/interpolate/dfitpack.cpython-310-x86_64-linux-gnu.so __________
/home/br/mambaforge/envs/scipy-dev/lib/python3.10/site-packages/_pytest/runner.py:341: in from_call
    result: Optional[TResult] = func()
        cls        = <class '_pytest.runner.CallInfo'>
        duration   = 0.00018689199350774288
        excinfo    = <ExceptionInfo ModuleNotFoundError("No module named 'scipy.interpolate.dfitpack.cpython-310-x86_64-linux-gnu'; 'scipy.interpolate.dfitpack' is not a package") tblen=8>
        func       = <function pytest_make_collect_report.<locals>.<lambda> at 0x7f6556087370>
        precise_start = 689112.781180531
        precise_stop = 689112.781367423
        reraise    = None
        result     = None
        start      = 1695115283.4392905
        stop       = 1695115283.4394786
        when       = 'collect'

The file exists and can be imported, so I suspect something in pytest makes some assumptions about only dealing with pure python modules.

[nicoddemus]

Oh I see, but still I think it makes sense to centralize the discussion in the same location, as they are closely related.