tox trying to import packages outside of virtual environment

[dstansby]

I'm trying to test a package with compiled cython .pyx files. tox installs the package fine into a virtual environment, and I've verified that the correct files are present, including the compiled .so files generated from the .pyx files. When tox runs the tests it is trying to import files from the "local" copy of the package, not the one installed in the virtual environment, so the tests fail because there are no compiled .so files in the local copy of the package. I assume this is a bug, because the test command shouldn't have access to packages outside the virtual environment?

The full tox output is at https://pipelines.actions.githubusercontent.com/serviceHosts/0f5268b6-cca7-4d58-811c-161d02df8dd8/_apis/pipelines/1/runs/239/signedlogcontent/9?urlExpires=2022-04-07T18%3A17%3A04.9727616Z&urlSigningMethod=HMACV1&urlSignature=QD%2BhF9Tq9%2Fh2J33eP0LmJ5XlMDzIWf7WG2Pl6lBxY4Y%3D, and the code is at https://github.com/dstansby/cellfinder-core/tree/composite-actions. Sorry it's so long, I tried making a small package to reproduce this but couldn't get it to error 😞 . If there's anything else I can do to debug (I can reproduce locally) please let me know.

[jugmac00]

Hey David,

There are a couple of great articles out there which describe in which problem you ran, but in a nutshell...

Your package is called cellfinder-core and your directory is called cellfiner-core and when then your code imports from cellfinder-core ... well ... Python imports it from the directory, not the installed package.

This is not a tox issue.

The issue can be easily avoided when you put your source code in a src directory, see this very package ( https://github.com/tox-dev/tox ). Unfortunately using a src is a common pattern, but not a standard, so many people run in these kind of problems.

Here are a couple of articles which describe your problem way better than I could - and show a couple of solutions - if you do not like to use a src directory - which you should :-)

https://blog.ganssle.io/articles/2019/08/test-as-installed.html
https://web.archive.org/web/20170615032800/https://enotuniq.org/
https://hynek.me/articles/testing-packaging/

[dstansby]

That makes sense, thanks a lot for the quick response! It would be super nice if tox could work around situations like this to make sure it only imports from the virtual environment, but I imagine this is an obvious enough request to be either tricky to do or not possible 😄

[dstansby]

🎉 putting it in src worked a treat