Emergent failure (regression) "no module named posixpath" #1998

jaraco · 2020-02-15T15:13:35Z

In #1997, I discovered emergent test failures in the maint/44.x branch on Python 2.7. There are three failures where invoking python in a subprocess results "no module named posixpath".

I suspect a new release of virtualenv is implicated.

The text was updated successfully, but these errors were encountered:

jaraco · 2020-02-15T15:21:58Z

I've been able to replicate the issue locally by reinstalling my tox environment from scratch.

jaraco · 2020-02-15T15:40:43Z

Confirmed, virtualenv 20 is implicated.

virtualenv 16

setuptools maint/44.x $ ~/.local/pipx/venvs/tox/bin/python -m pip install 'virtualenv<20'
Collecting virtualenv<20
  Using cached virtualenv-16.7.9-py2.py3-none-any.whl (3.4 MB)
Installing collected packages: virtualenv
  Attempting uninstall: virtualenv
    Found existing installation: virtualenv 20.0.4
    Uninstalling virtualenv-20.0.4:
      Successfully uninstalled virtualenv-20.0.4
Successfully installed virtualenv-16.7.9

setuptools maint/44.x $ tox -r -e py27 -- -k test_develop
py27 recreate: /Users/jaraco/code/public/pypa/setuptools/.tox/py27
py27: pip_version is pip
py27 installdeps: -r/Users/jaraco/code/public/pypa/setuptools/tests/requirements.txt
py27 develop-inst: /Users/jaraco/code/public/pypa/setuptools
py27 installed: DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support,apipkg==1.5,appdirs==1.4.3,atomicwrites==1.3.0,attrs==19.3.0,configparser==4.0.2,contextlib2==0.6.0.post1,coverage==5.0.3,distlib==0.3.0,entrypoints==0.3,enum34==1.1.6,execnet==1.7.1,filelock==3.0.12,flake8==3.7.9,funcsigs==1.0.2,functools32==3.2.3.post2,futures==3.3.0,importlib-metadata==1.5.0,importlib-resources==1.0.2,mccabe==0.6.1,mock==3.0.5,more-itertools==5.0.0,packaging==20.1,path.py==11.5.2,pathlib2==2.3.5,pip==20.0.2,pluggy==0.13.1,py==1.8.1,pycodestyle==2.5.0,pyflakes==2.1.1,pyparsing==2.4.6,pytest==4.6.9,pytest-cov==2.8.1,pytest-fixture-config==1.7.0,pytest-flake8==1.0.4,pytest-shutil==1.7.0,pytest-virtualenv==1.7.0,scandir==1.10.0,-e git+gh://pypa/setuptools@0559f17d4588553eb428d57ecfebab9e0e988e65#egg=setuptools,six==1.14.0,termcolor==1.1.0,typing==3.7.4.1,virtualenv==20.0.4,wcwidth==0.1.8,wheel==0.33.6,zipp==1.1.0
py27 run-test-pre: PYTHONHASHSEED='619171264'
py27 run-test: commands[0] | pytest --cov-config=/Users/jaraco/code/public/pypa/setuptools/tox.ini --cov-report= -k test_develop
========================================================================== test session starts ===========================================================================
platform darwin -- Python 2.7.16, pytest-4.6.9, py-1.8.1, pluggy-0.13.1
cachedir: .tox/py27/.pytest_cache
rootdir: /Users/jaraco/code/public/pypa/setuptools, inifile: pytest.ini
plugins: shutil-1.7.0, virtualenv-1.7.0, flake8-1.0.4, cov-2.8.1
collected 715 items / 709 deselected / 2 skipped / 4 selected                                                                                                            

setuptools/tests/test_develop.py ss....                                                                                                                            [100%]

======================================================================== short test summary info =========================================================================
SKIPPED [1] setuptools/tests/test_develop.py:65: Cannot run when invoked in a virtualenv or venv
SKIPPED [2] /Users/jaraco/code/public/pypa/setuptools/setuptools/tests/test_msvc.py:18: could not import 'distutils.msvc9compiler': No module named _winreg
SKIPPED [1] /Users/jaraco/code/public/pypa/setuptools/setuptools/tests/test_develop.py:108: TODO: needs a fixture to cause 'develop' to be invoked without mutating environment.
========================================================== 4 passed, 4 skipped, 709 deselected in 2.43 seconds ===========================================================
________________________________________________________________________________ summary _________________________________________________________________________________
  py27: commands succeeded
  congratulations :)

virtualenv 20

setuptools maint/44.x $ ~/.local/pipx/venvs/tox/bin/python -m pip install -U 'virtualenv'
Collecting virtualenv
  Using cached virtualenv-20.0.4-py2.py3-none-any.whl (4.6 MB)
Requirement already satisfied, skipping upgrade: distlib<1,>=0.3.0 in /Users/jaraco/.local/pipx/venvs/tox/lib/python3.8/site-packages (from virtualenv) (0.3.0)
Requirement already satisfied, skipping upgrade: filelock<4,>=3.0.0 in /Users/jaraco/.local/pipx/venvs/tox/lib/python3.8/site-packages (from virtualenv) (3.0.12)
Requirement already satisfied, skipping upgrade: six<2,>=1.9.0 in /Users/jaraco/.local/pipx/venvs/tox/lib/python3.8/site-packages (from virtualenv) (1.14.0)
Requirement already satisfied, skipping upgrade: appdirs<2,>=1.4.3 in /Users/jaraco/.local/pipx/venvs/tox/lib/python3.8/site-packages (from virtualenv) (1.4.3)
Installing collected packages: virtualenv
  Attempting uninstall: virtualenv
    Found existing installation: virtualenv 16.7.9
    Uninstalling virtualenv-16.7.9:
      Successfully uninstalled virtualenv-16.7.9
Successfully installed virtualenv-20.0.4
setuptools maint/44.x $ tox -r -e py27 -- -k test_develop
py27 recreate: /Users/jaraco/code/public/pypa/setuptools/.tox/py27
py27: pip_version is pip
py27 installdeps: -r/Users/jaraco/code/public/pypa/setuptools/tests/requirements.txt
py27 develop-inst: /Users/jaraco/code/public/pypa/setuptools
py27 installed: DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support,apipkg==1.5,appdirs==1.4.3,atomicwrites==1.3.0,attrs==19.3.0,configparser==4.0.2,contextlib2==0.6.0.post1,coverage==5.0.3,distlib==0.3.0,entrypoints==0.3,enum34==1.1.6,execnet==1.7.1,filelock==3.0.12,flake8==3.7.9,funcsigs==1.0.2,functools32==3.2.3.post2,futures==3.3.0,importlib-metadata==1.5.0,importlib-resources==1.0.2,mccabe==0.6.1,mock==3.0.5,more-itertools==5.0.0,packaging==20.1,path.py==11.5.2,pathlib2==2.3.5,pip==20.0.2,pluggy==0.13.1,py==1.8.1,pycodestyle==2.5.0,pyflakes==2.1.1,pyparsing==2.4.6,pytest==4.6.9,pytest-cov==2.8.1,pytest-fixture-config==1.7.0,pytest-flake8==1.0.4,pytest-shutil==1.7.0,pytest-virtualenv==1.7.0,scandir==1.10.0,-e git+gh://pypa/setuptools@0559f17d4588553eb428d57ecfebab9e0e988e65#egg=setuptools,six==1.14.0,termcolor==1.1.0,typing==3.7.4.1,virtualenv==20.0.4,wcwidth==0.1.8,wheel==0.34.2,zipp==1.1.0
py27 run-test-pre: PYTHONHASHSEED='2159178289'
py27 run-test: commands[0] | pytest --cov-config=/Users/jaraco/code/public/pypa/setuptools/tox.ini --cov-report= -k test_develop
========================================================================== test session starts ===========================================================================
platform darwin -- Python 2.7.16, pytest-4.6.9, py-1.8.1, pluggy-0.13.1
cachedir: .tox/py27/.pytest_cache
rootdir: /Users/jaraco/code/public/pypa/setuptools, inifile: pytest.ini
plugins: shutil-1.7.0, virtualenv-1.7.0, flake8-1.0.4, cov-2.8.1
collected 715 items / 709 deselected / 2 skipped / 4 selected                                                                                                            

setuptools/tests/test_develop.py ss...F                                                                                                                            [100%]

================================================================================ FAILURES ================================================================================
____________________________________________________________ TestNamespaces.test_namespace_package_importable ____________________________________________________________

self = <setuptools.tests.test_develop.TestNamespaces instance at 0x111132248>
tmpdir = local('/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-10/test_namespace_package_importa0')

    @pytest.mark.skipif(
        bool(os.environ.get("APPVEYOR")),
        reason="https://github.com/pypa/setuptools/issues/851",
    )
    @pytest.mark.skipif(
        platform.python_implementation() == 'PyPy' and not six.PY2,
        reason="https://github.com/pypa/setuptools/issues/1202",
    )
    def test_namespace_package_importable(self, tmpdir):
        """
        Installing two packages sharing the same namespace, one installed
        naturally using pip or `--single-version-externally-managed`
        and the other installed using `develop` should leave the namespace
        in tact and both packages reachable by import.
        """
        pkg_A = namespaces.build_namespace_package(tmpdir, 'myns.pkgA')
        pkg_B = namespaces.build_namespace_package(tmpdir, 'myns.pkgB')
        target = tmpdir / 'packages'
        # use pip to install to the target directory
        install_cmd = [
            sys.executable,
            '-m',
            'pip',
            'install',
            str(pkg_A),
            '-t', str(target),
        ]
        subprocess.check_call(install_cmd)
        self.install_develop(pkg_B, target)
        namespaces.make_site_dir(target)
        try_import = [
            sys.executable,
            '-c', 'import myns.pkgA; import myns.pkgB',
        ]
        with test.test.paths_on_pythonpath([str(target)]):
>           subprocess.check_call(try_import)

setuptools/tests/test_develop.py:194: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

popenargs = (['/Users/jaraco/code/public/pypa/setuptools/.tox/py27/bin/python', '-c', 'import myns.pkgA; import myns.pkgB'],), kwargs = {}, retcode = 1
cmd = ['/Users/jaraco/code/public/pypa/setuptools/.tox/py27/bin/python', '-c', 'import myns.pkgA; import myns.pkgB']

    def check_call(*popenargs, **kwargs):
        """Run command with arguments.  Wait for command to complete.  If
        the exit code was zero then return, otherwise raise
        CalledProcessError.  The CalledProcessError object will have the
        return code in the returncode attribute.
    
        The arguments are the same as for the Popen constructor.  Example:
    
        check_call(["ls", "-l"])
        """
        retcode = call(*popenargs, **kwargs)
        if retcode:
            cmd = kwargs.get("args")
            if cmd is None:
                cmd = popenargs[0]
>           raise CalledProcessError(retcode, cmd)
E           CalledProcessError: Command '['/Users/jaraco/code/public/pypa/setuptools/.tox/py27/bin/python', u'-c', u'import myns.pkgA; import myns.pkgB']' returned non-zero exit status 1

/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py:190: CalledProcessError
-------------------------------------------------------------------------- Captured stdout call --------------------------------------------------------------------------
Processing /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-10/test_namespace_package_importa0/myns.pkgA
Building wheels for collected packages: myns.pkgA
  Building wheel for myns.pkgA (setup.py): started
  Building wheel for myns.pkgA (setup.py): finished with status 'done'
  Created wheel for myns.pkgA: filename=myns.pkgA-1.0-py2-none-any.whl size=1810 sha256=9f0432a755fc08f7f253e0d9070bfd0eab3731e519a616d623f4fcc29b79d0c3
  Stored in directory: /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-ephem-wheel-cache-r7MhMJ/wheels/ee/49/59/355121a21016aef70753808c5bfae27d2f6ca77c179b1a2811
Successfully built myns.pkgA
Installing collected packages: myns.pkgA
Successfully installed myns.pkgA-1.0
running develop
running egg_info
creating myns.pkgB.egg-info
writing namespace_packages to myns.pkgB.egg-info/namespace_packages.txt
writing dependency_links to myns.pkgB.egg-info/dependency_links.txt
writing myns.pkgB.egg-info/PKG-INFO
writing top-level names to myns.pkgB.egg-info/top_level.txt
writing manifest file 'myns.pkgB.egg-info/SOURCES.txt'
reading manifest file 'myns.pkgB.egg-info/SOURCES.txt'
writing manifest file 'myns.pkgB.egg-info/SOURCES.txt'
running build_ext
Creating /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-10/test_namespace_package_importa0/packages/site.py
Installing /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-10/test_namespace_package_importa0/packages/myns.pkgB-nspkg.pth
Creating /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-10/test_namespace_package_importa0/packages/myns.pkgB.egg-link (link to .)
Adding myns.pkgB 1.0 to easy-install.pth file

Installed /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-10/test_namespace_package_importa0/myns.pkgB
Processing dependencies for myns.pkgB==1.0
Finished processing dependencies for myns.pkgB==1.0
-------------------------------------------------------------------------- Captured stderr call --------------------------------------------------------------------------
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Traceback (most recent call last):
  File "/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-10/test_namespace_package_importa0/packages/site.py", line 73, in <module>
    __boot()
  File "/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-10/test_namespace_package_importa0/packages/site.py", line 3, in __boot
    import os
  File "/Users/jaraco/code/public/pypa/setuptools/.tox/py27/lib/python2.7/os.py", line 49, in <module>
    import posixpath as path
ImportError: No module named posixpath
======================================================================== short test summary info =========================================================================
SKIPPED [2] /Users/jaraco/code/public/pypa/setuptools/setuptools/tests/test_msvc.py:18: could not import 'distutils.msvc9compiler': No module named _winreg
SKIPPED [1] /Users/jaraco/code/public/pypa/setuptools/setuptools/tests/test_develop.py:108: TODO: needs a fixture to cause 'develop' to be invoked without mutating environment.
SKIPPED [1] setuptools/tests/test_develop.py:65: Cannot run when invoked in a virtualenv or venv
===================================================== 1 failed, 3 passed, 4 skipped, 709 deselected in 2.41 seconds ======================================================
ERROR: InvocationError for command /Users/jaraco/code/public/pypa/setuptools/.tox/py27/bin/pytest --cov-config=/Users/jaraco/code/public/pypa/setuptools/tox.ini --cov-report= -k test_develop (exited with code 1)
________________________________________________________________________________ summary _________________________________________________________________________________
ERROR:   py27: commands failed

Pin virtualenv to <20 as workaround for #1998.

jaraco · 2020-02-15T17:19:04Z

Fixed in #1999.

Summary: As per pypa/setuptools#1998, we need a virtualenv<20 for our ybops setup.py package. This is part of getting releases working through Jenkins Test Plan: testing from Jenkins slave with virtualenv v20 Reviewers: mikhail, ram, wesley Reviewed By: wesley Subscribers: jenkins-bot, ybase Differential Revision: https://phabricator.dev.yugabyte.com/D8185

jaraco mentioned this issue Feb 15, 2020

Avoid installing setuptools 45 on Python 2. #1997

Merged

jaraco mentioned this issue Feb 15, 2020

"no module named posixpath" with virtualenv 20 pypa/virtualenv#1638

Closed

jaraco added a commit that referenced this issue Feb 15, 2020

Pin virtualenv to <20 as workaround for #1998.

a99aeb8

jaraco mentioned this issue Feb 15, 2020

Pin virtualenv to <20 as workaround for #1998. #1999

Merged

jaraco added a commit that referenced this issue Feb 15, 2020

Merge pull request #1999 from pypa/bugfix/1998-virtualenv-site-conflict

35d0527

Pin virtualenv to <20 as workaround for #1998.

jaraco closed this as completed Feb 15, 2020

jaraco mentioned this issue May 29, 2020

Remove site file usage #2165

Closed

jaraco added a commit that referenced this issue May 29, 2020

The workaround is only needed for Python 2. Ref #1998.

74de4e9

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Emergent failure (regression) "no module named posixpath" #1998

Emergent failure (regression) "no module named posixpath" #1998

jaraco commented Feb 15, 2020

jaraco commented Feb 15, 2020

jaraco commented Feb 15, 2020 •

edited

jaraco commented Feb 15, 2020

Emergent failure (regression) "no module named posixpath" #1998

Emergent failure (regression) "no module named posixpath" #1998

Comments

jaraco commented Feb 15, 2020

jaraco commented Feb 15, 2020

jaraco commented Feb 15, 2020 • edited

virtualenv 16

virtualenv 20

jaraco commented Feb 15, 2020

jaraco commented Feb 15, 2020 •

edited