Requirements do not install, wheel missing

[cliffckerr]

For some reason (I think related to installing from the .tar.gz instead of the .whl), pip isn't picking up the requirements in line_profiler 3.2, causing GitHub Actions to fail. Compare the logs for 3.1 installation:

Run pip install line_profiler==3.1
Collecting line_profiler==3.1
  Downloading line_profiler-3.1.0-cp38-cp38-manylinux2010_x86_64.whl (65 kB)
Collecting IPython
  Downloading ipython-7.22.0-py3-none-any.whl (785 kB)
Collecting decorator
  Downloading decorator-4.4.2-py2.py3-none-any.whl (9.2 kB)
Collecting traitlets>=4.2
  Downloading traitlets-5.0.5-py3-none-any.whl (100 kB)
Collecting pickleshare
  Downloading pickleshare-0.7.5-py2.py3-none-any.whl (6.9 kB)
[...]

with the logs for 3.2:

Run pip install line_profiler
Collecting line_profiler
  Downloading line_profiler-3.2.0.tar.gz (17 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
    Preparing wheel metadata: started
    Preparing wheel metadata: finished with status 'done'
Building wheels for collected packages: line-profiler

I can also reproduce this locally in a clean Conda environment on Ubuntu 20.04:

finzi:~> conda create -n lp3.1 python=3.8
finzi:~> conda activate lp3.1
finzi:~> pip install line_profiler==3.1
Collecting line_profiler==3.1
  Using cached line_profiler-3.1.0-cp38-cp38-manylinux2010_x86_64.whl (65 kB)
Collecting IPython
  Downloading ipython-7.22.0-py3-none-any.whl (785 kB)
     |████████████████████████████████| 785 kB 2.3 MB/s 
Collecting decorator
  Using cached decorator-4.4.2-py2.py3-none-any.whl (9.2 kB)
Collecting pexpect>4.3
  Using cached pexpect-4.8.0-py2.py3-none-any.whl (59 kB)
Collecting pickleshare
  Using cached pickleshare-0.7.5-py2.py3-none-any.whl (6.9 kB)
Collecting prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0
  Downloading prompt_toolkit-3.0.18-py3-none-any.whl (367 kB)
     |████████████████████████████████| 367 kB 19.6 MB/s 
Collecting backcall
  Using cached backcall-0.2.0-py2.py3-none-any.whl (11 kB)
Collecting traitlets>=4.2
  Using cached traitlets-5.0.5-py3-none-any.whl (100 kB)
Requirement already satisfied: setuptools>=18.5 in /software/anaconda3/envs/tmpb2/lib/python3.8/site-packages (from IPython->line_profiler==3.1) (52.0.0.post20210125)
Collecting jedi>=0.16
  Using cached jedi-0.18.0-py2.py3-none-any.whl (1.4 MB)
Collecting pygments
  Using cached Pygments-2.8.1-py3-none-any.whl (983 kB)
Collecting parso<0.9.0,>=0.8.0
  Using cached parso-0.8.1-py2.py3-none-any.whl (93 kB)
Collecting ptyprocess>=0.5
  Using cached ptyprocess-0.7.0-py2.py3-none-any.whl (13 kB)
Collecting wcwidth
  Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting ipython-genutils
  Using cached ipython_genutils-0.2.0-py2.py3-none-any.whl (26 kB)
Installing collected packages: wcwidth, ptyprocess, parso, ipython-genutils, traitlets, pygments, prompt-toolkit, pickleshare, pexpect, jedi, decorator, backcall, IPython, line-profiler
Successfully installed IPython-7.22.0 backcall-0.2.0 decorator-4.4.2 ipython-genutils-0.2.0 jedi-0.18.0 line-profiler-3.1.0 parso-0.8.1 pexpect-4.8.0 pickleshare-0.7.5 prompt-toolkit-3.0.18 ptyprocess-0.7.0 pygments-2.8.1 traitlets-5.0.5 wcwidth-0.2.5

vs.

finzi:~> conda create -n lp3.2 python=3.8
finzi:~> conda activate lp3.2
finzi:~> pip install line_profiler==3.2
Collecting line_profiler==3.2
  Using cached line_profiler-3.2.0-py3-none-any.whl
Installing collected packages: line-profiler
Successfully installed line-profiler-3.2.0

finzi:~> python
Python 3.8.8 (default, Feb 24 2021, 21:46:12) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import line_profiler
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/software/anaconda3/envs/tmpbuild/lib/python3.8/site-packages/line_profiler/__init__.py", line 10, in <module>
    from .line_profiler import __version__
  File "/software/anaconda3/envs/tmpbuild/lib/python3.8/site-packages/line_profiler/line_profiler.py", line 23, in <module>
    from IPython.core.magic import (Magics, magics_class, line_magic)
ModuleNotFoundError: No module named 'IPython'

I can't see any obvious changes in requirements.txt or setup.py that would've caused this, so guessing it has something to do with the wheel uploaded to pypi.

[cliffckerr]

Incidentally, the initial error I saw (cmake) pointed to this issue, so I tried upgrading pip from 20.0.2 to 21.0.1, but that didn't help.

[cliffckerr]

Update: I see that there is no wheel uploaded to pypi, so that's probably the issue!

[Erotemic]

Yup, 3.2 hasn't been released yet. I am upgrading the release structure so hopefully it will be on pypi by the end of the day.

[cliffckerr]

Thanks...weird that pypi thinks it was released yesterday!
https://pypi.org/project/line-profiler/

[cliffckerr]

Fixed now, thanks @Erotemic !

[cliffckerr]

@Erotemic Or...spoke too soon. 3.2.0 is still there (or reappeared?!), and GitHub builds are still failing:
https://pypi.org/simple/line-profiler/

[cliffckerr]

...But now is resolved it seems!

[cliffckerr]

Okay, finally figured out what's going on. pip install -e . works:
https://github.com/sciris/sciris/runs/2240326821

Collecting line_profiler
  Downloading line_profiler-3.1.0-cp38-cp38-manylinux2010_x86_64.whl (65 kB)

But python setup.py develop does not work:
https://github.com/sciris/sciris/runs/2240341541

Searching for line_profiler
Reading https://pypi.org/simple/line_profiler/
Downloading https://files.pythonhosted.org/packages/80/fc/87235848376bf59686364208269423fd523f979deffd6b73085629d6fdbf/line_profiler-3.2.0.tar.gz#sha256=113a17d4023eff3008c807d895dec11f9ad872986edd0816fa523ecdeffe7bf8
Best match: line-profiler 3.2.0
Processing line_profiler-3.2.0.tar.gz
Writing /tmp/easy_install-fqnt9tru/line_profiler-3.2.0/setup.cfg
Running line_profiler-3.2.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-fqnt9tru/line_profiler-3.2.0/egg-dist-tmp-y_yddgpi
/opt/hostedtoolcache/Python/3.8.8/x64/lib/python3.8/site-packages/memory_profiler-0.58.0-py3.8.egg/mprof.py:76: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if args is "all":
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.8/x64/lib/python3.8/site-packages/setuptools/sandbox.py", line 154, in save_modules
    yield saved
  File "/opt/hostedtoolcache/Python/3.8.8/x64/lib/python3.8/site-packages/setuptools/sandbox.py", line 195, in setup_context
    yield
  File "/opt/hostedtoolcache/Python/3.8.8/x64/lib/python3.8/site-packages/setuptools/sandbox.py", line 250, in run_setup
    _execfile(setup_script, ns)
  File "/opt/hostedtoolcache/Python/3.8.8/x64/lib/python3.8/site-packages/setuptools/sandbox.py", line 45, in _execfile
    exec(code, globals, locals)
  File "/tmp/easy_install-fqnt9tru/line_profiler-3.2.0/setup.py", line 215, in <module>
ModuleNotFoundError: No module named 'skbuild'

The issue is that line_profiler-3.2.0.tar.gz is still in https://pypi.org/simple/line-profiler/, which is what setup.py seems to be using (I can't find any documentation on why the latter but not the former uses pypi.org/simple).

Obviously, one solution is to just ensure that it's pip install everywhere and not python setup.py. But would be nice if it worked for both, either by removing the offending tar.gz file or by releasing the 3.2 wheel. cc @Erotemic @mikekeda

[Erotemic]

@cliffckerr yanking a release just means pypi won't default to it. You can still hard require it (which I don't recommend as it doesn't have any wheels!)

The reason why pip works and setup.py doesn't is because of the pyproject.toml file. That specifies scikit-build as a pre-requirement to calling setup.py. Pip recognizes this, but obviously setup.py can't if can't be run because its dependencies aren't met.

I'm working on releasing wheels for 3.2.1, but I'm running into a security issue that I don't know if I can solve given the capabilities I have with CircleCI. I might have to try moving to github actions or just manually publishing the builds on my local machine (and the thought of publishing something "that worked on my machine" scares me to no end, so I'd rather get the CI secure first).

[cliffckerr]

Thanks @Erotemic -- not sure my CI/build chops are up to the task but happy to try to help if I can! (Likewise for the windows build issue I opened -- but I'm not a Windows user so that's even more of a stretch :) )

For the record, setup.py is still pulling version 3.2 automatically, even if a version isn't specified. I think the issue might be if the release was just yanked, rather than deleted -- as I mentioned, the 3.2 tar.gz file is still on pypi, which is what's getting picked up by setup.py. On the pypi, there should be the option to manually delete this source file, e.g. for a project I manage:

[Erotemic]

This problem should now be resolved as 3.2.1 seems to have been released correctly.