Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] Cython 0.29.31 breaks building SciPy 1.8.1 #4928

Closed
oscarbenjamin opened this issue Jul 28, 2022 · 5 comments
Closed

[BUG] Cython 0.29.31 breaks building SciPy 1.8.1 #4928

oscarbenjamin opened this issue Jul 28, 2022 · 5 comments
Labels
defect Error Reporting
Milestone
2.29.32

Comments

@oscarbenjamin
Copy link

To Reproduce
Code to reproduce the behaviour:

$ pip install -U pip setuptools wheel cython numpy
Collecting pip
  Downloading pip-22.2.1-py3-none-any.whl (2.0 MB)
     |████████████████████████████████| 2.0 MB 4.8 MB/s 
Collecting setuptools
  Downloading setuptools-63.2.0-py3-none-any.whl (1.2 MB)
     |████████████████████████████████| 1.2 MB 28.2 MB/s 
Collecting wheel
  Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)
Collecting cython
  Downloading Cython-0.29.31-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (1.9 MB)
     |████████████████████████████████| 1.9 MB 2.8 MB/s 
Collecting numpy
  Using cached numpy-1.23.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.1 MB)
Installing collected packages: pip, setuptools, wheel, cython, numpy
  Attempting uninstall: pip
    Found existing installation: pip 20.2.3
    Uninstalling pip-20.2.3:
      Successfully uninstalled pip-20.2.3
  Attempting uninstall: setuptools
    Found existing installation: setuptools 49.2.1
    Uninstalling setuptools-49.2.1:
      Successfully uninstalled setuptools-49.2.1
Successfully installed cython-0.29.31 numpy-1.23.1 pip-22.2.1 setuptools-63.2.0 wheel-0.37.1
$ pip install --no-binary scipy scipy
Collecting scipy
  Downloading scipy-1.8.1.tar.gz (38.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 38.2/38.2 MB 8.0 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [102 lines of output]
      Running scipy/special/_generate_pyx.py
      Running scipy/stats/_generate_pyx.py
      Running scipy/linalg/_generate_pyx.py
      Processing scipy/special/_ufuncs.pyx
      Processing scipy/special/_test_round.pyx
      Processing scipy/special/_ufuncs_cxx.pyx
      Processing scipy/special/_ellip_harm_2.pyx
      Processing scipy/special/cython_special.pyx
      Processing scipy/special/_comb.pyx
      Processing scipy/spatial/_voronoi.pyx
      Processing scipy/spatial/_hausdorff.pyx
      Processing scipy/spatial/_qhull.pyx
      Processing scipy/spatial/_ckdtree.pyx
      Processing scipy/spatial/transform/_rotation.pyx
      Processing scipy/interpolate/_ppoly.pyx
      Processing scipy/interpolate/_bspl.pyx
      Processing scipy/interpolate/interpnd.pyx
      Processing scipy/cluster/_vq.pyx
      Processing scipy/cluster/_hierarchy.pyx
      Processing scipy/cluster/_optimal_leaf_ordering.pyx
      Processing scipy/_lib/messagestream.pyx
      Processing scipy/_lib/_test_deprecation_call.pyx
      Processing scipy/_lib/_test_deprecation_def.pyx
      Processing scipy/_lib/_ccallback_c.pyx
      Processing scipy/signal/_upfirdn_apply.pyx
      Processing scipy/signal/_max_len_seq_inner.pyx
      Processing scipy/signal/_sosfilt.pyx
      Processing scipy/signal/_spectral.pyx
      Processing scipy/signal/_peak_finding_utils.pyx
      Processing scipy/io/matlab/_streams.pyx
      Processing scipy/io/matlab/_mio5_utils.pyx
      Processing scipy/io/matlab/_mio_utils.pyx
      Processing scipy/stats/_stats.pyx
      Processing scipy/stats/_qmc_cy.pyx
      Processing scipy/stats/_biasedurn.pyx
      Processing scipy/stats/_sobol.pyx
      Processing scipy/stats/_boost/src/nbinom_ufunc.pyx
      Processing scipy/stats/_boost/src/beta_ufunc.pyx
      
      Error compiling Cython file:
      ------------------------------------------------------------
      ...
          return np.array(result, dtype=np.int64)
      
      
      @cython.wraparound(False)
      @cython.boundscheck(False)
      def _weightedrankedtau(ordered[:] x, ordered[:] y, intp_t[:] rank, weigher, bool additive):
      ^
      ------------------------------------------------------------
      
      _stats.pyx:173:0: Referring to a memoryview typed argument directly in a nested closure function is not supported in Cython 0.x. Either upgrade to Cython 3, or assign the argument to a local variable and use that in the nested function.
      Processing scipy/stats/_boost/src/binom_ufunc.pyx
      Traceback (most recent call last):
        File "/tmp/pip-install-wnazhy0r/scipy_3ee8a876e4f54aa7bfa3d9fdba592754/tools/cythonize.py", line 317, in <module>
          main()
        File "/tmp/pip-install-wnazhy0r/scipy_3ee8a876e4f54aa7bfa3d9fdba592754/tools/cythonize.py", line 313, in main
          find_process_files(root_dir)
        File "/tmp/pip-install-wnazhy0r/scipy_3ee8a876e4f54aa7bfa3d9fdba592754/tools/cythonize.py", line 302, in find_process_files
          for result in pool.imap_unordered(lambda args: process(*args), jobs):
        File "/usr/lib/python3.8/multiprocessing/pool.py", line 868, in next
          raise value
        File "/usr/lib/python3.8/multiprocessing/pool.py", line 125, in worker
          result = (True, func(*args, **kwds))
        File "/tmp/pip-install-wnazhy0r/scipy_3ee8a876e4f54aa7bfa3d9fdba592754/tools/cythonize.py", line 302, in <lambda>
          for result in pool.imap_unordered(lambda args: process(*args), jobs):
        File "/tmp/pip-install-wnazhy0r/scipy_3ee8a876e4f54aa7bfa3d9fdba592754/tools/cythonize.py", line 236, in process
          processor_function(fromfile, tofile, cwd=path)
        File "/tmp/pip-install-wnazhy0r/scipy_3ee8a876e4f54aa7bfa3d9fdba592754/tools/cythonize.py", line 102, in process_pyx
          raise Exception('Cython failed')
      Exception: Cython failed
      setup.py:486: UserWarning: Unrecognized setuptools command ('dist_info --egg-base /tmp/pip-modern-metadata-tcs7khbn'), proceeding with generating Cython sources and expanding templates
        warnings.warn("Unrecognized setuptools command ('{}'), proceeding with "
      Running from SciPy source directory.
      Cythonizing sources
      Traceback (most recent call last):
        File "setup.py", line 350, in generate_cython
          import pip
      ModuleNotFoundError: No module named 'pip'
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "/home/oscar/current/sympy/sympy.git/tmpvenv/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
          main()
        File "/home/oscar/current/sympy/sympy.git/tmpvenv/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/home/oscar/current/sympy/sympy.git/tmpvenv/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 164, in prepare_metadata_for_build_wheel
          return hook(metadata_directory, config_settings)
        File "/tmp/pip-build-env-nvuzm8iv/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 174, in prepare_metadata_for_build_wheel
          self.run_setup()
        File "/tmp/pip-build-env-nvuzm8iv/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 267, in run_setup
          super(_BuildMetaLegacyBackend,
        File "/tmp/pip-build-env-nvuzm8iv/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 158, in run_setup
          exec(compile(code, __file__, 'exec'), locals())
        File "setup.py", line 628, in <module>
          setup_package()
        File "setup.py", line 612, in setup_package
          generate_cython()
        File "setup.py", line 352, in generate_cython
          raise RuntimeError("Running cythonize failed!")
      RuntimeError: Running cythonize failed!
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Actually it isn't necessary to install Cython above since SciPy will install its own Cython when building. The version constraint allows for 0.29.x updates though and so it will choose 0.29.31 even if cython==0.29.30 is used above:
https://github.com/scipy/scipy/blob/b76137f0d4fd82bd74b2e51af33af3ece44e45bc/pyproject.toml#L1-L14

Expected behavior

SciPy would build as it did with Cython 0.29.30

Environment (please complete the following information):

  • OS: I've seen this in CI and locally on different Ubuntu versions
  • Python version: It seems to be any version.
  • Cython version 0.29.31

Additional context

SciPy issue: scipy/scipy#16723
SymPy issue: sympy/sympy#23847
@oscarbenjamin oscarbenjamin mentioned this issue Jul 28, 2022
Closed
@da-woods
Copy link
Contributor

Yeah we've been told this by SciPy. They've fixed it and we've got a PR to fit it #4929

@da-woods
Copy link
Contributor

Here's the original Scipy issue scipy/scipy#16718

@oscarbenjamin
Copy link
Author

Thanks. Should this be closed then or should it be kept open until the pip install above succeeds?

@da-woods
Copy link
Contributor

Probably keep it open until we've merged the PR here

@scoder scoder added this to the 2.29.32 milestone Jul 28, 2022
@scoder
Copy link
Contributor

scoder commented Jul 31, 2022

Fixed in #4929 by reverting the new error.

@scoder scoder closed this as completed Jul 31, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
defect Error Reporting
Projects
None yet
Milestone
2.29.32
Development

No branches or pull requests
3 participants
@scoder @oscarbenjamin @da-woods