Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

maint(ci): avoid cython 0.29.31 under PyPy #23847

Merged
merged 1 commit into from
Jul 28, 2022

Conversation

oscarbenjamin
Copy link
Contributor

References to other Issues or PRs

#23061 (comment)

Brief description of what is fixed or changed

The PyPy optional dependency job avoids Cython 0.29.31 which fails to build SciPy 1.8.1. This is probably an upstream bug in either Cython or SciPy.

Other comments

For a brief moment the CPython 3.11.0b5 release hit CI and we actually had everything pass on master (25cbe76). Within hours though two different problems emerged and the next merge to master (dc86589) has two jobs failing both in the build of SciPy.

It's not clear yet what the problems are that cause this or if they are related. They seem to be because they both show the same error pointing at

      FAILED: scipy/stats/_stats.cpython-311-x86_64-linux-gnu.so.p/_stats.c
      /opt/hostedtoolcache/Python/3.11.0-beta.5/x64/bin/python /tmp/pip-req-build-ftutz8xt/scipy/_build_utils/cythoner.py ../../scipy/stats/_stats.pyx scipy/stats/_stats.cpython-311-x86_64-linux-gnu.so.p/_stats.c
      
      Error compiling Cython file:
      ------------------------------------------------------------
      ...
          return np.array(result, dtype=np.int64)
      
      
      @cython.wraparound(False)
      @cython.boundscheck(False)
      def _weightedrankedtau(ordered[:] x, ordered[:] y, intp_t[:] rank, weigher, bool additive):
      ^

However the two jobs use different versions of everything. The PyPy job installs cython, numpy and scipy all from pip to retrieve the latest versions. The only thing that seems to have changed there is the release of Cython 0.29.31.

The 3.11 job now uses CPython 3.11.0b5 and installs cython, numpy and scipy from git master/main. The Cython master branch currently reports as Cython-3.0.0a10. It's not clear if there has been a change in Cython master or a change in SciPy main that has caused this to fail but since the error message is similar to the one seen with Cython 0.29.31 maybe they are releated.

Release Notes

NO ENTRY

@sympy-bot
Copy link

Hi, I am the SymPy bot (v167). I'm here to help you write a release notes entry. Please read the guide on how to write release notes.

  • No release notes entry will be added for this pull request.
Click here to see the pull request description that was parsed.
<!-- Your title above should be a short description of what
was changed. Do not include the issue number in the title. -->

#### References to other Issues or PRs
<!-- If this pull request fixes an issue, write "Fixes #NNNN" in that exact
format, e.g. "Fixes #1234" (see
https://tinyurl.com/auto-closing for more information). Also, please
write a comment on that issue linking back to this pull request once it is
open. -->

https://github.com/sympy/sympy/issues/23061#issuecomment-1197517304

#### Brief description of what is fixed or changed

The PyPy optional dependency job avoids Cython 0.29.31 which fails to build SciPy 1.8.1. This is probably an upstream bug in either Cython or SciPy.

#### Other comments

For a brief moment the CPython 3.11.0b5 release hit CI and we actually had everything pass on master (25cbe76557c2685afc209dcca13c9e0ec3f9a68e). Within hours though two different problems emerged and the next merge to master (dc86589ada8739dfdbf3bf014e6361bc4a42ff70) has two jobs failing both in the build of SciPy.

It's not clear yet what the problems are that cause this or if they are related. They seem to be because they both show the same error pointing at
```
      FAILED: scipy/stats/_stats.cpython-311-x86_64-linux-gnu.so.p/_stats.c
      /opt/hostedtoolcache/Python/3.11.0-beta.5/x64/bin/python /tmp/pip-req-build-ftutz8xt/scipy/_build_utils/cythoner.py ../../scipy/stats/_stats.pyx scipy/stats/_stats.cpython-311-x86_64-linux-gnu.so.p/_stats.c
  
      Error compiling Cython file:
      ------------------------------------------------------------
      ...
          return np.array(result, dtype=np.int64)
  
  
      @cython.wraparound(False)
      @cython.boundscheck(False)
      def _weightedrankedtau(ordered[:] x, ordered[:] y, intp_t[:] rank, weigher, bool additive):
      ^
```
However the two jobs use different versions of everything. The PyPy job installs cython, numpy and scipy all from pip to retrieve the latest versions. The only thing that seems to have changed there is the release of Cython 0.29.31.

The 3.11 job now uses CPython 3.11.0b5 and installs cython, numpy and scipy from git master/main. The Cython master branch currently reports as Cython-3.0.0a10. It's not clear if there has been a change in Cython master or a change in SciPy main that has caused this to fail but since the error message is similar to the one seen with Cython 0.29.31 maybe they are releated.


#### Release Notes

<!-- Write the release notes for this release below between the BEGIN and END
statements. The basic format is a bulleted list with the name of the subpackage
and the release note for this PR. For example:

* solvers
  * Added a new solver for logarithmic equations.

* functions
  * Fixed a bug with log of integers.

or if no release note(s) should be included use:

NO ENTRY

See https://github.com/sympy/sympy/wiki/Writing-Release-Notes for more
information on how to write release notes. The bot will check your release
notes automatically to see if they are formatted correctly. -->

<!-- BEGIN RELEASE NOTES -->
NO ENTRY
<!-- END RELEASE NOTES -->

@oscarbenjamin
Copy link
Contributor Author

This doesn't work because I think when SciPy builds it installs its own version of Cython and so still picks up 0.29.31 even if we don't install that version.

@oscarbenjamin
Copy link
Contributor Author

I've opened a SciPy issue: scipy/scipy#16723

For now I'll make the PyPy job an allowed failure in CI (the 3.11 job already is).

@github-actions
Copy link

Benchmark results from GitHub Actions

Lower numbers are good, higher numbers are bad. A ratio less than 1
means a speed up and greater than 1 means a slowdown. Green lines
beginning with + are slowdowns (the PR is slower then master or
master is slower than the previous release). Red lines beginning
with - are speedups.

Significantly changed benchmark results (PR vs master)

Significantly changed benchmark results (master vs previous release)

Full benchmark results can be found as artifacts in GitHub Actions
(click on checks at the top of the PR).

@oscarbenjamin oscarbenjamin merged commit 8ad2cf2 into sympy:master Jul 28, 2022
@oscarbenjamin oscarbenjamin deleted the pr_pin_cython branch July 28, 2022 17:29
oscarbenjamin added a commit to oscarbenjamin/sympy that referenced this pull request Jul 29, 2022
Reverts sympyGH-23847 since Cython 0.29.32 was released making it
unnecessary.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants