Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test suite gets really slow in pypy3.9-v7.3.15 on ubuntu 22.04.3 #1738

Open
ecerulm opened this issue Jan 23, 2024 · 4 comments
Open

test suite gets really slow in pypy3.9-v7.3.15 on ubuntu 22.04.3 #1738

ecerulm opened this issue Jan 23, 2024 · 4 comments
Labels
bug Something isn't working needs triage

Comments

@ecerulm
Copy link

ecerulm commented Jan 23, 2024

Describe the bug

In urllib3, we just downgraded to pypy3.9-v7.3.13 (urllib3/urllib3#3308) because the test suite takes more that 30 minutes when run on pypy3.9-v7.3.15 (we don't know how long exactly because the CI jobs is cancelled at the 30 minutes mark) and less than 10 minutes when run on pypy3.9-v7.3.15.

To Reproduce
How can we reproduce the problem? Please be specific. Don't link to a failing CI job. Answer the questions below:

  1. What version of Python are you using?
    pyp3.9-v7.3.15 on Ubuntu 22.04.3 LTS
  2. What version of coverage.py shows the problem? The output of coverage debug sys is helpful.
    7.4.0
  3. What versions of what packages do you have installed? The output of pip freeze is helpful.
aiofiles==23.2.1
anyio==4.2.0
attrs==23.2.0
blinker==1.7.0
brotlicffi==1.1.0.0
certifi==2023.11.17
cffi==1.16.0
click==8.1.7
click-default-group==1.2.4
coverage==7.4.0
cryptography==41.0.6
exceptiongroup==1.2.0
Flask==3.0.1
greenlet==0.4.13
h11==0.14.0
h2==4.1.0
hpack==4.0.0
hpy==0.9.0
httpcore==1.0.2
httpx==0.25.2
Hypercorn @ git+https://github.com/urllib3/hypercorn@d1719f8c1570cbd8e6a3719ffdb14a4d72880abb
hyperframe==6.0.1
idna==3.4
importlib-metadata==7.0.1
importlib-resources==6.1.1
incremental==22.10.0
iniconfig==2.0.0
itsdangerous==2.1.2
Jinja2==3.1.3
MarkupSafe==2.1.4
outcome==1.3.0.post0
packaging==23.2
pluggy==1.3.0
priority==2.0.0
pyOpenSSL==23.2.0
PySocks==1.7.1
pytest==7.4.2
pytest-timeout==2.1.0
Quart==0.19.4
quart-trio==0.11.1
readline==6.2.4.1
sniffio==1.3.0
sortedcontainers==2.4.0
taskgroup==0.0.0a4
tomli==2.0.1
towncrier==23.6.0
trio==0.23.1
trustme==1.1.0
typing_extensions==4.9.0
urllib3
Werkzeug==3.0.1
wsproto==1.2.0
zipp==3.17.0
zstandard==0.22.0
  1. What code shows the problem? Give us a specific commit of a specific repo that we can check out. If you've already worked around the problem, please provide a commit before that fix.

The workaround if was to use pypy3.9-v7.3.13 (urllib3/urllib3@f14b508)

The version of the urllib3 that still shows the problem in CI is urllib3/urllib3@6b2b377

  1. What commands should we run to reproduce the problem? Be specific. Include everything, even git clone, pip install, and so on. Explain like we're five!

I'll expand later because I can't reproduce this myself on my laptop (macOS Sonoma pypy3.9-v7.3.15) so I think it maybe only reproducible on Ubuntu 22.04.3 LTS

Expected behavior

I expected it to complete the test suite (with coverage) in under 10 minutes while using pypy3.9-v7.3.15 as it does when using pypy3.9-v7.3.13

Additional context

@ecerulm ecerulm added bug Something isn't working needs triage labels Jan 23, 2024
@ecerulm
Copy link
Author

ecerulm commented Jan 23, 2024

Examples of slow test suite execution under coverage for pypy3.9-v7.3.15

https://github.com/urllib3/urllib3/actions/runs/7603721375/job/20705793502?pr=3296
https://github.com/urllib3/urllib3/actions/runs/7559660055/job/20583952938?pr=3275
https://github.com/urllib3/urllib3/actions/runs/7615341858/job/20739660497?pr=3273

they all stop at the 30 minutes mark (the CI jobs get cancelled after 30 minutes) but they show that it's not stuck in any particular test case , all of them progress to a different number of test cases.

I haven't been able to reproduce it in macOS , I'll later try on Ubuntu 22.04 (outside of CI) and post the findings here.

@ecerulm
Copy link
Author

ecerulm commented Jan 23, 2024

I performed a test with pypy3.9-7.3.15 and without coverage and it still takes more that 30 minutes (although it progresses faster that with coverage), so I guess this is probably an issue with pypy3.9-7.3.15 and not really related to coverage.

@ecerulm ecerulm changed the title test suite gets really slow in pypy3.9-v7.3.15 on ubuntu 20 test suite gets really slow in pypy3.9-v7.3.15 on ubuntu 22.04.3 Jan 23, 2024
@ecerulm
Copy link
Author

ecerulm commented Jan 23, 2024

pypy/pypy#4877

@nedbat
Copy link
Owner

nedbat commented Jan 23, 2024

Thanks, we have our own issue with 7.3.15: #1737 . I assume it's the same root cause, whatever it is. For us, it's only on Windows.

di added a commit to pypi/warehouse that referenced this issue Feb 22, 2024
This is the likely cause of our test suite hanging in CI.

Possibly related issues:
- nedbat/coveragepy#1737
- nedbat/coveragepy#1738
di added a commit to pypi/warehouse that referenced this issue Feb 22, 2024
This is the likely cause of our test suite hanging in CI.

Possibly related issues:
- nedbat/coveragepy#1737
- nedbat/coveragepy#1738
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working needs triage
Projects
None yet
Development

No branches or pull requests

2 participants