-
-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] The _distutils_hack
pth file causes significant startup slowdown
#3006
Comments
Thanks for reporting. I was aware this was a risk. I do believe the
That said, there may be some performance optimizations that can be made, and probably should be made if they don't encumber the implementation too much. |
For |
_distutils_hack
pth file casuses significant startup slowdown_distutils_hack
pth file causes significant startup slowdown
You could reduce the impact on startup performance by not using |
In order to avoid importing How temporary is the hack? Will it be gone by the time 3.11 final is released (coming October)? |
If you wanted to make us really happy, you could get rid of the |
Answering my own question, this seems to get rid of the warning printed by importing (@jaraco, would it be helpful if I submitted a PR?) |
``_distutils_hack`` is imported by a ``.pth`` file at every start of a Python interpreter. The import of costly modules like ``re`` and ``contextlib`` almost doubles the initial startup time of an interpreter. - replace ``contextlib`` with simple context manager and try/except - replace ``re`` with simple string match - move import of ``importlib`` into function body - remove ``warnings.filterwarnings()``, which imports ``re``, too. Fixes: pypa#3006 Signed-off-by: Christian Heimes <christian@python.org>
The warnings filter is problematic for two reasons. It not only slows down startup. It also hides Python core's deprecation warning from all users that have a recent version of setuptools. Since the |
Not quite. If you set |
setuptools main branch
with my fix
import of site went from 21774 to 8471. |
``_distutils_hack`` is imported by a ``.pth`` file at every start of a Python interpreter. The import of costly modules like ``re`` and ``contextlib`` almost doubles the initial startup time of an interpreter. - replace ``contextlib`` with simple context manager and try/except - replace ``re`` with simple string match - move import of ``importlib`` into function body - remove ``warnings.filterwarnings()``, which imports ``re``, too. Fixes: pypa#3006 Signed-off-by: Christian Heimes <christian@python.org>
It should not be needed since WeblateOrg/weblate@7949fb5 and can cause performance regressions (see pypa/setuptools#3006).
It should not be needed since WeblateOrg/weblate@7949fb5 and can cause performance regressions (see pypa/setuptools#3006).
setuptools version
Python version
ALL
OS
ALL
Additional environment information
No response
Description
Since setuptools 60.0.0 has been released, Python startup has been impacted noticeably:
https://speed.python.org/timeline/#/?exe=12&ben=python_startup&env=1&revs=50&equid=off&quarts=on&extr=on
This is due to the new pth file that setuptools installs for injecting the
distutils
hack. The pth imports the_distutils_hack
module and the ramifications of this import has a noticeable impact on startup time as you can see in the benchmark server.Here is an estimation of the cost using
-X importtime
on my machine:This almost doubles the startup time (from 13501 us to 24093 us) and unfortunately, as almost all Python installations will have the pth file installed by default, all Python installations will double its startup time.
We have been working very hard to reduce startup time with things like frozen modules (https://bugs.python.org/issue45020) and other optimizations and is unfortunate that this gains are being lost due to this.
See: https://bugs.python.org/issue46110 for more information and discussion
Expected behavior
The distutils hack doesn't impact startup time.
How to Reproduce
Measure the time spent in
python -c pass
. You can usepython -Ximporttime -c pass
to generate the import time table for convenience and debugging.Output
Relevant output of
python -X importtime -c pass
:Code of Conduct
The text was updated successfully, but these errors were encountered: