New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Speedup startup time #4768
Comments
I did notice one can save about 80ms by using the option to disable the version check - it'd probably make sense not to do that at all for completions by default. Still, a lot remains to be improved. |
I wonder how much of this time is taken up by importing stuff. |
A rudimentary test of running the above 5 times gives me an average of 0.377s. |
Importing |
pip/src/pip/_vendor/vendor.txt Line 21 in 4092707
|
Yes, and it's faster than pip 9.0, but there are some further changes in setuptools 36.5 that might help: https://github.com/pypa/setuptools/blob/master/CHANGES.rst#v3650. |
|
Oh, nice. That means the next round of vendor updates would bring some speedup. :) I, personally, am waiting on a new distlib release before giving the vendored libraries another round of updates. |
I fired up the profiler and ran
|
PS: Need better profiling tools. |
Lazy importing will probably solve some of those. |
There are plans to (hopefully) make lazy importing easy to switch on for CLI apps like pip in Python 3.7. There is also now a |
What does "updated pkg_resources" mean? On my machine the initial import is still quite slow even though I have setuptools 38.1.0, so your comment seems very interesting to me! :P |
@boxed: it means with pip's vendored version of setuptools updated. |
Aha. I tried copying over pkg_resources from my main install over the one inside pip/_vendor, but I didn't see any difference in speed :/ |
Using CPython 3.7.0's
|
Just in case y'all are unaware, pkg_resources is tracking the slowness in pypa/setuptools#510, I didn't see it linked in this issue yet. |
Could you not move the imports from the top of the file |
@CSDUMMI I have a PR that does this. It helps somewhat. |
Could I have a link? |
Here is a PR that improves the import situation for the vcs imports: #6545 It removes the |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Let's move the discussion to boxed/p#4 |
FYI, PR #6694 ("Only import a Command class when needed") was recently merged, which helps with this. |
I posted PR #6835 to help with this. |
I noticed a pretty significant slowdown in the latest released version -- I've tracked it down to here -- might be worth bumping |
I also looked into pip startup time a bit. First thing I noticed is Another one I noticed is If there is interest I can try to prepare a PR to replace the |
Note that we can only vendor pure Python libraries. I don't have any strong opinions on whether we should switch, I'm just noting these points as things to consider if we do. |
Hmm the |
The developer did a lot to make it more acceptable to the If the developer is still as accommodating, I'd imagine pip would benefit in performance and ease of maintenance with |
OK, submitted PR #12638. BTW, another big import-time hit is With pyparsing, asyncio and chardet gone it will be a decent improvement to startup time. The remaining big ones are |
There's a lot to gain from speeding up pip's startup time.
For one, pip takes around 600ms to just print the completion text, which is laggy. (as mentioned in #4755). Further, faster startup time might help with the test-suite situation too.
The text was updated successfully, but these errors were encountered: