New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Integration feedback report: psf/black #886
Comments
It's been a long while I've updated this post, I'll try to write an update soon but the TL;DR is that we are shipping mypyc compiled wheels in Release 22.1.0 of Black. Other than psf/black#2845 and unexpected unofficial API breakage it's been smooth sailing so far :) I'm actually in the midst of writing a blog post describing my experience integrating mypyc into Black so that's currently eating my OSS time 🙂 If any mypyc core developers wants to review it once I finish it (no promises on when!) feel free to @ me and we can discuss. Obviously please don't feel like you must, it's just an invitation, I get we're all busy as OSS maintainers 🙂 Once again, thank you for the great product! |
So I'm going to assume that people have heard of my blog series since it kinda blew up 😅 but just in case, here's part one. Anyway, there's not actually too much to comment on even after five months. With the help of Jelle, I was able to land fixes for #917 to allow us to use the latest mypy[c] to compile Black. Turns out that's not possible quite yet thanks to #941. psf/black#2845 is still a pain and not much progress has been made on it, but there doesn't seem to be any other crashes or issues using mypyc with Black as of writing 🎉 For what it's worth, I've been learning C this summer and slowly familiarizing myself with the mypyc codebase so I can contribute fixes and other improvements. My hope is to be able to eventually fix the issues impacting Black myself, ... but some of these issues do look very involved so I'm not too sure about that :) |
Hello! So I'm Richard from the psf/black maintainer team, and for the past few months I've been working on finishing up the work done by Sully in psf/black#1009 so mypyc can finally be deployed. As a token of appreciation (and to help out the project) I've written up a report containing my experience integrating mypyc with psf/black :)
First of all, here's the relevant links:
The good
Final
, etc.) - additional performance wins of 5-15% were achieved :Dsetup.py
logic) - I did steal lots of ideas from mypyc/mypy_mypyc-wheels :PThe bad
ast.NodeVisitor
crashes withAttributeError: attribute 'dict' of 'type' objects is not writable
(maybe not fixable?)abc.ABC
+ dataclass crashes toomypy.ini
hard-codingplatform=linux
. Other interesting case was one involving a module typed asAny
causing an else branch to appears as it's never ever taken. The resulting unanalyzed code crashed when hit. More details in this commit: psf/black@acb77f7And other various issues already reported on the mypyc issue tracker:
Suggestions
platform=$OS
being problematicCFLAGS=-g0
brought the wheels to a much more reasonable ~1MB each.python -c "import foo"
Questions
setup.py
end up triggering the above and it turns out bothpip wheel
andpython -m build
call at-least two PEP 517 hooks that eventually end up evaluatingsetup.py
. It's not a big deal, but it slows down wheel builds a little bit. Sounds hard to avoid tho (caching would be complicated).What's next?
Currently the stability and performance data for Linux were just collected and look good so a PR was opened against psf/black. There's a lot of TODOs listed in that PR but it is progress.
The main thing is that I need to do is getting the wheels significantly more battle tested so they can eventually find their way to PyPI. First of all I need to run a mypy-primer equivalent on Windows and MacOS. Assuming that doesn't turn up anything scary, the PR will probably get merged once the reviews are over with. Then at that point, it's off to community testing where we'll be asking members of the community to use the experimental wheels and report issues. So expect updates on this issue as more stability data comes in ... but I'm sure this is already enough insights for a post ^^
Finally
As much as this is riddled with negativity and bugs, I appreciate all of the work y'all put into mypyc. It's awesome to see so much work into making Python code faster lately! 🖤 If you have any questions, feel free to ask away.
The text was updated successfully, but these errors were encountered: