Why does pyodide re-build pure python packages such as typing-extensions? #4667
-
|
Hi, I see that the pure Python package typing-extensions is added here: https://github.com/pyodide/pyodide/blob/main/packages/typing-extensions/meta.yaml What advantage does this have? Since pyodide can also install files from pypi directly, this seems to put a limit/version constraint on this package. Regards, Maarten |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 5 replies
-
|
We don't actually rebuild it, we just grab the wheel from pypi. We need it because it's a transitive dependency of one or more native extensions we do include. |
Beta Was this translation helpful? Give feedback.
-
|
But then why is this file in the repo and do you republish it on the jsdelivr cdn. Couldn't pyodide just grab the dependency from pypi (as happens with micropip)? I'm trying to understand the design choices here to better understand pyodide. |
Beta Was this translation helpful? Give feedback.
-
We want to lock all of the transitive dependencies of the packages we include so that what we test matches what we ship. Ideally that means they will all work together correctly.
Yes, our lockfile works with PyPI urls so that would work fine. We try to remove tests from wheels that include them and do various other things to shrink the size of some wheels. We can also insert patches or add files. But some don't get modified. |
Beta Was this translation helpful? Give feedback.
-
|
So, i'm free to remove for instance In fact, I want to remove any non-build dependency from this repodata_packages structure to pick up anything modern from pypi. The underlying reason for this is that I need a more modern version for installing OpenAI. |
Beta Was this translation helpful? Give feedback.
-
Dunno, if it works it works. Does it not work to do |
Beta Was this translation helpful? Give feedback.
-
|
Yes, but then I run into pyodide/micropip#103 |
Beta Was this translation helpful? Give feedback.
We want to lock all of the transitive dependencies of the packages we include so that what we test matches what we ship. Ideally that means they will all work together correctly.
Yes, our lockfile works with PyPI urls so that would work fine. We try to remove tests from wheels that include them and do various other things to shrink the size of some wheels. We can also insert patches or add files. But some don't get modified.