Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move away from setup.py #2189

Closed
rth opened this issue Feb 19, 2022 · 2 comments · Fixed by #2272
Closed

Move away from setup.py #2189

rth opened this issue Feb 19, 2022 · 2 comments · Fixed by #2272

Comments

@rth
Copy link
Member

rth commented Feb 19, 2022

As mentioned by @henryiii is

Building via setup.py is very heavily deprecated. I wouldn't recommend adding anything that depends on it.

With the other alternatives mentioned being build and installer.

I think build isolation would be a bit difficult to use in our case, as for instance when building scipy we need the patched numpy on the host and not the numpy version specified in pyproject.toml (which would be unpatched). However trying to switch to build, for instance, without build isolation would certainly be worthwhile.

@henryiii
Copy link
Contributor

flit now supports bootstrapping (as of 3.7, yesterday), and there's now a clear path for creating a working install/build system from scratch. See pypa/build#394.

One alternative to disabling build isolation is to point at the local wheelhouse. Also, it seems to me that there should be a) a rule that all packages must match the PyPI name (something conda didn't do, largely because conda packages may not be Python packages), and b) a list of pure Python packages that simply get downloaded as wheels, rather than "being built" like a binary package with a formula.

@hoodmane
Copy link
Member

I am not sure that bootstrapping is important for us anyways because we are not intending for Pyodide to be self hosting so we don't need to "cross build" our build system.

hoodmane added a commit that referenced this issue Mar 22, 2022
This resolves #2189.

> build isolation would be a bit difficult to use in our case, as for instance
> when building scipy we need the patched numpy on the host and not the numpy
> version specified in pyproject.toml (which would be unpatched)

This is indeed the case, certain packages cannot be isolated. My strategy is to
make a list of packages that shouldn't be isolated and add symlinks from the
isolated build environment into the `.artifacts` directory to "unisolate" them.
Then we remove the unisolated package requirements from the list of packages to
install, in case pesky constraints aren't satisfied. In particular, packages
that expect to be used with `pypa/build` often feel free to put very specific
constraints on their build dependencies (often asking them to be == to a
particular version). Specific version constraints is good for build
reproducibility and with build isolation doesn't cost anything. So we just
ignore the constraints. Hopefully nothing goes wrong.

In particular, any package that does stuff both at build time and at runtime and
requires synchronization between the build time and run time environments needs
the unisolation. This includes cffi with `_cffi_backend.so`, and of course numpy
and scipy. pycparser needs to be unisolated because it is a dependency of cffi.

Currently I have also unisolated pythran and cython, though these are build time
only tools and do not really need to be unisolated. Cython I unisolated
specifically because numcodecs needs it but it isn't in the numcodecs build
dependencies. Pythran I unisolated because of a problem with the scipy build
which I don't fully understand (some problem with long double feature
detection).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants