Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Switch to maturin. #8815

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Switch to maturin. #8815

wants to merge 1 commit into from

Conversation

alex
Copy link
Member

@alex alex commented Apr 24, 2023

It seems to be much faster at doing things locally.

@alex
Copy link
Member Author

alex commented Apr 24, 2023 •
edited

TODO before mergable:

  • Get this working in CI...
  • Figure out how to handle version numbers
  • Make sure the abi3 stuff works with PyPy
  • Update wheel builder to no longer use setup.py directly (this should be done in a separate PR)
  • Figure out what the correct behavior with respect to MANIFEST.in is
  • Maturin stable release
  • Figure out how maturin's MSRV interacts with our MSRV

@alex alex force-pushed the maturin branch 8 times, most recently from 0ed4d74 to 78c881c Compare April 26, 2023 23:37
@messense
Copy link

Hey, maturin's sdist support is be a bit hacky so you may run into obscure problems, feel free to ping me or open an issue if something doesn't work.

@alex
Copy link
Member Author

alex commented Apr 27, 2023 via email

@alex
Copy link
Member Author

alex commented Apr 30, 2023

@messense is there a way to disable the rewriting things to be local_dependencies? My preference is to just include everything at its normal location with include.

@messense
Copy link

messense commented May 2, 2023

is there a way to disable the rewriting things to be local_dependencies?

No way ATM, I do want to support it to avoid PyO3/maturin#1442.

@alex
Copy link
Member Author

alex commented May 2, 2023 via email

@messense
Copy link

messense commented May 2, 2023

Would it be useful to file a bug for that, or should I just follow 1442?
…

Yeah, file a bug is useful.

@alex
Copy link
Member Author

alex commented May 4, 2023

@messense Is it intentional that maturin doesn't set the PYO3_PYTHON env var when running cargo? setuptools-rust does (https://github.com/PyO3/setuptools-rust/blob/main/setuptools_rust/build.py#L600), and we use it in our build.rs (I think you even wrote that code :-))

Specifically, this is happening when pip wheel invokes ['maturin', 'pep517', 'build-wheel', '-i', '/Users/alex_gaynor/projects/cryptography/.venv/bin/python3.11', '--compatibility', 'off'], so I'd expect the -i value to go into PYO3_PYTHON.

@messense
Copy link

messense commented May 5, 2023 

  🔗 Found pyo3 bindings with abi3 support for Python ≥ 3.7
  🐍 Not using a specific python interpreter

I think it has something to do with abi3, we can certainly pass down the interpreter path if it's specified. I can take a look this weekend.

@alex
Copy link
Member Author

alex commented May 5, 2023

Thanks!

@messense messense mentioned this pull request May 6, 2023
Merged
bors bot added a commit to PyO3/maturin that referenced this pull request May 6, 2023
@bors @messense
Merge #1592
997630c 
1592: Always set `PYO3_PYTHON` if interpreter is runnable regardless of abi3 r=messense a=messense

It can be useful in build scripts, like pyca/cryptography#8815 (comment)

Co-authored-by: messense <messense@icloud.com>
bors bot added a commit to PyO3/maturin that referenced this pull request May 6, 2023
@bors @messense
Merge #1592
93bbf2b 
1592: Always set `PYO3_PYTHON` if interpreter is runnable regardless of abi3 r=messense a=messense

It can be useful in build scripts, like pyca/cryptography#8815 (comment)

Co-authored-by: messense <messense@icloud.com>
bors bot added a commit to PyO3/maturin that referenced this pull request May 6, 2023
@bors @messense
Merge #1592
8af9efe 
1592: Always set `PYO3_PYTHON` if interpreter is runnable regardless of abi3 r=messense a=messense

It can be useful in build scripts, like pyca/cryptography#8815 (comment)

Co-authored-by: messense <messense@icloud.com>
@messense
Copy link

messense commented May 6, 2023

Please try maturin 1.0.0b9.

@alex
Copy link
Member Author

alex commented May 6, 2023

Ok, we are green here, huzzah!

The remaining things we need are: a) Maturin stable release (this currently uses a beta), b) @reaperhulk we need to discuss MSRV implications:

maturin currently has an MSRV of 1.64, which is obviously higher than our MSRV. It also has wheels for every possible platform it seems, https://pypi.org/project/maturin/1.0.0b9/#files

@alex
Copy link
Member Author

alex commented May 13, 2023

After discussion with @reaperhulk, we think the MSRV is too aggressive. Going to hold this in draft for now.

@alex alex marked this pull request as draft May 13, 2023 00:28
@messense
Copy link

I do think 1.63.0 is going to be easier for us than 1.64.0, because of Debian.

Let me know when you plan to switch, I'll try to get a new maturin release out with MSRV of 1.63.0 for Debian.

BTW, consider upgrade to maturin 1.3.1 in this PR, it's currently using 1.2.0.

@alex
Copy link
Member Author

alex commented Oct 27, 2023

Our next release is going to have an MSRV of 1.63.

And yes, I should bump the pin, I've mostly been lazy -- I've only been rebasing to resolve conflicts, and I've been assuming dependabot would bump it for me when we finally merge :-)

@alex
Copy link
Member Author

alex commented Oct 27, 2023

And I should have said: Thank you for considering a release with a lower MSRV!

@messense
Copy link

FYI, there is already a python-maturin package in Debian unstable, but it's a bit old, version is 1.1.0. Not sure when will it go to Debian stable.

@alex
Copy link
Member Author

alex commented Oct 28, 2023

The case we want to support is people who are installing packages with pip, and not using wheels.

@alex alex force-pushed the maturin branch 3 times, most recently from 53c5651 to 27590a4 Compare November 1, 2023 18:08
@messense
Copy link

The case we want to support is people who are installing packages with pip, and not using wheels.

In that case I think lowering MSRV of maturin only helps a bit but creates too much maintenance burden for maturin. If people are installing packages with pip on Debian, they are also fine to use rustup to install a recent version of Rust I think?

@alex
Copy link
Member Author

alex commented Nov 10, 2023

I definitely understand the maintenance burden concern. Thanks for considering it.

@alex alex force-pushed the maturin branch 3 times, most recently from d65784b to 3fd7767 Compare February 24, 2024 20:12
@alex alex marked this pull request as ready for review February 24, 2024 20:12
@alex alex changed the title Attempt to switch to maturin. Switch to maturin. Feb 24, 2024
@alex alex force-pushed the maturin branch 2 times, most recently from 5c13ed1 to 78f9e71 Compare February 25, 2024 23:08
@alex alex added this to the Forty Fourth Release milestone Feb 26, 2024
@alex alex force-pushed the maturin branch 2 times, most recently from 7bb47e8 to 0f22527 Compare March 6, 2024 00:42
@alex alex force-pushed the maturin branch 2 times, most recently from dc027af to dc1924e Compare March 30, 2024 18:28
@alex
Switch to maturin.
3198b5b 
It seems to be much faster at doing things locally.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
blocked-on-msrv
Milestone
Forty Fourth Release
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@alex @messense