Use cibuildwheel to build wheels.

[bwoodsend]

Fixes #456.

Changes proposed in this pull request:

• Jettison our build wheels on CI infrastructure in favour of cibuildwheel.

We should have done this sooner. Look at all that code we get to delete!

I've rejiggled the work distribution. Now Windows, macOS and normal (x86_64 and i686) Linux get one job each. Then the really slow qemu emulation (Linux on aarch64) is split per Python version. Overall runtime is 13 minutes with Windows being the slowest (despite have half as much work to do as the native Linux runner 🙄).

[codecov-commenter]

Codecov Report

Merging #491 (104e852) into main (6833495) will not change coverage.
The diff coverage is n/a.

❗ Current head 104e852 differs from pull request most recent head 04286a6. Consider uploading reports for the commit 04286a6 to get more accurate results

@@           Coverage Diff           @@
##             main     #491   +/-   ##
=======================================
  Coverage   88.96%   88.96%           
=======================================
  Files           6        6           
  Lines        1685     1685           
=======================================
  Hits         1499     1499           
  Misses        186      186           

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6833495...04286a6. Read the comment docs.

[hugovk]

Excellent, thank you!

We should have done this sooner. Look at all that code we get to delete!

👏

---

I've rejiggled the work distribution. Now Windows, macOS and normal (x86_64 and i686) Linux get one job each. Then the really slow qemu emulation (Linux on aarch64) is split per Python version. Overall runtime is 13 minutes with Windows being the slowest (despite have half as much work to do as the native Linux runner 🙄).

What do you think about also splitting the deploy-wheels-native.yml matrix into Python versions? Then each one would be maybe 90 seconds or less.

Then the bottleneck would be a QEMU job at around 9 minutes, but all the others should have finished by then.

Another nice benefit of separate jobs is each wheel is uploaded to PyPI a few minutes earlier, and so narrowing the window someone ends up getting and building the sdist.

---

Finally, I think it's worth bundling this in with the 3.6 drop = major version bump. Might be safer just in case of incompatibilities. What do you think?

[bwoodsend]

What do you think about also splitting the deploy-wheels-native.yml matrix into Python versions? Then each one would be maybe 90 seconds or less.

Honestly, I find firing off 58 different jobs to save a few minutes thumb twiddling time slightly excessive but, sure, your call...

Then the bottleneck would be a QEMU job at around 9 minutes, but all the others should have finished by then.

Those qemu jobs are doing two jobs each (one glibc and one musl) so we could slice them in half too to bring it down to about 5 minutes.

Another nice benefit of separate jobs is each wheel is uploaded to PyPI a few minutes earlier, and so narrowing the window someone ends up getting and building the sdist.

Surely if the timing is that critical then it would be best to maket the sdist be uploaded last? i.e. Stick it in its own job to be launched only when the wheel building jobs succeed?

Finally, I think it's worth bundling this in with the 3.6 drop = major version bump. Might be safer just in case of incompatibilities. What do you think?

The Windows and macOS wheels are now being built with https://python.org/downloads Pythons instead of those on https://github.com/actions/python-versions so any difference in the compiler flags used building Python will now be affecting those wheels too. Without rummaging through either's build systems, the only difference I know of is that python.org sets the macOS deployment target to 10.9 whereas Github's is unset and defaults to 10.14 (the macOS VM version they build on). So these new wheels should be more compatible in that they also support macOS 10.9 and up. I'd be quite surprised if this is a breaking change but if you're already about to do a major version bump then we might as well assume the worst.

[hugovk]

Surely if the timing is that critical then it would be best to make the sdist be uploaded last? i.e. Stick it in its own job to be launched only when the wheel building jobs succeed?

Ah, let's go for that :) Can we have deploy.yml depend on these two wheel workflows or something like that?

So these new wheels should be more compatible in that they also support macOS 10.9 and up. I'd be quite surprised if this is a breaking change but if you're already about to do a major version bump then we might as well assume the worst.

Yep, let's ditch 3.6 from this one too. The new macosx_11_0_arm64 ones are only for 3.8+ so they won't really be losing out on anything.

[bwoodsend]

Can we have deploy.yml depend on these two wheel workflows or something like that?

You can indeed but it looks like they must be defined all in one file so I've merged all 3 deploy*.yml files into one and added the magic line that ensures that the sdist job doesn't happen until the other two succeed. If you look at the run you should see that Github has even drawn a cute little dependency graph to make the point.

[hugovk]

Because deploy.yml can now be triggered by PR, we'll need a guard for the Test PyPI upload, like for the other two:

      if: |
        github.repository == 'ultrajson/ultrajson' &&
        github.ref == 'refs/heads/main'

[hugovk]

Thank you!

Will merge this and #490 in December, to coincide with 3.6 EOL month.

Will likely release before the actual EOL date, the previous 3.6-compatible versions will still be available on PyPI and python_requires will help out.

[hugovk]

Thank you!

[hugovk]

All deployed nicely, thanks again!

https://github.com/ultrajson/ultrajson/actions/runs/1587833653
https://pypi.org/project/ujson/5.0.0/#files

[bwoodsend]

You're welcome 🙂