Apple Silicon Wheels

[henryiii]

I'd like to start this thread to collect user's experiences in adding Apple Silicon wheels, either Universal2 or Arm64, using the new cibuildwheel 1.9.0 and Pip 21.0.1 (at least, that one is needed for Universal2 on AS).

[henryiii]

I now ship Universal2 wheels for boost-histogram; I've added them for 0.12.0 by back porting the CI change and manually uploading the new wheel. My CI file is here, and the PR adding it was scikit-hep/boost-histogram#495. It worked beautifully the first time; initial performance test look like it's faster on a Mac Mini than on my 8 core i9, maybe by as much as 2x when multithreading (very initial testing, though!). Added it took very little change, and I later also split the lengthy Windows build into auto32/auto64, to keep all the jobs similar in length. Very elegant, I'd say!

[Czaki]

initial performance test look like it's faster on a Mac Mini than on my 8 core i9, maybe by as much as 2x when multithreading

Did you try to compare with wheels compiled against 10.15, not 10.9? Compatybility with old system version could reduce speed.
I would really see comparision between x86 compiled against current processor and system (maybe also on Ubuntu) and arm64 .

My base knowledge about compilers and difference of programs speeds on HPC, when using gereral binary and compiled for given machine suggest me that part of these speedup is no need of backward compatybility, and this profit may be lost in few next years (because of a new processors).

[henryiii]

No, it was a 10.9+ wheel in both cases, but of course, the AS part of the 10.9 wheel is 11.0+. I'll try a full 11+ on Intel in the near future, excellent suggestion! I'll also try using the universal wheel on Intel, rather than using the normal x86 one. I should get my own AS Mac on Thursday so doing comparisons will be easier then. One benefit of a new architecture is the fact there is no baggage like older systems or even older Python (it's 3.9 only at the moment); it takes many years to drop an old operating system (10.9 is what, 7 years old now?)

[Czaki]

Also, it could be interesting to test compilation with gcc and usage of march flag https://www.alibabacloud.com/blog/gcc-vs--clangllvm-an-in-depth-comparison-of-cc%2B%2B-compilers_595309

Most probably I will have no access to M1 hardware in a short time perspective.

[YannickJadoul]

I'm having a bit of a hiccup, adding this to Parselmouth: https://github.com/YannickJadoul/Parselmouth/runs/1845996721

Repairing wheel...
  
  + delocate-listdeps /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibuildwheela86aupic/built_wheel/praat_parselmouth-0.4.0.dev0-cp39-cp39-macosx_10_9_x86_64.whl && delocate-wheel --require-archs arm64 -w /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibuildwheela86aupic/repaired_wheel /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibuildwheela86aupic/built_wheel/praat_parselmouth-0.4.0.dev0-cp39-cp39-macosx_10_9_x86_64.whl
                                                              ✓ 3.61s

delocate seems to be accepting things it shouldn't, here?

[YannickJadoul]

It seems like setting MACOSX_DEPLOYMENT_TARGET in before_all is a clean win. I wonder if there's any value in us exporting e.g. CIBUILDWHEEL_BUILD_IDENTIFIER that would make setting things like CMAKE_OSX_ARCHITECTURES a bit cleaner? How do you decide that currently?

Yes on MACOSX_DEPLOYMENT_TARGET.
To set CMAKE_OSX_ARCHITECTURES, I do this based on my GHA matrix (where x86_64, arm64, and universal2 are split up), such that I know there's only 1 architecture. It's kind of inevitable to rebuild everything, anyway, so I don't mind.

Actually, I'm now realizing I already did this naturally for Windows. There's also only 1 CIBW_BEFORE_ALL for both architectures (but they were already split up for time concerns).
So CIBW_BEFORE_ARCH would provide another, more principled solution to build non-Python binary dependencies, but... not sure if I'm a big fan of introducing yet another variable, either :-/

I was wondering if the natural thing would be to build 'fat' universal2 shared libraries in the BEFORE_ALL step. But maybe that would inflate your binary size too much?

Yeah, so you mean to include a fat dependency in arm64 and x86_64? I suppose that would work, but it feels reasonably wasteful, indeed. Most of the code is actually in this external dependency that's independent from Python, and indeed, the universal2 wheel size already doubled (to 19 MB), now.

[YannickJadoul]

Alternatively, @Czaki, do you know how to inspect these wheels? Can I easily check the MACOSX_DEPLOYMENT_TARGET to be 10.9 and the advertised architectures on a Linux machine?

I'm not sure if I good understand your question. It is possible to read this information. I wrote such code for wheel package. https://github.com/pypa/wheel/blob/master/src/wheel/macosx_libfile.py This is pure python code and should work on any system.

... I now figured out that for the architectures (not MACOSX_DEPLOYMENT_TARGET), just a simple file works as well :-|

$ file parselmouth.cpython-39-darwin.so 
parselmouth.cpython-39-darwin.so: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit x86_64 bundle, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|WEAK_DEFINES|BINDS_TO_WEAK>] [arm64]

[henryiii]

The snag is that I had to install it with --no-deps, as numpy doesn't publish a universal2 wheel yet.

I would assume many users would use brew (which recently announced official AS support) instead of downloading from python.org, and NumPy should be available there. The downside is you'd have to use --system-site-packages to use the system NumPy if you make a venv. I'd love to seem NumPy include Universal2 and/or arm64 wheels!

[henryiii]

I got my M1 today. First thing I noticed: python 2.7.16 is preinstalled, and installing the command line tools adds Python 3.8.2. Neither of these actually officially support AS. And while python doesn't come with pip, python3 comes with 19.2.3.

[YannickJadoul]

python 2.7.16 is preinstalled

What the hell...

[henryiii]

Interesting, here a project is adding 3.8 (native arm64 cross-compile) wheels outside of cibuildwheel using the Xcode Python 3.8: indygreg/python-zstandard#137

[Czaki]

Maybe write them that it works currently?

[joerick]

Huh. I wonder if they have a specific use case. Btw, I can confirm that the Python 3.8 that's distributed by Apple inside the Xcode toolchain is universal:

$ /Applications/Xcode.app/Contents/Developer/usr/bin/python3 --version
Python 3.8.2
$ /Applications/Xcode.app/Contents/Developer/usr/bin/python3 -c 'import platform; print(platform.machine())'
arm64

I wonder if Apple had to make their own modifications to get it to build?

Maybe write them that it works currently?

We don't do universal builds on Python3.8, yet, because there is no official Python.org universal2 release of Python3.8.

[henryiii]

wonder if they have a specific use case

Probably supporting the default version of CPython on Big Sur? Sadly, we probably can't make this 10.9+ (or maybe even universal2 - 11.0 arm64 would not be unreasonable), and it's quite different from the normal procedure, so likely hard to integrate in. However, if we did add a "use existing Python" mode that has been thrown around before, maybe it would be possible in the future?

is universal

I'm guessing universal and not arm64 only; I noticed Ruby at least is clearly universal.

[joerick]

I hadn't realised but yeah, you're right, /usr/bin/python3 on Big Sur is a universal2 Python3.8.

$ /usr/bin/python3 -c 'import platform; print(platform.machine())'
arm64

$ file /usr/bin/python3
/usr/bin/python3: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e]
/usr/bin/python3 (for architecture x86_64):	Mach-O 64-bit executable x86_64
/usr/bin/python3 (for architecture arm64e):	Mach-O 64-bit executable arm64e

$  /usr/bin/python3 --version                                
Python 3.8.2

But yeah, I agree, it's probably not worth us supporting it, with it being such a diversion from our existing workflow.