MacOS arch is not set for CIBW_BEFORE_BUILD

[Unmoon]

I have a Gitlab CI pipeline like following, on an M1 runner, where I had to add arch -x86_64 to CIBW_BEFORE_BUILD in order to install a package to the environment. I would expect this to not be needed.

  variables:
    CIBW_BUILD: "cp39*"
    CIBW_ARCHS: "x86_64"
    CIBW_BEFORE_BUILD: "arch -x86_64 ./before_build.sh"
  script:
    - softwareupdate --install-rosetta --agree-to-license
    - python3 -m pip install cibuildwheel
    - python3 -m cibuildwheel --output-dir wheelhouse

[joerick]

This is something broader I was thinking about raising with the other maintainers, actually. Thanks for reminding me!

Building x86_64 wheels on arm64 is currently implemented as a cross-compile, just like it is in the opposite direction. That means it's not just before-build that detects the wrong arch, many build backends will likely also mess up the cross-compile because of missing configuration somewhere.

In this one case, though, there's another solution, we can run the whole build in Rosetta emulation. Perf is generally pretty good in Rosetta (normally within 0-30% of native).

Seems like a good idea to me but maybe I'm missing something!

[Czaki]

You mean build x86 wheels?

[Czaki]

or you mean to detect such case and run subprocess with new copy of cibuildwheel under roseta?

[joerick]

Building x86_64 wheels on arm64 (Apple Silicon). No, we wouldn't need to run cibuildwheel itself in emulation, just the shell-outs to pip, virtualenv, before-build, build frontend, repair, etc.

We already do this for the testing, the tests are run under Rosetta 2.

[Czaki]

I think it is good idea

[henryiii]

I think we need a way to choose. It could be reusable if we support cross compiles for Linux eventually too.