When testing x86_64 on macOS arm64, CIBW_TEST_COMMAND isn't ran under Rosetta when using pipes

[Jackenmen]

Description

So basically, the problem is that the way cibuildwheel runs stuff under Rosetta is by prefixing the command with arch -x86_64. This works great for regular subprocess calls but when used with shell=True, it only prefixes the first command with it:

cibuildwheel/cibuildwheel/macos.py

Lines 497 to 508 in fb63392

	arch_prefix = []
	if testing_arch != machine_arch:
	if machine_arch == "arm64" and testing_arch == "x86_64":
	# rosetta2 will provide the emulation with just the arch prefix.
	arch_prefix = ["arch", "-x86_64"]
	else:
	msg = f"don't know how to emulate {testing_arch} on {machine_arch}"
	raise RuntimeError(msg)
	# define a custom 'call' function that adds the arch prefix each time
	call_with_arch = functools.partial(call, *arch_prefix)
	shell_with_arch = functools.partial(shell, *arch_prefix)

So when CIBW_TEST_COMMAND is set to something like false || python -c 'import spam', it doesn't work because the latter part of the command isn't run under Rosetta:

+ arch -x86_64 false || python -c 'import spam'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: dlopen(/private/var/folders/0d/08wbx2j52nx5hjg16cshb8g80000gn/T/cibw-run-qbqflk30/cp39-macosx_x86_64/venv-test/lib/python3.9/site-packages/spam.cpython-39-darwin.so, 0x0002): tried: '/private/var/folders/0d/08wbx2j52nx5hjg16cshb8g80000gn/T/cibw-run-qbqflk30/cp39-macosx_x86_64/venv-test/lib/python3.9/site-packages/spam.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have (x86_64), need (arm64e)))

I'm unsure how this should be resolved, perhaps we should have a simple python script that can run shell commands? Then it would be possible to do arch -x86_64 python -m cibuildwheel._run_shell_command "false || python -c 'import spam'" and I think Rosetta emulation would then apply to everything that's run there?

---

The linked build log and CI config are for Cirrus CI which I'm adding support for in #1191 as this is the only available CI that has M1 runners and I don't have any macOS system myself. The issue is not related to the used CI service at all though so that should not matter here.

Build log

https://cirrus-ci.com/task/4706399904071680?logs=run_cibuildwheel#L660

CI config

https://raw.githubusercontent.com/jack1142/cibuildwheel/25d1b927d828047928aea2c6fb7e9dd0eab42fc8/.cirrus.yml

[joerick]

Ah yes, good catch! It looks to me that the solution could be

cibuildwheel/cibuildwheel/macos.py

Line 508 in fb63392

shell_with_arch = functools.partial(shell, *arch_prefix)

                    shell_with_arch = functools.partial(call, *arch_prefix, "sh", "-c")