Wrong architecture installed during building cp39-macosx_arm64

[sfc-gh-mkeller]

Hi,

In this GitHub Action I'm compiling a PEP-517 compliant cp39-macosx_arm64 wheel on a x86_64 Macos worker.
However; our C-extension links against .dylib files in one of the dependencies and it appears as x86_64 variants of the build dependencies are installed. The job definition can be found here
Is there some sort of setting to force wheels of the same architecture to be installed as the target wheel's?

Any help is greatly appreciated, thank you!