[Python][macOS] 3.10 wheel is mislabeled as universal2

[DriveAthelete]

(venv) grigoryvorobiev@MacBook-Pro-Grigory pythonProject1 % python main.py                    
Traceback (most recent call last):
  File "/Users/grigoryvorobiev/PycharmProjects/pythonProject1/main.py", line 1, in <module>
    from google.cloud import translate
  File "/Users/grigoryvorobiev/PycharmProjects/pythonProject1/venv/lib/python3.10/site-packages/google/cloud/translate/__init__.py", line 17, in <module>
    from google.cloud.translate_v3.services.translation_service.client import (
  File "/Users/grigoryvorobiev/PycharmProjects/pythonProject1/venv/lib/python3.10/site-packages/google/cloud/translate_v3/__init__.py", line 17, in <module>
    from .services.translation_service import TranslationServiceClient
  File "/Users/grigoryvorobiev/PycharmProjects/pythonProject1/venv/lib/python3.10/site-packages/google/cloud/translate_v3/services/translation_service/__init__.py", line 16, in <module>
    from .client import TranslationServiceClient
  File "/Users/grigoryvorobiev/PycharmProjects/pythonProject1/venv/lib/python3.10/site-packages/google/cloud/translate_v3/services/translation_service/client.py", line 25, in <module>
    from google.api_core import gapic_v1  # type: ignore
  File "/Users/grigoryvorobiev/PycharmProjects/pythonProject1/venv/lib/python3.10/site-packages/google/api_core/gapic_v1/__init__.py", line 16, in <module>
    from google.api_core.gapic_v1 import config
  File "/Users/grigoryvorobiev/PycharmProjects/pythonProject1/venv/lib/python3.10/site-packages/google/api_core/gapic_v1/config.py", line 23, in <module>
    import grpc
  File "/Users/grigoryvorobiev/PycharmProjects/pythonProject1/venv/lib/python3.10/site-packages/grpc/__init__.py", line 22, in <module>
    from grpc import _compression
  File "/Users/grigoryvorobiev/PycharmProjects/pythonProject1/venv/lib/python3.10/site-packages/grpc/_compression.py", line 15, in <module>
    from grpc._cython import cygrpc
ImportError: dlopen(/Users/grigoryvorobiev/PycharmProjects/pythonProject1/venv/lib/python3.10/site-packages/grpc/_cython/cygrpc.cpython-310-darwin.so, 0x0002): tried: '/Users/grigoryvorobiev/PycharmProjects/pythonProject1/venv/lib/python3.10/site-packages/grpc/_cython/cygrpc.cpython-310-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/lib/cygrpc.cpython-310-darwin.so' (no such file)

The following problem: on 3.9 python everything works, but when upgrading to 3.10, the following problem an import error occurs
Are there any solutions?

[lidizheng]

According to the error details, it seems you own a Macbook with M1 chip, we currently don't have the machine to build binary wheels for them. This error is not Python version specific. On the other hand, you can try to emulate x86 architecture:

arch -x86_64 /bin/bash
# Then install Python, Python packages, and run the application

[benesch]

This error is not Python version specific.

Actually, it is. I similarly am able to install grpcio on Python 3.9 but not on Python 3.10, just like the issue reporter.

I agree that you wouldn't expect this to depend on the Python version, and indeed that gives us a clue as to what the problem is. On Python 3.9, pip install grpcio builds from source (and succeeds), while on Python 3.10, pip install grpcio downloads a precompiled wheel... for the wrong architecture. And indeed... grpcio is publishing this wheel: grpcio-1.43.0-cp310-cp310-macosx_10_10_universal2.whl .

Note the "universal2", which is incorrectly claiming that the wheel supports both x86_64 and arm64 architectures.

Not having an M1 mac is fine, but please adjust your build process to tag the wheel with x86_64 instead of universal2 so that M1 macs don't incorrectly download the wheel.

[davissclark]

This error is not Python version specific.

Actually, it is. I similarly am able to install grpcio on Python 3.9 but not on Python 3.10, just like the issue reporter.

I agree that you wouldn't expect this to depend on the Python version, and indeed that gives us a clue as to what the problem is. On Python 3.9, pip install grpcio builds from source (and succeeds), while on Python 3.10, pip install grpcio downloads a precompiled wheel... for the wrong architecture. And indeed... grpcio is publishing this wheel: grpcio-1.43.0-cp310-cp310-macosx_10_10_universal2.whl .

Note the "universal2", which is incorrectly claiming that the wheel supports both x86_64 and arm64 architectures.

Not having an M1 mac is fine, but please adjust your build process to tag the wheel with x86_64 instead of universal2 so that M1 macs don't incorrectly download the wheel.

Super annoying.

[lidizheng]

Thanks for the report. We are using the same build process for macOS wheels right now. And we uses auditwheel to make sure we are labelling the wheels correctly. It seems this tagging behavior is changed upstream for 3.10.

We might need some volunteer here to investigate why setuptools or auditwheel is tagging the wheel as universal2.

[benesch]

Doesn't auditwheel only work for Linux?

[lidizheng]

Whoops, yep, I was wrong about the auditwheel part. It's intended for manylinux:

grpc/tools/run_tests/artifacts/artifact_targets.py

Line 150 in 7cfd399

environ['GRPC_RUN_AUDITWHEEL_REPAIR'] = 'TRUE'

[gabriel-viviani]

I'm running on the same problem from using mac air m1 and I've tried multiple python versions, including isolating python with miniconda.
Although when I try to use older versions (3.9.9) to install grpcio I only can see:

[lidizheng]

The log suggests that pyenv picked Python 3.10.1.

[amitsaha]

We might need some volunteer here to investigate why setuptools or auditwheel is tagging the wheel as universal2.

I am happy to do the investigation i will post some notes when i get started.

[amitsaha]

@lidizheng if you could point me to the instructions to build and tag the wheel and hence investigate it on M1, i might be able to give you the information you are looking for.

[lidizheng]

@amitsaha All Python wheels are built by this script for Linux/macOS: https://github.com/grpc/grpc/blob/master/tools/run_tests/helper_scripts/build_python.sh

(Sorry for its messiness)

[SaneBow]

Encountered the same error here with Python 3.10 on Macbook M1.
Quick solution for users is to force pip to build the wheel: pip install --no-binary :all: grpcio --ignore-installed

[amitsaha]

@lidizheng could we update the CI scripts to use this new option added by cibuildwheel? https://github.com/pypa/cibuildwheel/blob/461bed51d3619c4c670f008148de8ce1465ba586/docs/faq.md#building-macos-wheels-for-apple-silicon--apple-silicon

[amitsaha]

@amitsaha All Python wheels are built by this script for Linux/macOS: https://github.com/grpc/grpc/blob/master/tools/run_tests/helper_scripts/build_python.sh

(Sorry for its messiness)

Environment:

• Python 3.10 virtual environment on Mac M1
• I modified the locally installed pip in the virtual environment to not delete the built wheels.

Ran:

$ git clone <grpc>
$ GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1 PYTHON=python3 ./tools/run_tests/helper_scripts/build_python.sh

I have the following wheels built now:

grpcio_health_checking-1.44.0.dev0-py3-none-any.whl
grpcio_admin-1.44.0.dev0-py3-none-any.whl
grpcio_csds-1.44.0.dev0-py3-none-any.whl
grpcio_tools-1.44.0.dev0-cp310-cp310-macosx_10_10_universal2.whl
grpcio_channelz-1.44.0.dev0-py3-none-any.whl
grpcio_testing-1.44.0.dev0-py3-none-any.whl
grpcio-1.44.0.dev0-cp310-cp310-macosx_10_10_universal2.whl
grpcio_status-1.44.0.dev0-py3-none-any.whl
grpcio_reflection-1.44.0.dev0-py3-none-any.whl
grpcio_tests-1.44.0.dev0-py3-none-any.whl

Now, i use them to build an example protobuf file:

$ cat example.proto 
syntax = "proto3";

// The greeting service definition.
service Greeter {
  // Sends a greeting
  rpc SayHello (HelloRequest) returns (HelloReply) {}
}

// The request message containing the user's name.
message HelloRequest {
  string name = 1;
}

// The response message containing the greetings
message HelloReply {
  string message = 1;
}


$ python -m grpc_tools.protoc --proto_path . --python_out=. --grpc_python_out=. example.proto

Everything works. Also for additional investigation:

$ file ./grpc_tools/_protoc_compiler.cpython-310-darwin.so

./grpc_tools/_protoc_compiler.cpython-310-darwin.so: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit bundle x86_64] [arm64]
./grpc_tools/_protoc_compiler.cpython-310-darwin.so (for architecture x86_64):	Mach-O 64-bit bundle x86_64
./grpc_tools/_protoc_compiler.cpython-310-darwin.so (for architecture arm64):	Mach-O 64-bit bundle arm64

Build the wheels in x86-64 emulation mode:

GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1 PYTHON=python3 arch -arch x86_64 ./tools/run_tests/helper_scripts/build_python.sh

The created wheels are identical - no difference in tagging.

I perform the same step above installing the new wheels and I can build the proto file successfully.

The .so file details:

 % file ./grpc_tools/_protoc_compiler.cpython-310-darwin.so
./grpc_tools/_protoc_compiler.cpython-310-darwin.so: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit bundle x86_64] [arm64]
./grpc_tools/_protoc_compiler.cpython-310-darwin.so (for architecture x86_64):	Mach-O 64-bit bundle x86_64
./grpc_tools/_protoc_compiler.cpython-310-darwin.so (for architecture arm64):	Mach-O 64-bit bundle arm64

So, it seems on my local system running M1 in X86_64 emulation mode, the tagging system is correct, since the .so file is indeed in the universal format.

I will get hold of an Intel Mac and run the above steps.

[lidizheng]

@amitsaha Can you try to run the helloworld example from the universal wheel you built? The .so might cause issue when it's loaded. https://github.com/grpc/grpc/tree/master/examples/python/helloworld

[amitsaha]

@lidizheng great idea and i see this error when using the x86_64 build:

$ python3 greeter_server.py 
Traceback (most recent call last):
  File "/Users/echorand/work/github.com/grpc/grpc/examples/python/helloworld/greeter_server.py", line 19, in <module>
    import grpc
  File "/Users/echorand/.virtualenvs/grpc-build-issue/lib/python3.10/site-packages/grpc/__init__.py", line 22, in <module>
    from grpc import _compression
  File "/Users/echorand/.virtualenvs/grpc-build-issue/lib/python3.10/site-packages/grpc/_compression.py", line 15, in <module>
    from grpc._cython import cygrpc
ImportError: dlopen(/Users/echorand/.virtualenvs/grpc-build-issue/lib/python3.10/site-packages/grpc/_cython/cygrpc.cpython-310-darwin.so, 0x0002): tried: '/Users/echorand/.virtualenvs/grpc-build-issue/lib/python3.10/site-packages/grpc/_cython/cygrpc.cpython-310-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/lib/cygrpc.cpython-310-darwin.so' (no such file)

No such error when i build them in the native arm64 mode.

[benesch]

This issue looks relevant: pypa/wheel#406

In particular:

is your CPython a universal2 one ?
If so, the default platform tag will always be universal2 when building a platform specific wheel.

pypa/wheel#406 (comment)

It seems like maybe ARCHFLAGS just needs to be set correctly?

[lidizheng]

@benesch Thank you! I'm experimenting this label, let's see if it fixes the wheel #28714.

[xinluleo]

Encountered the same error here with Python 3.10 on Macbook M1. Quick solution for users is to force pip to build the wheel: pip install --no-binary :all: grpcio --ignore-installed

Thanks. This enables me to resolve the previous error message. However, I am seeing the following error now:

File "/Users/xin/Program/Pinecone/iac/envaas/./dev.py", line 7, in <module>
    from pulumi import automation as pa
  File "/Users/xin/miniconda3/envs/iac_py3/lib/python3.10/site-packages/pulumi/__init__.py", line 33, in <module>
    from .config import (
  File "/Users/xin/miniconda3/envs/iac_py3/lib/python3.10/site-packages/pulumi/config.py", line 22, in <module>
    from . import log
  File "/Users/xin/miniconda3/envs/iac_py3/lib/python3.10/site-packages/pulumi/log.py", line 22, in <module>
    from .runtime.settings import get_engine
  File "/Users/xin/miniconda3/envs/iac_py3/lib/python3.10/site-packages/pulumi/runtime/__init__.py", line 29, in <module>
    from .mocks import (
  File "/Users/xin/miniconda3/envs/iac_py3/lib/python3.10/site-packages/pulumi/runtime/mocks.py", line 24, in <module>
    from . import rpc, rpc_manager
  File "/Users/xin/miniconda3/envs/iac_py3/lib/python3.10/site-packages/pulumi/runtime/rpc.py", line 29, in <module>
    from . import known_types, settings
  File "/Users/xin/miniconda3/envs/iac_py3/lib/python3.10/site-packages/pulumi/runtime/settings.py", line 22, in <module>
    import grpc
  File "/Users/xin/miniconda3/envs/iac_py3/lib/python3.10/site-packages/grpc/__init__.py", line 22, in <module>
    from grpc import _compression
  File "/Users/xin/miniconda3/envs/iac_py3/lib/python3.10/site-packages/grpc/_compression.py", line 15, in <module>
    from grpc._cython import cygrpc
ImportError: dlopen(/Users/xin/miniconda3/envs/iac_py3/lib/python3.10/site-packages/grpc/_cython/cygrpc.cpython-310-darwin.so, 0x0002): symbol not found in flat namespace '_CFRelease'

Tried with both 1.43.0 and 1.41.1 version grpc, got the same above error.

[hx-markterry]

I added this to my install script after my usual pip install command:

if [[ `uname -s` == "Darwin" && `uname -m` == "arm64" ]]; then
  echo "Reinstalling GRPCIO without binary support for Mac M1..."
  # Reinstall grpcio but this time ignore the wheel as it is not compatible with Mac M1
  GRPCIO_VERSION=`pip list | grep "grpcio " | cut -w -f 2`
  pip uninstall grpcio -y
  pip install --no-binary :all: grpcio==${GRPCIO_VERSION} --ignore-installed
fi

[aronchick]

just wanted to +1 @hx-markterry's steps worked perfectly for me

[gnossen]

1.47.0rc1 has been released to PyPi.. This does not yet include native arm64 wheels, but it does remove the erroneous universal2 labelled wheels. This will allow you to fall back to a from-source build when using an M1 machine without Rosetta. You can consume it with pip install grpcio --pre.

If everything goes well, you should see this in 1.47.0 in a few days. Meanwhile, I'll be working on getting our CI to produce native arm64 wheels and true universal2 artifacts.

[pietrodn]

I confirm this is solved. pip install grpcio==1.47.0rc1 on my M1 Pro MacBook Pro now downloads the source distribution, and not the broken wheel. Thank you!

[gnossen]

1.47.0 is now released to PyPi. This fixes this particular issue. Sorry for the long wait on this. Stay tuned for further work on Mac OS arm64 support.

[abax1]

Any updates on the fix to support apple silicon? This is still not working for me.

[pietrodn]

@abax1 there is no fix to support Apple Silicon AFAIK. If you're interested I made this unofficial repo with Apple Silicon builds (look in Releases)

[wildone]

Had to downgrade to 1.44.0 as current version 1.54.2 does not have a wheel for Bug Sur 11.7.4 (20G1120). I'll upgrade to 11.7.7 see if it will work without building from source.

[peggyxia-alj]

1.47.0 is now released to PyPi. This fixes this particular issue. Sorry for the long wait on this. Stay tuned for further work on Mac OS arm64 support.

Tried on grpc-1.55.1. This issue is still there on Mac M1 chips with Bazel build.

[gnossen]

@peggyxia-alj The Bazel build is not universal2 -- the artifacts labelled universal2 on pypi are. However, Bazel is a native local build, so if you build on an arm64 machine, the artifacts should be compatible with arm64.

In general, I would not recommend using our Bazel build to produce artifacts to run anywhere but on the machine which did the build.

[abasu0713]

Encountered the same error here with Python 3.10 on Macbook M1.
Quick solution for users is to force pip to build the wheel: pip install --no-binary :all: grpcio --ignore-installed

Best response in all of internet related to this issue! Concise. Uses basic understanding of how pip wheels work! It's just elegant! Thank you so much!