arm64 support for M1 #1298
Comments
|
@redmlr In order to get M1 support I think we needs
I don't have access to M1 laptop yet, may take a look after I get one. |
|
@yongtang Thank you for answering, |
|
@redmlr The tensorflow-io package will need to be re-compiled for all functionalities to work. Though if you are only looking for functions that does not requires C++ code (only need python code) then it should be possible to just package the source tree. Which function (or list of functions) you intend to use? |
|
@yongtang I'm planning to use the following:
|
|
@redmlr With PR #1302 in you can build the wheel with: The wheel file will be located under You can then install the wheel with: That will at least allows you to use python only functions in tensorflow-io. Note Note normally the wheel file needs to pass where Note also the wheel package built from the above will not have file systems support (e.g, azure/etc) as well (file system support also requires compiled C++ kernels). |
|
Are there any updates on this front? I am interested in doing some development on TensorFlow itself, and I'm unable to install my tf-nightly wheel as there's no compatible version of tensorflow-io-gcs on pypi for arm64 macOS. I'll try setting up the System Setup: Steps Taken: The below steps are how I am building TensorFlow itself, all taken from the official documentation. This stage works without issue. $ python3.9 -m venv ./tensorflow_39_compile
$ source compile_tensorflow_39/bin/activate
$ pip install -U pip numpy wheel
$ pip install -U keras_preprocessing --no-deps
$ git clone https://github.com/tensorflow/tensorflow.git
$ cd tensorflow
$ ./configure
$ bazel build //tensorflow/tools/pip_package:build_pip_package
$ ./bazel-bin/tensorflow/tools/pip_package/build_pip_package --nightly_flag /tmp/tensorflow_pkgNow I try to install the wheel I made, and it fails $ cd /tmp/tensorflow_pkg
$ pip3 install -v tf_nightly-2.8.0-cp39-cp39-macosx_12_0_arm64.whl
Using pip 21.3.1 from /Users/tylerdavis/tensorflow_39_compile/lib/python3.9/site-packages/pip (python 3.9)
Processing ./tf_nightly-2.8.0-cp39-cp39-macosx_12_0_arm64.whl
Collecting keras-nightly~=2.8.0.dev
Using cached keras_nightly-2.8.0.dev2021110107-py2.py3-none-any.whl (1.4 MB)
Collecting gast<0.5.0,>=0.2.1
Using cached gast-0.4.0-py3-none-any.whl (9.8 kB)
Collecting astunparse>=1.6.0
Downloading astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
ERROR: Could not find a version that satisfies the requirement tensorflow-io-gcs-filesystem>=0.21.0 (from tf-nightly) (from versions: none)
ERROR: No matching distribution found for tensorflow-io-gcs-filesystem>=0.21.0
|
|
I can confirm that the below let me install my copy of tensorflow built from HEAD: # From root of io project
$ python3 setup.py -q bdist_wheel --project tensorflow_io_gcs_filesystem
$ cd dist
$ pip3 install tensorflow_io_gcs_filesystem-0.21.0-cp39-cp39-macosx_12_0_arm64.whl
$ cd /tmp/tensorflow_pkg
# Install h5py following the instructions in https://github.com/h5py/h5py/issues/1982
# as pip install h5py is currently broken on arm64 macs
$ pip3 install ./tf_nightly-2.8.0-cp39-cp39-macosx_12_0_arm64.whl |
|
I too am looking to install MacOS Monterey 12.0 Apple M1 Pro It looks like @TylerADavis has the right commands to produce the wheel. How can we publish it to PyPI? |
|
@matteosantama I think to make the above work out-of-the-box, it would need Github Actions to support ARM runners which is not yet on the roadmap. Also see actions/runner#805 Cross-compilation could work in theory I guess, but the above commands aren't for cross-compiling, and I imagine it will be a little more fiddly to get it going. |
|
No, it can't be built (pornstar) yingshaoxo@macos io % python3 setup.py -q bdist_wheel --project tensorflow_io_gcs_filesystem
Project: tensorflow-io-gcs-filesystem
Exclude: ['tests', 'tests.*', 'tensorflow_io', 'tensorflow_io.*']
Install Requires: []
Project Rootpath: tensorflow_io_gcs_filesystem
/Users/yingshaoxo/miniforge3/envs/pornstar/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
Traceback (most recent call last):
File "setup.py", line 138, in <module>
setuptools.setup(
File "/Users/yingshaoxo/miniforge3/envs/pornstar/lib/python3.8/site-packages/setuptools/__init__.py", line 155, in setup
return distutils.core.setup(**attrs)
File "/Users/yingshaoxo/miniforge3/envs/pornstar/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 148, in setup
return run_commands(dist)
File "/Users/yingshaoxo/miniforge3/envs/pornstar/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 163, in run_commands
dist.run_commands()
File "/Users/yingshaoxo/miniforge3/envs/pornstar/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 967, in run_commands
self.run_command(cmd)
File "/Users/yingshaoxo/miniforge3/envs/pornstar/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
cmd_obj.run()
File "/Users/yingshaoxo/miniforge3/envs/pornstar/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 328, in run
impl_tag, abi_tag, plat_tag = self.get_tag()
File "/Users/yingshaoxo/miniforge3/envs/pornstar/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 278, in get_tag
assert tag in supported_tags, "would build wheel with unsupported tag {}".format(tag)
AssertionError: would build wheel with unsupported tag ('cp38', 'cp38', 'macosx_11_0_arm64') |
i got same result |
What do you guys get when you run the below? xcrun --show-sdk-path && \
xcrun --show-sdk-version && \
sw_vers -productVersionEx. on my machine: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
12.1
12.1For some of my builds I had to |
same output running the above commands |
Hmmm. so still Is it still the same when running the below? |
|
|
I just built again without issue by doing the following from HEAD today ❯ python3.9 -m venv ~/io_v2
❯ source ~/io_v2/bin/activate
❯ pip3 install --upgrade pip
Requirement already satisfied: pip in /Users/tylerdavis/io_v2/lib/python3.9/site-packages (21.2.3)
Collecting pip
Downloading pip-22.0.3-py3-none-any.whl (2.1 MB)
|████████████████████████████████| 2.1 MB 2.0 MB/s
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 21.2.3
Uninstalling pip-21.2.3:
Successfully uninstalled pip-21.2.3
Successfully installed pip-22.0.3
❯ pip install wheel
Collecting wheel
Downloading wheel-0.37.1-py2.py3-none-any.whl (35 kB)
Installing collected packages: wheel
Successfully installed wheel-0.37.1
❯ python3 setup.py -q bdist_wheel --project tensorflow_io_gcs_filesystem
Project: tensorflow-io-gcs-filesystem
Exclude: ['tests', 'tests.*', 'tensorflow_io', 'tensorflow_io.*']
Install Requires: []
Project Rootpath: tensorflow_io_gcs_filesystem
❯ cd dist
❯ pip3 install tensorflow_io_gcs_filesystem-0.21.0-cp39-cp39-macosx_12_0_arm64.whl
Processing ./tensorflow_io_gcs_filesystem-0.21.0-cp39-cp39-macosx_12_0_arm64.whl
Installing collected packages: tensorflow-io-gcs-filesystem
Successfully installed tensorflow-io-gcs-filesystem-0.21.0 |
Hello, I am having a similar issue installing tensorflow-io on my M1 macbook. Initially I was failing to build the wheel file just as the other two examples in this thread, however, using the commands that you suggested here I was able to do just that. Unfortunately the installation now fails at when running: |
|
I faced the same issue as Ekhao, but when I upgraded to Python 3.9 and ran it, it worked. |
|
I was able to install |
|
Anyone have a wheel they'd care to spare? |
|
|
|
tensorflow whl for aarch64 is now available on pypi. https://pypi.org/project/tensorflow/2.10.0rc0/#files I installed this tensorflow in a container on m1 mac and confirmed that tensorflow-io bazel can be built. Buildings for AppleSilicon with MacOS are still failing due to many factors. |
It looks like github actions has m1 build in GA now! https://github.blog/changelog/2022-08-09-github-actions-self-hosted-runners-now-support-apple-m1-hardware/ Would be great to get tf io to support mac m1 now that tensorflow supports it. |
|
The torch project uses xcode's cross-compilation feature to build binaries for the Apple silicon version on an x86 mac. If we try this, it seems that we can build binary for apple silicon without preparing a self-hosted runner. |
|
This worked for me on my iMac with the latest macOS release: pip install wheel
cd tensorflow_io
python3 setup.py -q bdist_wheel --project tensorflow_io_gcs_filesystem
cd ../project
poetry add ../tensorflow_io/dist/xxxxxxx.whl |
|
@pshiko , I'm having issues with your Dockerfile. The last line is empty, so I removed it. Then it has lots of warnings while building, but it seems to work. When I run the container, the commands in the example on the readme of this project fail. This is the code in that example. And this is the error I get. If you know where is the issue or if you could help me build a container that works, I would really appreciate it. I've been trying for a while, but I'm unable to run some projects that depend on TensorFlow on my m1 Mac. |
After some further digging, I was able to get this working on my setup with
Select yes, after the prompt to uninstall. After this, you can install
You can then use the steps shown on this thread (by @TylerADavis) to install Hopefully it should solve the issue. Cheers. ;) |
|
My solution: devcontainer and VS Code. Work in Docker give it 90% of Respources and enjoy TF without the headache |
@muuvmuuv Can you share your dev container config? I'm running in a dev container on my M1 and still experiencing issues. |
|
Sure @cmditch, I have provided it as a Gist, so we can make adjustments for others :)
|
Hello, xcrun --show-sdk-path && \ Project: tensorflow-io-gcs-filesystem |
|
This was just a warning for me. I was still able to get new |
|
I was able to build on my 2020 M1 Macbook Pro. The exact command I was trying to do: New command to build: Here is the reasoning: You might need to set up or modify a pyproject.toml file or other configuration to achieve the same effect as the --project option, depending on what that option does. It's essential to consult the documentation or maintainers of the specific project (tensorflow_io_gcs_filesystem in this case) to understand how to handle custom options with the new build tools. |
|
Any update on getting this working? There's 3 versions of Apple Also, the specs on the M3 Max makes it look like one of the best machines for running this stuff. |
and others
Dear TensorFlow-io team and community,
are there any plans for adding support for arm64 machines? I use a M1 mac and I'm not able to install the wheel. Anyone who's reading this, do you know any alternatives that I can use? My goal is to create a mel-spectogram from a .wav file. ps: M1 cannot use Scipy at this time.
The text was updated successfully, but these errors were encountered: