Mac ARM builds resulting in unusable wheels #96

timkpaine · 2023-02-03T12:30:30Z

Mac ARM builds resulting in unusable wheels

pip._internal.exceptions.UnsupportedWheel: polars-0.16.2-cp37-abi3-macosx_11_0_arm64.whl is not a supported wheel on this platform.

The text was updated successfully, but these errors were encountered:

timkpaine · 2023-02-04T22:13:17Z

@conda-forge/help-osx-arm64 @conda-forge/polars any ideas?

2023-02-04T21:03:31.4138020Z Setting up cross-python
2023-02-04T21:03:34.0054460Z WARNING: The cross-compiler ('arm64-apple-darwin20.0.0-clang') does not appear to be for the correct architecture (got arm64-apple-darwin20.0.0, expected aarch64-apple-darwin20.0.0). Use --cc to correct, if necessary.
2023-02-04T21:03:35.0265600Z + [[ osx-arm64 == \l\i\n\u\x\-\6\4 ]]
2023-02-04T21:03:35.0286890Z + maturin build --release --strip --manylinux off --interpreter /Users/runner/miniforge3/conda-bld/polars_1675544201420/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/bin/python
2023-02-04T21:03:35.0290350Z Finished setting up cross-python
2023-02-04T21:03:37.9731150Z     Updating crates.io index
2023-02-04T21:04:35.7788330Z     Updating git repository `https://github.com/ritchie46/arrow2`
2023-02-04T21:04:37.9540180Z     Updating git submodule `https://github.com/apache/arrow-testing`
2023-02-04T21:04:40.8725880Z     Updating git submodule `https://github.com/apache/parquet-testing`
2023-02-04T21:04:41.6978030Z     Updating git repository `https://github.com/ritchie46/jsonpath`
2023-02-04T21:04:43.3004230Z  Downloading crates ...

< omitted >

2023-02-04T21:04:55.5169620Z 🍹 Building a mixed python/rust project
2023-02-04T21:04:55.5171240Z 🔗 Found pyo3 bindings with abi3 support for Python ≥ 3.7
2023-02-04T21:04:56.2352550Z 🐍 Not using a specific python interpreter
2023-02-04T21:04:56.2605100Z warning: /Users/runner/miniforge3/conda-bld/polars_1675544201420/work/local_dependencies/polars-sql/Cargo.toml: unused manifest key: target.cfg(not(target_os = "linux")).features
2023-02-04T21:04:56.2607470Z warning: /Users/runner/miniforge3/conda-bld/polars_1675544201420/work/local_dependencies/polars/Cargo.toml: dependency (getrandom) specified without providing a local path, Git repository, or version to use. This will be considered an error in future versions

< omitted >

2023-02-04T21:19:26.3466320Z    Compiling py-polars v0.16.2 (/Users/runner/miniforge3/conda-bld/polars_1675544201420/work)
2023-02-04T21:47:16.6864420Z     Finished release [optimized] target(s) in 42m 20s
2023-02-04T21:47:18.5664110Z 📦 Built wheel for abi3 Python ≥ 3.7 to $SRC_DIR/target/wheels/polars-0.16.2-cp37-abi3-macosx_11_0_arm64.whl
2023-02-04T21:47:18.5807140Z + /Users/runner/miniforge3/conda-bld/polars_1675544201420/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/bin/python -m pip install /Users/runner/miniforge3/conda-bld/polars_1675544201420/work/target/wheels/polars-0.16.2-cp37-abi3-macosx_11_0_arm64.whl --no-deps -vv
2023-02-04T21:47:25.6153420Z Using pip 23.0 from $BUILD_PREFIX/venv/lib/python3.11/site-packages/pip (python 3.11)
2023-02-04T21:47:25.6156000Z Non-user install because user site-packages disabled
2023-02-04T21:47:25.7194840Z Ignoring indexes: https://pypi.org/simple
2023-02-04T21:47:25.7212800Z Created temporary directory: /private/tmp/pip-build-tracker-uo69nh7p
2023-02-04T21:47:25.7216010Z Initialized build tracking at /private/tmp/pip-build-tracker-uo69nh7p
2023-02-04T21:47:25.7219190Z Created build tracker: /private/tmp/pip-build-tracker-uo69nh7p
2023-02-04T21:47:25.7222080Z Entered build tracker: /private/tmp/pip-build-tracker-uo69nh7p
2023-02-04T21:47:25.7228380Z Created temporary directory: /private/tmp/pip-install-t_inpkv1
2023-02-04T21:47:25.7253350Z Created temporary directory: /private/tmp/pip-ephem-wheel-cache-gkvc5jeb
2023-02-04T21:47:25.7742290Z ERROR: polars-0.16.2-cp37-abi3-macosx_11_0_arm64.whl is not a supported wheel on this platform.
2023-02-04T21:47:25.7865470Z Exception information:
2023-02-04T21:47:25.7936530Z Traceback (most recent call last):
2023-02-04T21:47:25.7939840Z   File "$BUILD_PREFIX/venv/lib/python3.11/site-packages/pip/_internal/cli/base_command.py", line 160, in exc_logging_wrapper
2023-02-04T21:47:25.7943790Z     status = run_func(*args)
2023-02-04T21:47:25.7945600Z              ^^^^^^^^^^^^^^^
2023-02-04T21:47:25.7947510Z   File "$BUILD_PREFIX/venv/lib/python3.11/site-packages/pip/_internal/cli/req_command.py", line 247, in wrapper
2023-02-04T21:47:25.7948960Z     return func(self, options, args)
2023-02-04T21:47:25.7950130Z            ^^^^^^^^^^^^^^^^^^^^^^^^^
2023-02-04T21:47:25.7951930Z   File "$BUILD_PREFIX/venv/lib/python3.11/site-packages/pip/_internal/commands/install.py", line 415, in run
2023-02-04T21:47:25.7953420Z     requirement_set = resolver.resolve(
2023-02-04T21:47:25.7954640Z                       ^^^^^^^^^^^^^^^^^
2023-02-04T21:47:25.7964030Z   File "$BUILD_PREFIX/venv/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 73, in resolve
2023-02-04T21:47:25.7965690Z     collected = self.factory.collect_root_requirements(root_reqs)
2023-02-04T21:47:25.7967010Z                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-02-04T21:47:25.7968960Z   File "$BUILD_PREFIX/venv/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 491, in collect_root_requirements
2023-02-04T21:47:25.7970720Z     req = self._make_requirement_from_install_req(
2023-02-04T21:47:25.7971810Z           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-02-04T21:47:25.7973690Z   File "$BUILD_PREFIX/venv/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 452, in _make_requirement_from_install_req
2023-02-04T21:47:25.7975360Z     self._fail_if_link_is_unsupported_wheel(ireq.link)
2023-02-04T21:47:25.7977440Z   File "$BUILD_PREFIX/venv/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 138, in _fail_if_link_is_unsupported_wheel
2023-02-04T21:47:25.7981130Z     raise UnsupportedWheel(msg)
2023-02-04T21:47:25.7988130Z pip._internal.exceptions.UnsupportedWheel: polars-0.16.2-cp37-abi3-macosx_11_0_arm64.whl is not a supported wheel on this platform.
2023-02-04T21:47:25.7990440Z Removed build tracker: '/private/tmp/pip-build-tracker-uo69nh7p'
2023-02-04T21:47:26.5730610Z Traceback (most recent call last):
2023-02-04T21:47:26.5733080Z   File "/Users/runner/miniforge3/bin/conda-mambabuild", line 10, in <module>
2023-02-04T21:47:26.5742620Z     sys.exit(main())
2023-02-04T21:47:26.5744310Z   File "/Users/runner/miniforge3/lib/python3.10/site-packages/boa/cli/mambabuild.py", line 256, in main
2023-02-04T21:47:26.5772360Z     call_conda_build(action, config)
2023-02-04T21:47:26.5774460Z   File "/Users/runner/miniforge3/lib/python3.10/site-packages/boa/cli/mambabuild.py", line 228, in call_conda_build
2023-02-04T21:47:26.5775720Z     result = api.build(
2023-02-04T21:47:26.5777200Z   File "/Users/runner/miniforge3/lib/python3.10/site-packages/conda_build/api.py", line 180, in build
2023-02-04T21:47:26.5797680Z     return build_tree(
2023-02-04T21:47:26.5799700Z   File "/Users/runner/miniforge3/lib/python3.10/site-packages/conda_build/build.py", line 3097, in build_tree
2023-02-04T21:47:26.5825520Z     packages_from_this = build(metadata, stats,
2023-02-04T21:47:26.5827680Z   File "/Users/runner/miniforge3/lib/python3.10/site-packages/conda_build/build.py", line 2209, in build
2023-02-04T21:47:26.5831310Z     utils.check_call_env(cmd, env=env, rewrite_stdout_env=rewrite_env,
2023-02-04T21:47:26.5833560Z   File "/Users/runner/miniforge3/lib/python3.10/site-packages/conda_build/utils.py", line 402, in check_call_env
2023-02-04T21:47:26.6661300Z     return _func_defaulting_env_to_os_environ('call', *popenargs, **kwargs)
2023-02-04T21:47:26.6663220Z   File "/Users/runner/miniforge3/lib/python3.10/site-packages/conda_build/utils.py", line 382, in _func_defaulting_env_to_os_environ
2023-02-04T21:47:26.6664290Z     raise subprocess.CalledProcessError(proc.returncode, _args)
2023-02-04T21:47:26.6665840Z subprocess.CalledProcessError: Command '['/bin/bash', '-o', 'errexit', '/Users/runner/miniforge3/conda-bld/polars_1675544201420/work/conda_build.sh']' returned non-zero exit status 1.

h-vetinari · 2023-02-05T04:59:26Z

Have you considered unpinning maturin? It's currently at 0.14.12, you're pinning to <0.14.

timkpaine · 2023-02-05T11:58:57Z

It fails on >=0.14,<0.15 #94 which matches upstream pins

arturdaraujo · 2023-02-10T11:26:40Z

Is there anything we could do on polars repository to make it easier to publish on conda-forge?

timkpaine · 2023-02-11T17:31:51Z

im turning off osx-arm64 until its more stable: #98

timkpaine · 2023-02-11T17:35:25Z

adding --target aarch64-apple-darwin to Maturin didn't do anything

timkpaine · 2023-02-11T17:36:45Z

possibly related / helpful
conda-forge/cryptography-feedstock#78
pyca/cryptography#6466
conda-forge/cryptography-feedstock#79
PyO3/maturin#835
PyO3/maturin#979
PyO3/maturin#982
https://github.com/PyO3/maturin/pull/1015/files

same issue on ruff
conda-forge/ruff-feedstock#89

erykoff · 2023-02-11T23:43:42Z

I do not know really anything about rust or maturin ... but something is going strange with the build system. First of all, I tried to build the latest version (basically the PR branch) by running a (Rosetta) x86 env on my osx-arm64 machine, and cross compiling to arm64. (This usually is a good test, but sometimes does get tripped up). I get a build failure like this:

   Compiling hashbrown v0.13.2         ] 145/358: syn, lz4-sys(build), chrono, fallible-st...
   Compiling chrono-tz v0.6.3          ] 146/358: syn, hashbrown, lz4-sys(build), chrono, ...
error[E0554]: `#![feature]` may not be used on the stable release channeltz(build.rs), lz4...
  --> $BUILD_PREFIX/.cargo/registry/src/github.com-1ecc6299db9ec823/hashbrown-0.13.2/src/lib.rs:15:5
   |
15 | /     feature(
16 | |         test,
17 | |         core_intrinsics,
18 | |         dropck_eyepatch,
...  |
25 | |         build_hasher_simple_hash_one
26 | |     )
   | |_____^

So a) is it supposed to try to build 350 packages? b) what is this feature failure, and why is it not happening elsewhere?

Meanwhile, looking at the error above what looks like is happening is the build succeeds but then something is trying to install the cross-compiled build on the build system, which is then failing because it's the wrong architecture. Is this for tests, or something about how the build system works? And then the question is how it can be used for cross compilation ...

erykoff · 2023-02-13T05:11:20Z

Okay, so the problem is that for cross-compilation you can't use maturin to build the wheels and then try to install the wheel (because the python is the wrong architecture). But you can just call the cross-python pip directly and it will call out to maturin and everything works. I've done this on #99 and it's working, but now the windows builds are failing (but maybe this is a known problem; is #95 relevant?

timkpaine added the bug Something isn't working label Feb 3, 2023

arturdaraujo mentioned this issue Feb 3, 2023

Windows builds go OOM and need to be redone #95

Closed

timkpaine mentioned this issue Feb 4, 2023

[bot-automerge] polars v0.16.2 #94

Closed

3 tasks

timkpaine closed this as completed Feb 13, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mac ARM builds resulting in unusable wheels #96

Mac ARM builds resulting in unusable wheels #96

timkpaine commented Feb 3, 2023 •

edited

timkpaine commented Feb 4, 2023 •

edited

h-vetinari commented Feb 5, 2023 •

edited

timkpaine commented Feb 5, 2023

arturdaraujo commented Feb 10, 2023

timkpaine commented Feb 11, 2023

timkpaine commented Feb 11, 2023

timkpaine commented Feb 11, 2023 •

edited

erykoff commented Feb 11, 2023

erykoff commented Feb 13, 2023

Mac ARM builds resulting in unusable wheels #96

Mac ARM builds resulting in unusable wheels #96

Comments

timkpaine commented Feb 3, 2023 • edited

timkpaine commented Feb 4, 2023 • edited

h-vetinari commented Feb 5, 2023 • edited

timkpaine commented Feb 5, 2023

arturdaraujo commented Feb 10, 2023

timkpaine commented Feb 11, 2023

timkpaine commented Feb 11, 2023

timkpaine commented Feb 11, 2023 • edited

erykoff commented Feb 11, 2023

erykoff commented Feb 13, 2023

timkpaine commented Feb 3, 2023 •

edited

timkpaine commented Feb 4, 2023 •

edited

h-vetinari commented Feb 5, 2023 •

edited

timkpaine commented Feb 11, 2023 •

edited