Build error during installation on MacOS

[markkvdb]

Describe the bug

Installing PyVRP (0.6.1) on MacOS Ventura fails unexpectedly during the build step.

Code sample

pip install pyvrp==0.6.1

Note: Please be sure you are using the latest released version of pyvrp, or a recent build of main.
If your problem has been fixed in an unreleased version, you might be able to use main until a new release occurs.

Note: If you are using a released version, have you verified that the bug exists in the master branch of this repository?

The output of the command is:

Collecting pyvrp==0.6.1
  Using cached pyvrp-0.6.1.tar.gz (398 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: matplotlib>=2.2.0 in /Users/<redacted>/.asdf/installs/python/3.10.7/lib/python3.10/site-packages (from pyvrp==0.6.1) (3.7.2)
Requirement already satisfied: numpy>=1.15.2 in /Users/<redacted>/.asdf/installs/python/3.10.7/lib/python3.10/site-packages (from pyvrp==0.6.1) (1.25.1)
Requirement already satisfied: tomli<3.0.0,>=2.0.1 in /Users/<redacted>/.asdf/installs/python/3.10.7/lib/python3.10/site-packages (from pyvrp==0.6.1) (2.0.1)
Collecting tqdm<5.0.0,>=4.64.1 (from pyvrp==0.6.1)
  Obtaining dependency information for tqdm<5.0.0,>=4.64.1 from https://files.pythonhosted.org/packages/00/e5/f12a80907d0884e6dff9c16d0c0114d81b8cd07dc3ae54c5e962cc83037e/tqdm-4.66.1-py3-none-any.whl.metadata
  Downloading tqdm-4.66.1-py3-none-any.whl.metadata (57 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.6/57.6 kB 2.6 MB/s eta 0:00:00
Collecting vrplib<2.0.0,>=1.0.0 (from pyvrp==0.6.1)
  Obtaining dependency information for vrplib<2.0.0,>=1.0.0 from https://files.pythonhosted.org/packages/23/57/269b0901cb25f97af89b03b07f4215e76e862b685aca5f14d88084e6de5d/vrplib-1.2.0-py3-none-any.whl.metadata
  Downloading vrplib-1.2.0-py3-none-any.whl.metadata (9.1 kB)
Requirement already satisfied: contourpy>=1.0.1 in /Users/<redacted>/.asdf/installs/python/3.10.7/lib/python3.10/site-packages (from matplotlib>=2.2.0->pyvrp==0.6.1) (1.1.0)
Requirement already satisfied: cycler>=0.10 in /Users/<redacted>/.asdf/installs/python/3.10.7/lib/python3.10/site-packages (from matplotlib>=2.2.0->pyvrp==0.6.1) (0.11.0)
Requirement already satisfied: fonttools>=4.22.0 in /Users/<redacted>/.asdf/installs/python/3.10.7/lib/python3.10/site-packages (from matplotlib>=2.2.0->pyvrp==0.6.1) (4.40.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /Users/<redacted>/.asdf/installs/python/3.10.7/lib/python3.10/site-packages (from matplotlib>=2.2.0->pyvrp==0.6.1) (1.4.4)
Requirement already satisfied: packaging>=20.0 in /Users/<redacted>/.asdf/installs/python/3.10.7/lib/python3.10/site-packages (from matplotlib>=2.2.0->pyvrp==0.6.1) (23.1)
Requirement already satisfied: pillow>=6.2.0 in /Users/<redacted>/.asdf/installs/python/3.10.7/lib/python3.10/site-packages (from matplotlib>=2.2.0->pyvrp==0.6.1) (10.0.0)
Requirement already satisfied: pyparsing<3.1,>=2.3.1 in /Users/<redacted>/.asdf/installs/python/3.10.7/lib/python3.10/site-packages (from matplotlib>=2.2.0->pyvrp==0.6.1) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7 in /Users/<redacted>/.asdf/installs/python/3.10.7/lib/python3.10/site-packages (from matplotlib>=2.2.0->pyvrp==0.6.1) (2.8.2)
Requirement already satisfied: six>=1.5 in /Users/<redacted>/.asdf/installs/python/3.10.7/lib/python3.10/site-packages (from python-dateutil>=2.7->matplotlib>=2.2.0->pyvrp==0.6.1) (1.16.0)
Using cached tqdm-4.66.1-py3-none-any.whl (78 kB)
Using cached vrplib-1.2.0-py3-none-any.whl (21 kB)
Building wheels for collected packages: pyvrp
  Building wheel for pyvrp (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for pyvrp (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [67 lines of output]
      The Meson build system
      Version: 1.2.1
      Source dir: /private/var/folders/f7/l5j9yjqj5410g5_7cs3gvp2h0000gn/T/pip-install-ud3ahgnd/pyvrp_b5de5dccd1d5472aa15008e5a2914ca9
      Build dir: /private/var/folders/f7/l5j9yjqj5410g5_7cs3gvp2h0000gn/T/pip-install-ud3ahgnd/pyvrp_b5de5dccd1d5472aa15008e5a2914ca9/build
      Build type: native build
      Project name: pyvrp
      Project version: 0.6.1
      
      C++ compiler for the host machine: c++ (clang 14.0.3 "Apple clang version 14.0.3 (clang-1403.0.22.14.1)")
      C++ linker for the host machine: c++ ld64 857.1
      Host machine cpu family: aarch64
      Host machine cpu: aarch64
      Program python3 found: YES (/Users/<redacted>/.asdf/installs/python/3.10.7/bin/python3.10)
      Found pkg-config: /opt/homebrew/bin/pkg-config (0.29.2)
      Run-time dependency python found: YES 3.10
      pybind11-config found: NO
      Found CMake: /opt/homebrew/bin/cmake (3.27.4)
      Run-time dependency pybind11 found: NO (tried pkgconfig, config-tool and cmake)
      Looking for a fallback subproject for the dependency pybind11
      Downloading pybind11 source from https://github.com/pybind/pybind11/archive/refs/tags/v2.10.4.tar.gz
      Downloading pybind11 patch from https://wrapdb.mesonbuild.com/v2/pybind11_2.10.4-1/get_patch
      
      Executing subproject pybind11
      
      pybind11| Project name: pybind11
      pybind11| Project version: 2.10.4
      pybind11| C++ compiler for the host machine: c++ (clang 14.0.3 "Apple clang version 14.0.3 (clang-1403.0.22.14.1)")
      pybind11| C++ linker for the host machine: c++ ld64 857.1
      pybind11| Build targets in project: 1
      pybind11| Subproject pybind11 finished.
      
      Dependency pybind11 from subproject subprojects/pybind11-2.10.4 found: YES 2.10.4
      Message: Going to build extension module _pyvrp.
      
      meson.build:97:24: ERROR: File extract_docstrings.py does not exist.
      
      A full log can be found at /private/var/folders/f7/l5j9yjqj5410g5_7cs3gvp2h0000gn/T/pip-install-ud3ahgnd/pyvrp_b5de5dccd1d5472aa15008e5a2914ca9/build/meson-logs/meson-log.txt
      Traceback (most recent call last):
        File "/private/var/folders/f7/l5j9yjqj5410g5_7cs3gvp2h0000gn/T/pip-install-ud3ahgnd/pyvrp_b5de5dccd1d5472aa15008e5a2914ca9/build_extensions.py", line 133, in <module>
          main()
        File "/private/var/folders/f7/l5j9yjqj5410g5_7cs3gvp2h0000gn/T/pip-install-ud3ahgnd/pyvrp_b5de5dccd1d5472aa15008e5a2914ca9/build_extensions.py", line 120, in main
          build(
        File "/private/var/folders/f7/l5j9yjqj5410g5_7cs3gvp2h0000gn/T/pip-install-ud3ahgnd/pyvrp_b5de5dccd1d5472aa15008e5a2914ca9/build_extensions.py", line 104, in build
          check_call(["meson", cmd, build_dir, *args])
        File "/Users/<redacted>/.asdf/installs/python/3.10.7/lib/python3.10/subprocess.py", line 369, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['meson', 'setup', PosixPath('/private/var/folders/f7/l5j9yjqj5410g5_7cs3gvp2h0000gn/T/pip-install-ud3ahgnd/pyvrp_b5de5dccd1d5472aa15008e5a2914ca9/build'), '--buildtype', 'release', '-Dpython.platlibdir=/private/var/folders/f7/l5j9yjqj5410g5_7cs3gvp2h0000gn/T/pip-install-ud3ahgnd/pyvrp_b5de5dccd1d5472aa15008e5a2914ca9', '-Dproblem=vrptw', '-Dstrip=true', '-Dprecision=integer']' returned non-zero exit status 1.
      Traceback (most recent call last):
        File "/Users/<redacted>/.asdf/installs/python/3.10.7/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/Users/<redacted>/.asdf/installs/python/3.10.7/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/Users/<redacted>/.asdf/installs/python/3.10.7/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
        File "/private/var/folders/f7/l5j9yjqj5410g5_7cs3gvp2h0000gn/T/pip-build-env-pzna9r97/overlay/lib/python3.10/site-packages/poetry/core/masonry/api.py", line 57, in build_wheel
          return WheelBuilder.make_in(
        File "/private/var/folders/f7/l5j9yjqj5410g5_7cs3gvp2h0000gn/T/pip-build-env-pzna9r97/overlay/lib/python3.10/site-packages/poetry/core/masonry/builders/wheel.py", line 87, in make_in
          wb.build(target_dir=directory)
        File "/private/var/folders/f7/l5j9yjqj5410g5_7cs3gvp2h0000gn/T/pip-build-env-pzna9r97/overlay/lib/python3.10/site-packages/poetry/core/masonry/builders/wheel.py", line 122, in build
          self._build(zip_file)
        File "/private/var/folders/f7/l5j9yjqj5410g5_7cs3gvp2h0000gn/T/pip-build-env-pzna9r97/overlay/lib/python3.10/site-packages/poetry/core/masonry/builders/wheel.py", line 171, in _build
          self._run_build_script(self._package.build_script)
        File "/private/var/folders/f7/l5j9yjqj5410g5_7cs3gvp2h0000gn/T/pip-build-env-pzna9r97/overlay/lib/python3.10/site-packages/poetry/core/masonry/builders/wheel.py", line 231, in _run_build_script
          subprocess.check_call([self.executable.as_posix(), build_script])
        File "/Users/<redacted>/.asdf/installs/python/3.10.7/lib/python3.10/subprocess.py", line 369, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['/Users/<redacted>/.asdf/installs/python/3.10.7/bin/python3.10', 'build_extensions.py']' returned non-zero exit status 1.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pyvrp
Failed to build pyvrp
ERROR: Could not build wheels for pyvrp, which is required to install pyproject.toml-based projects

Relevant software versions can be found in the output above. I tried to install PyVRP using the most recent version of Python (3.11.5) but got the same error.

Expected Output

PyVRP should be successfully built and installed.

[N-Wouda]

You're building from source, but I forgot to include extract_docstrings.py in the sdist build (in tool.poetry.include in the pyproject.toml file). My bad!

[N-Wouda]

I'll fix this in a minute and then push 0.6.2

[markkvdb]

I just looked at the prebuilt distributions on PyPi and it seems that you only provide a distribution for Intel-based MacOS. Mac's sold recently (2020-present) are all ARM-based, so it might make sense to add this to the prebuilt distributions.

[markkvdb]

I just looked at the prebuilt distributions on PyPi and it seems that you only provide a distribution for Intel-based MacOS. Mac's sold recently (2020-present) are all ARM-based, so it might make sense to add this to the prebuilt distributions.

Seems that this has to wait to at least later this year: actions/runner#805 (comment)

[N-Wouda]

v0.6.2 is building and should be on PyPI in 15min or so!

[markkvdb]

Thanks a lot!

[N-Wouda]

I just looked at the prebuilt distributions on PyPi and it seems that you only provide a distribution for Intel-based MacOS. Mac's sold recently (2020-present) are all ARM-based, so it might make sense to add this to the prebuilt distributions.

Seems that this has to wait to at least later this year: actions/runner#805 (comment)

I think we can cross-compile in OS X runs. Might be as simple as setting the appropriate variables in pyproject.toml, like here. I'll have a look at that later via #378.