Unhelpful error message: "Problem with the CMake installation"

[nog642]

I encountered an error when trying to install opencv-python with pip on an ARM machine:

Building wheels for collected packages: opencv-python
  Created temporary directory: /tmp/pip-wheel-r69aupoj
  Destination directory: /tmp/pip-wheel-r69aupoj
  Running command /home/username/.virtualenvs/opencv-python-build/bin/python /home/username/.virtualenvs/opencv-python-build/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py build_wheel /tmp/tmpc5qf0654
    File "/tmp/pip-build-env-bazpw8_h/overlay/lib/python3.8/site-packages/skbuild/setuptools_wrap.py", line 560, in setup
      cmkr = cmaker.CMaker(cmake_executable)
    File "/tmp/pip-build-env-bazpw8_h/overlay/lib/python3.8/site-packages/skbuild/cmaker.py", line 95, in __init__
      self.cmake_version = get_cmake_version(self.cmake_executable)
    File "/tmp/pip-build-env-bazpw8_h/overlay/lib/python3.8/site-packages/skbuild/cmaker.py", line 81, in get_cmake_version
      raise SKBuildError(
  Traceback (most recent call last):

  Problem with the CMake installation, aborting build. CMake executable is cmake
  Building wheel for opencv-python (PEP 517) ... error
  ERROR: Failed building wheel for opencv-python
Failed to build opencv-python

I tried debugging this but the error made no sense to me. I found the skbuild code that raises this error:

    try:
        version_string = subprocess.check_output([cmake_executable, '--version'])
    except (OSError, subprocess.CalledProcessError):
        raise SKBuildError(
            "Problem with the CMake installation, aborting build. CMake executable is %s" % cmake_executable)

But running the code on its own, there was no error. The cmake executable was perfectly fine. This message is all the output I was able to get, since I didn't know how to get pip to run a modified version of the code for me or something. What would really be helpful is the actual error message raised by subprocess.check_output, but I couldn't get it.

Eventually, days later, I figured out by dumb luck that pip was actually installing the cmake PyPI package, and that executable was the one that was broken (I don't know why yet; I'll have to figure it out. That's not what this issue is about.)

System info

I was running pip install opencv-python, or pip wheel . --verbose when trying to building opencv-python from source; same error. Was running this on Ubuntu 18.04 on aarch64 so it was building from source since there was no wheel available. Was running the pip command in a virtualenv (Python version 3.8.5, pip version 20.2.3). The skbuild version that pip installed automatically for the build was 0.11.1.

Solution

I think the following would work:

    try:
        version_string = subprocess.check_output([cmake_executable, '--version'])
    except (OSError, subprocess.CalledProcessError) as e:
        raise SKBuildError(
            "Problem with the CMake installation, aborting build. CMake executable is %s" % cmake_executable) from e

The from e would probably print the traceback for the original CalledProcessError as well as the SKBuildError. I'm not entirely sure it will work though, since pip seems to be catching the SKBuildError further upstream and just printing the traceback, and I have no way of testing this. I'll leave it to people who know what they're doing.

A better error message here would have saved me lots of time, and would hopefully help people in the future too.

Maybe an even better solution than the above would be if the stderr of cmake --version was printed to the console.

[skvark]

The error message is indeed very confusing since it hides the actual error. The root cause is that the cmake aarch64 wheels are broken (cmake --version segfaults).

The issue has been solved, but new wheels haven't been released yet:

• aarch64 wheel not working; cmake binary segfault cmake-python-distributions#115
• Update to CMake 3.18.4 cmake-python-distributions#118

[jcfr]

Thanks for the detailed analysis. Improving the error message makes sense. In the meantime, we will most likely finalize the release of CMake arm wheel next week.

[mikkelam]

Is there a way to resolve this error temporarily atm? I'm trying to install opencv on an arm64 device and it's not working

[skvark]

If you are using opencv-python then the answer is no, because cmake does not allow to use system CMake: scikit-build/cmake-python-distributions#80

You'll have to wait for the new cmake release or follow the official manual build guidance here: https://docs.opencv.org/master/d2/de6/tutorial_py_setup_in_ubuntu.html

[PhilMiller]

Previously, broken cross-built cmake manylinux wheels were deleted from PyPI, so that users would get a working locally-built installation. Could that be repeated for now, until the new properly-built version gets uploaded?

This is causing downstream problems with code that depends on line_profiler, because it does this horrifying mess

Collecting line_profiler
  Using cached line_profiler-3.0.2.tar.gz (45 kB)
  Installing build dependencies ... error
  ERROR: Command errored out with exit status 1:
   command: /mnt/installs/first/bin/python /mnt/installs/first/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-q2iwqun_/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=41.0.1' wheel Cython 'scikit-build>=0.9.0' cmake ninja
       cwd: None
  Complete output (50 lines):
  Collecting setuptools>=41.0.1
    Using cached setuptools-50.3.2-py3-none-any.whl (785 kB)
  Collecting wheel
    Using cached wheel-0.35.1-py2.py3-none-any.whl (33 kB)
  Collecting Cython
    Using cached Cython-0.29.21-py2.py3-none-any.whl (974 kB)
  Collecting scikit-build>=0.9.0
    Using cached scikit_build-0.11.1-py2.py3-none-any.whl (72 kB)
  Collecting cmake
    Using cached cmake-3.18.2.post1-py3-none-manylinux2014_aarch64.whl (15.2 MB)
  Collecting ninja
    Using cached ninja-1.10.0.post2.tar.gz (25 kB)
    Installing build dependencies: started
    Installing build dependencies: finished with status 'done'
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
      Preparing wheel metadata: started
      Preparing wheel metadata: finished with status 'done'
  Collecting packaging
    Using cached packaging-20.4-py2.py3-none-any.whl (37 kB)
  Collecting distro
    Using cached distro-1.5.0-py2.py3-none-any.whl (18 kB)
  Collecting six
    Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
  Collecting pyparsing>=2.0.2
    Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
  Building wheels for collected packages: ninja
    Building wheel for ninja (PEP 517): started
    Building wheel for ninja (PEP 517): finished with status 'error'
    ERROR: Command errored out with exit status 1:
     command: /mnt/installs/first/bin/python /mnt/installs/first/lib/python3.8/site-packages/pep517/_in_process.py build_wheel /tmp/tmpyphb6xch
         cwd: /tmp/pip-install-2x49rmym/ninja
    Complete output (13 lines):
    Traceback (most recent call last):
      File "/mnt/installs/first/bin/cmake", line 5, in <module>
        from cmake import cmake
    ModuleNotFoundError: No module named 'cmake'
      File "/tmp/pip-build-env-at4l9xok/overlay/lib/python3.8/site-packages/skbuild/setuptools_wrap.py", line 560, in setup
        cmkr = cmaker.CMaker(cmake_executable)
      File "/tmp/pip-build-env-at4l9xok/overlay/lib/python3.8/site-packages/skbuild/cmaker.py", line 95, in __init__
        self.cmake_version = get_cmake_version(self.cmake_executable)
      File "/tmp/pip-build-env-at4l9xok/overlay/lib/python3.8/site-packages/skbuild/cmaker.py", line 81, in get_cmake_version
        raise SKBuildError(
    Traceback (most recent call last):
  
    Problem with the CMake installation, aborting build. CMake executable is cmake
    ----------------------------------------
    ERROR: Failed building wheel for ninja
  Failed to build ninja
  ERROR: Could not build wheels for ninja which use PEP 517 and cannot be installed directly
  ----------------------------------------
ERROR: Command errored out with exit status 1: /mnt/installs/first/bin/python /mnt/installs/first/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-q2iwqun_/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=41.0.1' wheel Cython 'scikit-build>=0.9.0' cmake ninja Check the logs for full command output.