add support for building windows/arm64 package #644

niyas-sait · 2022-03-28T09:20:30Z

This patch adds support for building windows/arm64 python package.

Python binary wheels can be created successfully with python setup.py bdist_wheel and passes all tests.

Test Results

python modules\python\test\test.py -v --repo .

Testing OpenCV 4.5.5
Local repo path: .
WARNING: OpenCV tests config file (opencv_python_tests.cfg) is missing, running subset of tests
Discovering python tests from: C:\users\niysai01\Workspace\opencv-python\opencv
    found 0 tests
Discovering python tests from: C:\users\niysai01\Workspace\opencv-python\opencv\modules\python\test
    found 106 tests
test_algorithm_rw (test_algorithm_rw.algorithm_rw_test) ... ok
test_async_exception (test_async.AsyncTest) ... ok
test_async_simple (test_async.AsyncTest) ... ok
....
test_umat_handle (test_umat.UMat) ... ok
test_umat_matching (test_umat.UMat) ... ok
test_umat_merge_mertens (test_umat.UMat) ... ok
test_umat_optical_flow (test_umat.UMat) ... ok
test_watershed (test_watershed.watershed_test) ... ok

----------------------------------------------------------------------
Ran 106 tests in 9.899s

setup.py

niyas-sait · 2022-03-31T19:10:17Z

CI failures look unrelated to this patch. Any idea?

asenyaev · 2022-04-01T07:52:39Z

Hi, @nsait-linaro!

It was a network issue on a runner. I've restarted jobs.

In addition, I'm going to test your changes on ARM Windows.

niyas-sait · 2022-04-05T10:55:57Z

Hi, @nsait-linaro!

It was a network issue on a runner. I've restarted jobs.

In addition, I'm going to test your changes on ARM Windows.

Thanks. Looks like passing now ! Can this be merged please ?

asmorkalov · 2022-04-06T05:27:21Z

asenyaev is working on ARM Windows setup right now to check the new platform. We will merge the patch soon, if it works well on our side too.

asenyaev · 2022-04-07T10:54:40Z

Hi @nsait-linaro!

I've tried to build opencv-python package on a Virtual Machine with Windows ARM with your changes, and a wheel was built as amd64 package.

In addition, Visual Studio docs said:

"Visual Studio is built to target processors based on the x64 architecture, and there are no versions of Visual Studio for ARM-based processors."
"For the best experience, we recommend you use Visual Studio on a separate, x64 powered, computer, and use the remote deployment and debugging features in Visual Studio to target the ARM-based device."

Could you tell me please, what is your workaround?

niyas-sait · 2022-04-07T11:03:35Z

Hi @nsait-linaro!

I've tried to build opencv-python package on a Virtual Machine with Windows ARM with your changes, and a wheel was built as amd64 package.

In addition, Visual Studio docs said:

"Visual Studio is built to target processors based on the x64 architecture, and there are no versions of Visual Studio for ARM-based processors."
"For the best experience, we recommend you use Visual Studio on a separate, x64 powered, computer, and use the remote deployment and debugging features in Visual Studio to target the ARM-based device."

Could you tell me please, what is your workaround?

Thanks, @asenyaev for trying!

Visual Studio doesn't yet have native ARM64 toolchain but that is okay as we can run in emulation and target ARM64. You will need to install few ARM64 specific components in visual studio. I have used the following but other versions probably would work as well.

C++ Universal Windows Platform support for v142 build tools (ARM64)
MSVC v142 - VS 2019 C++ x64/x86 build tools (latest)
MSVC v142 - VS 2019 C++ ARM64 build tools (latest)
MSVC v142 - VS 2019 C++ ARM64 Spectre-mitigated libs (latest)
C++ ATL for latest v142 build tools (ARM64)
C++ ATL for latest v142 build tools with Spectre Mitigations (ARM64)
Windows 10 SDK (10.0.19041.0)
Workload: Desktop development with C++ (this is essential to get a working env with vcvarsall)

Also just make sure you are using a Python (Arm64) rather than Amd64 version. Otherwise, it will generate AMD64 wheels.
Pre-built binaries are available on python.org and NuGet. Please check this link

asenyaev · 2022-04-07T15:43:13Z

Thanks, @asenyaev for trying!

Visual Studio doesn't yet have native ARM64 toolchain but that is okay as we can run in emulation and target ARM64. You will need to install few ARM64 specific components in visual studio. I have used the following but other versions probably would work as well.
* C++ Universal Windows Platform support for v142 build tools (ARM64)

* MSVC v142 - VS 2019 C++ x64/x86 build tools (latest)

* MSVC v142 - VS 2019 C++ ARM64 build tools (latest)

* MSVC v142 - VS 2019 C++ ARM64 Spectre-mitigated libs (latest)

* C++ ATL for latest v142 build tools (ARM64)

* C++ ATL for latest v142 build tools with Spectre Mitigations (ARM64)

* Windows 10 SDK (10.0.19041.0)

* Workload: Desktop development with C++ (this is essential to get a working env with vcvarsall)
Also just make sure you are using a Python (Arm64) rather than Amd64 version. Otherwise, it will generate AMD64 wheels. Pre-built binaries are available on python.org and NuGet. Please check this link

Thank you for the explanation!

I've also faced an issue with building cmake package (there are no wheels for windows arm in PyPi), it crashes immediately. How did you solve it?

niyas-sait · 2022-04-07T15:50:35Z

Thanks, @asenyaev for trying!
Visual Studio doesn't yet have native ARM64 toolchain but that is okay as we can run in emulation and target ARM64. You will need to install few ARM64 specific components in visual studio. I have used the following but other versions probably would work as well.
* C++ Universal Windows Platform support for v142 build tools (ARM64)

* MSVC v142 - VS 2019 C++ x64/x86 build tools (latest)

* MSVC v142 - VS 2019 C++ ARM64 build tools (latest)

* MSVC v142 - VS 2019 C++ ARM64 Spectre-mitigated libs (latest)

* C++ ATL for latest v142 build tools (ARM64)

* C++ ATL for latest v142 build tools with Spectre Mitigations (ARM64)

* Windows 10 SDK (10.0.19041.0)

* Workload: Desktop development with C++ (this is essential to get a working env with vcvarsall)
Also just make sure you are using a Python (Arm64) rather than Amd64 version. Otherwise, it will generate AMD64 wheels. Pre-built binaries are available on python.org and NuGet. Please check this link
Thank you for the explanation!

I've also faced an issue with building cmake package (there are no wheels for windows arm in PyPi), it crashes immediately. How did you solve it?

Did you try pip install cmake ? I used an emulated cmake - Try this

asenyaev · 2022-04-07T16:51:21Z

Did you try pip install cmake ? I used an emulated cmake - Try this

Yes, I tried it, it has been broken. Thanks for the suggestion, it works and a build started. I'll let you know the result when the build is complete.

asenyaev · 2022-04-07T17:05:12Z

@nsait-linaro, it's finally was built successfully!

Thank you for the contribution and support!

niyas-sait · 2022-04-07T19:29:32Z

@nsait-linaro, it's finally was built successfully!

Thank you for the contribution and support!

No worries ! Thanks for trying out the patch

asenyaev · 2022-04-08T18:07:33Z

I want to summarize the current status how to build opencv-python package on Windows ARM. You need to follow steps:

Firstly, install python for Windows ARM. At the moment, there are no released python installers, but worked pre-release exist (link).
Install an emulated cmake (link). Currently, you cannot install cmake with pip, there are no pre-built wheel and the build from source fails.
Install Visual Studio Build Tools 2019 marking "Individual Components":

C++ Universal Windows Platform support for v142 build tools (ARM64)
MSVC v142 - VS 2019 C++ x64/x86 build tools (latest)
MSVC v142 - VS 2019 C++ ARM64 build tools (latest)
MSVC v142 - VS 2019 C++ ARM64 Spectre-mitigated libs (latest)
C++ ATL for latest v142 build tools (ARM64)
C++ ATL for latest v142 build tools with Spectre Mitigations (ARM64)
Windows 10 SDK (10.0.19041.0)

Build opencv-python package using following command:

pip wheel . --verbose

Finally, you should have a opencv-python wheel

If you need install OpenCV python package with opencv_contrib or without UI, you have to define environment variables (check it here)

Thanks a lot to @nsait-linaro!

sid0710 · 2023-06-14T11:22:02Z

I want to summarize the current status how to build opencv-python package on Windows ARM. You need to follow steps:

Firstly, install python for Windows ARM. At the moment, there are no released python installers, but worked pre-release exist (link).

Install an emulated cmake (link). Currently, you cannot install cmake with pip, there are no pre-built wheel and the build from source fails.

Install Visual Studio Build Tools 2019 marking "Individual Components":

C++ Universal Windows Platform support for v142 build tools (ARM64)

MSVC v142 - VS 2019 C++ x64/x86 build tools (latest)

MSVC v142 - VS 2019 C++ ARM64 build tools (latest)

MSVC v142 - VS 2019 C++ ARM64 Spectre-mitigated libs (latest)

C++ ATL for latest v142 build tools (ARM64)

C++ ATL for latest v142 build tools with Spectre Mitigations (ARM64)

Windows 10 SDK (10.0.19041.0)

Build opencv-python package using following command:
pip wheel . --verbose
Finally, you should have a opencv-python wheel

If you need install OpenCV python package with opencv_contrib or without UI, you have to define environment variables (check it here)

Thanks a lot to @nsait-linaro!

I tried these steps and finally on the "pip wheel . --verbose" step, its failing -

opencv_stitching.vcxproj -> C:\Users\HCKTest\opencv-python\_skbuild\win-arm64-3.11\cmake-build\lib\Release\opencv_stitching470.lib
    Building Custom Rule C:/Users/HCKTest/opencv-python/opencv/modules/videoio/CMakeLists.txt
  Traceback (most recent call last):
    File "C:\Users\HCKTest\AppData\Local\Temp\pip-build-env-a2qnvhuj\overlay\Lib\site-packages\skbuild\setuptools_wrap.py", line 674, in setup
      cmkr.make(make_args, install_target=cmake_install_target, env=env)
    File "C:\Users\HCKTest\AppData\Local\Temp\pip-build-env-a2qnvhuj\overlay\Lib\site-packages\skbuild\cmaker.py", line 697, in make
      self.make_impl(clargs=clargs, config=config, source_dir=source_dir, install_target=install_target, env=env)
    File "C:\Users\HCKTest\AppData\Local\Temp\pip-build-env-a2qnvhuj\overlay\Lib\site-packages\skbuild\cmaker.py", line 742, in make_impl
      raise SKBuildError(msg)

  An error occurred while building with CMake.
    Command:
      'C:\Users\HCKTest\AppData\Local\Temp\pip-build-env-a2qnvhuj\overlay\Lib\site-packages\cmake\data\bin/cmake.exe' --build . --target install --config Release --
    Install target:
      install
    Source directory:
      C:\Users\HCKTest\opencv-python
    Working directory:
      C:\Users\HCKTest\opencv-python\_skbuild\win-arm64-3.11\cmake-build
  Please check the install target is valid and see CMake's output for more information.

Any guidance on how to proceed from here would be really appreciated.

asmorkalov · 2023-06-14T11:28:40Z

Actual issue is above in the console log. It's not visible in your message.

sid0710 · 2023-06-14T12:26:44Z

Sorry about that, here is the error -

C:\Users\HCKTest\opencv-python\opencv\modules\python\src2\cv2.hpp(36,10): fatal  error C1083: Cannot open include file: 'numpy/ndarrayobject.h': No such file or directory (compiling source file C:\Users\HCKTest\opencv-pytho
n\opencv\modules\python\src2\cv2_util.cpp) [C:\Users\HCKTest\opencv-python\_skbuild\win-arm64-3.11\cmake-build\modules\python3\opencv_python3.vcxproj]
C:\Users\HCKTest\opencv-python\opencv\modules\python\src2\cv2.hpp(36,10): fatal  error C1083: Cannot open include file: 'numpy/ndarrayobject.h': No such file or directory (compiling source file C:\Users\HCKTest\opencv-pytho
n\opencv\modules\python\src2\cv2_convert.cpp) [C:\Users\HCKTest\opencv-python\_skbuild\win-arm64-3.11\cmake-build\modules\python3\opencv_python3.vcxproj]
C:\Users\HCKTest\opencv-python\opencv\modules\python\src2\cv2.hpp(36,10): fatal  error C1083: Cannot open include file: 'numpy/ndarrayobject.h': No such file or directory (compiling source file C:\Users\HCKTest\opencv-pytho
n\opencv\modules\python\src2\cv2_numpy.cpp) [C:\Users\HCKTest\opencv-python\_skbuild\win-arm64-3.11\cmake-build\modules\python3\opencv_python3.vcxproj]
C:\Users\HCKTest\opencv-python\opencv\modules\python\src2\cv2.hpp(36,10): fatal  error C1083: Cannot open include file: 'numpy/ndarrayobject.h': No such file or directory (compiling source file C:\Users\HCKTest\opencv-pytho
n\opencv\modules\python\src2\cv2_highgui.cpp) [C:\Users\HCKTest\opencv-python\_skbuild\win-arm64-3.11\cmake-build\modules\python3\opencv_python3.vcxproj]
C:\Users\HCKTest\opencv-python\opencv\modules\python\src2\cv2.hpp(36,10): fatal  error C1083: Cannot open include file: 'numpy/ndarrayobject.h': No such file or directory (compiling source file C:\Users\HCKTest\opencv-pytho
n\opencv\modules\python\src2\cv2.cpp) [C:\Users\HCKTest\opencv-python\_skbuild\win-arm64-3.11\cmake-build\modules\python3\opencv_python3.vcxproj]
  Building Custom Rule C:/Users/HCKTest/opencv-python/opencv/modules/videoio/CMakeLists.txt
Traceback (most recent call last):
  File "C:\Program Files\Python311-arm64\Lib\site-packages\skbuild\setuptools_wrap.py", line 674, in setup
    cmkr.make(make_args, install_target=cmake_install_target, env=env)
  File "C:\Program Files\Python311-arm64\Lib\site-packages\skbuild\cmaker.py", line 697, in make
    self.make_impl(clargs=clargs, config=config, source_dir=source_dir, install_target=install_target, env=env)
  File "C:\Program Files\Python311-arm64\Lib\site-packages\skbuild\cmaker.py", line 742, in make_impl
    raise SKBuildError(msg)

add support for building windows/arm64 package

7419f58

asmorkalov requested changes Mar 29, 2022

View reviewed changes

setup.py Show resolved Hide resolved

setup.py Show resolved Hide resolved

asmorkalov requested a review from asenyaev March 29, 2022 05:04

asenyaev approved these changes Apr 7, 2022

View reviewed changes

asmorkalov approved these changes Apr 12, 2022

View reviewed changes

asmorkalov merged commit 5de8d66 into opencv:master Apr 12, 2022

asenyaev mentioned this pull request May 31, 2022

Update 3.4 branch with latest updates in master #669

Merged

asmorkalov mentioned this pull request Feb 25, 2023

Provide prebuilt Windows ARM64 packages #806

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add support for building windows/arm64 package #644

add support for building windows/arm64 package #644

niyas-sait commented Mar 28, 2022

niyas-sait commented Mar 31, 2022

asenyaev commented Apr 1, 2022 •

edited

niyas-sait commented Apr 5, 2022

asmorkalov commented Apr 6, 2022

asenyaev commented Apr 7, 2022

niyas-sait commented Apr 7, 2022

asenyaev commented Apr 7, 2022

niyas-sait commented Apr 7, 2022

asenyaev commented Apr 7, 2022

asenyaev commented Apr 7, 2022

niyas-sait commented Apr 7, 2022

asenyaev commented Apr 8, 2022 •

edited

sid0710 commented Jun 14, 2023

asmorkalov commented Jun 14, 2023

sid0710 commented Jun 14, 2023

add support for building windows/arm64 package #644

add support for building windows/arm64 package #644

Conversation

niyas-sait commented Mar 28, 2022

Test Results

niyas-sait commented Mar 31, 2022

asenyaev commented Apr 1, 2022 • edited

niyas-sait commented Apr 5, 2022

asmorkalov commented Apr 6, 2022

asenyaev commented Apr 7, 2022

niyas-sait commented Apr 7, 2022

asenyaev commented Apr 7, 2022

niyas-sait commented Apr 7, 2022

asenyaev commented Apr 7, 2022

asenyaev commented Apr 7, 2022

niyas-sait commented Apr 7, 2022

asenyaev commented Apr 8, 2022 • edited

sid0710 commented Jun 14, 2023

asmorkalov commented Jun 14, 2023

sid0710 commented Jun 14, 2023

asenyaev commented Apr 1, 2022 •

edited

asenyaev commented Apr 8, 2022 •

edited