Updated submodules to the release 4.5.4 and added python loader support #563

asenyaev · 2021-10-15T17:40:37Z

No description provided.

… updated Qt patch

alalek

Thank you!

alalek · 2021-10-15T22:33:17Z

scripts/__init__.py

@@ -0,0 +1,23 @@
+from .cv2 import *
+from .data import *


Perhaps we don't need these 2 lines

"data" submodule is imported automatically.

If we replace import cv2 to import cv2.cv2 for native_module in __init__.py, then we need first line, second can be removed.

Besides, I removed lines with a replacing, because rewrote config file, so, removed it.

alalek · 2021-10-15T22:34:18Z

scripts/__init__.py

+try:
+    from .version import ci_build, headless
+
+    ci_and_not_headless = ci_build and not headless


How I can verify that case that locally?

Update: Tried CI_BUILD=1, but it fails on missing files. So it is not easy.

Try to build with CI_BUILD=1 and also define ENABLE_HEADLESS=0, ENABLE_CONTRIB=0 and SDIST=0.

After the build you can reproduce these lines in the python interpreter. What I've done:

from cv2.version import ci_build, headless ci_and_not_headless = ci_build and not headless print (ci_and_not_headless)

In addition, to understand ci_and_not_headless working or not you can print QT_QPA_FONTDIR environment variable. Example:

import cv2, os print(os.environ["QT_QPA_FONTDIR"])

alalek · 2021-10-15T22:40:34Z

setup.py

+         % cmake_install_dir, 'r') as opencv_init:
+            opencv_init_data = ""
+            for line in opencv_init:
+                opencv_init_replacement = line.replace('importlib.import_module("cv2")', 'importlib.import_module("cv2.cv2")')


config-X.Y.py defines where we find native binary extension:

PYTHON_EXTENSIONS_PATHS = [ os.path.join(LOADER_DIR, 'python-3.9') ] + PYTHON_EXTENSIONS_PATHS

Currently it is placed to

site-packages/cv2/cv2.cpython-39-x86_64-linux-gnu.so

which is not consistent (lines have no effect)

It would work automatically with import_module("cv2")

if we move binary file to python-X.Y subdirectory (as it is located originally by CMake install)

site-packages/cv2/python-3.9/cv2.cpython-39-x86_64-linux-gnu.so

Or we replace these config lines to

PYTHON_EXTENSIONS_PATHS = [ LOADER_DIR ] + PYTHON_EXTENSIONS_PATHS

Note: We need to check/update RPATH to point properly to "Lib" directory with dependencies.

I tried the logic when cv2.cpython-39-x86_64-linux-gnu.so locates under site-packages/cv2/python-3.9/cv2.cpython-39-x86_64-linux-gnu.so, and then libs cannot work properly, because there are the wrong path to libs in config.py.

Added replacing lines into __init__.py file, to define the proper path.

alalek · 2021-10-15T22:52:14Z

setup.py

+        "cv2.gapi": [
+            "python/cv2" + r"/gapi/.*\.py"
+        ],
+        "cv2.mat_wrapper": [
+            "python/cv2" + r"/mat_wrapper/.*\.py"
+        ],
+        "cv2.misc": [
+            "python/cv2" + r"/misc/.*\.py"
+        ],
+        "cv2.utils": [
+            "python/cv2" + r"/utils/.*\.py"
+        ],


Can we copy all artifacts of "cmake install" automatically "as is"?

There is packages = ["cv2", "cv2.data"] line in this file. Perhaps it should be updated somehow (can we remove cv2.data?)

If we copy all artifacts of "cmake install" as is, then there will be several folders and files which we do not want to have in a package. We control what should be in a package using rearrange_cmake_output_data in setup.py.

What about cv2.data we have to keep it if we want to use haarcascades, because there are no __init__.py file in "cmake install" and we defining it here. However, we can write into a file after building. What do you think about anything of it?

In theory, we could specify haarcascades as package data, not a separate package. But it requires checking the code and import and so on, I'd delay it to the future.

sergregory · 2021-10-20T08:46:17Z

setup.py

+        "cv2.gapi": [
+            "python/cv2" + r"/gapi/.*\.py"
+        ],
+        "cv2.mat_wrapper": [
+            "python/cv2" + r"/mat_wrapper/.*\.py"
+        ],
+        "cv2.misc": [
+            "python/cv2" + r"/misc/.*\.py"
+        ],
+        "cv2.utils": [
+            "python/cv2" + r"/utils/.*\.py"
+        ],


In theory, we could specify haarcascades as package data, not a separate package. But it requires checking the code and import and so on, I'd delay it to the future.

sergregory · 2021-10-20T08:47:37Z

@asenyaev please also merge #565 before the release

asenyaev · 2021-10-20T08:48:13Z

@asenyaev please also merge #565 before the release

Sure, I'll do it.

alalek · 2021-10-21T17:26:31Z

opencv

https://github.com/opencv/opencv/commits/39c3334147ec02761b117f180c9c4518be18d1fa

Submodule should point to 4.5.4 tag, not on the "-dev" merge commit.

cv.__version__ => 4.5.4-dev

asenyaev · 2021-10-21T17:33:03Z

opencv

https://github.com/opencv/opencv/commits/39c3334147ec02761b117f180c9c4518be18d1fa

Submodule should point to 4.5.4 tag, not on the "-dev" merge commit.

cv.__version__ => 4.5.4-dev

As far as I remember, we always attached the latest commit for submodules. But in this case, when we have a wrong opencv version, we have to define a tag.

Related to changes made in opencv#563 $ `flake8 . --count --builtins=ml_ops --select=E9,F63,F7,F82,Y --show-source --statistics` ``` ./opencv-python/scripts/__init__.py:2:5: F821 undefined name 'LOADER_DIR' LOADER_DIR ^ ./opencv-python/scripts/__init__.py:3:5: F821 undefined name 'PYTHON_EXTENSIONS_PATHS' ] + PYTHON_EXTENSIONS_PATHS ^ ./opencv-python/scripts/__init__.py:15:4: F821 undefined name 'sys' if sys.platform.startswith("linux") and ci_and_not_headless: ^ ./opencv-python/scripts/__init__.py:16:5: F821 undefined name 'os' os.environ["QT_QPA_PLATFORM_PLUGIN_PATH"] = os.path.join( ^ ./opencv-python/scripts/__init__.py:16:49: F821 undefined name 'os' os.environ["QT_QPA_PLATFORM_PLUGIN_PATH"] = os.path.join( ^ ./opencv-python/scripts/__init__.py:17:9: F821 undefined name 'os' os.path.dirname(os.path.abspath(__file__)), "qt", "plugins" ^ ./opencv-python/scripts/__init__.py:17:25: F821 undefined name 'os' os.path.dirname(os.path.abspath(__file__)), "qt", "plugins" ^ ./opencv-python/scripts/__init__.py:21:4: F821 undefined name 'sys' if sys.platform.startswith("linux") and ci_and_not_headless: ^ ./opencv-python/scripts/__init__.py:22:5: F821 undefined name 'os' os.environ["QT_QPA_FONTDIR"] = os.path.join( ^ ./opencv-python/scripts/__init__.py:22:36: F821 undefined name 'os' os.environ["QT_QPA_FONTDIR"] = os.path.join( ^ ./opencv-python/scripts/__init__.py:23:9: F821 undefined name 'os' os.path.dirname(os.path.abspath(__file__)), "qt", "fonts" ^ ./opencv-python/scripts/__init__.py:23:25: F821 undefined name 'os' os.path.dirname(os.path.abspath(__file__)), "qt", "fonts" ^ 12 F821 undefined name 'LOADER_DIR' 12 ```

cclauss · 2021-11-26T10:40:57Z

scripts/__init__.py

+PYTHON_EXTENSIONS_PATHS = [
+    LOADER_DIR
+] + PYTHON_EXTENSIONS_PATHS


These are undefined names in Python code.
$ flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics

./scripts/__init__.py:2:5: F821 undefined name 'LOADER_DIR' LOADER_DIR ^ ./scripts/__init__.py:3:5: F821 undefined name 'PYTHON_EXTENSIONS_PATHS' ] + PYTHON_EXTENSIONS_PATHS ^

asenyaev added 2 commits October 15, 2021 19:35

Updated submodules to the release 4.5.4, added python loader support,…

6a1f61b

… updated Qt patch

Added comments for new changes

545fe24

asenyaev requested review from asmorkalov, alalek and sergregory October 15, 2021 17:40

asenyaev self-assigned this Oct 15, 2021

alalek reviewed Oct 15, 2021

View reviewed changes

egeakman mentioned this pull request Oct 16, 2021

Python 3.10 Update OpenCV #564

Closed

Removed useless lines and added replacing lines into init file

851182f

alalek approved these changes Oct 18, 2021

View reviewed changes

asmorkalov approved these changes Oct 20, 2021

View reviewed changes

sergregory approved these changes Oct 20, 2021

View reviewed changes

asenyaev merged commit a0f47e9 into opencv:master Oct 20, 2021

elmeriniemela mentioned this pull request Oct 21, 2021

Pip build fails: FileNotFoundError: [Errno 2] No such file or directory: 'scripts/__init__.py' #571

Closed

asenyaev mentioned this pull request Nov 1, 2021

Updated opencv submodule for a tag and the description #578

Merged

This was referenced Nov 25, 2021

Fix some but not all undefined names in scripts/__init__.py #587

Closed

Fix some but not all undefined names in scripts/__init__.py #589

Closed

cclauss reviewed Nov 26, 2021

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updated submodules to the release 4.5.4 and added python loader support #563

Updated submodules to the release 4.5.4 and added python loader support #563

asenyaev commented Oct 15, 2021

alalek left a comment

alalek Oct 15, 2021

asenyaev Oct 18, 2021

alalek Oct 15, 2021 •

edited

asenyaev Oct 18, 2021

alalek Oct 15, 2021 •

edited

asenyaev Oct 18, 2021

alalek Oct 15, 2021 •

edited

alalek Oct 16, 2021

asenyaev Oct 18, 2021

asenyaev Oct 18, 2021

sergregory Oct 20, 2021

sergregory Oct 20, 2021

sergregory commented Oct 20, 2021

asenyaev commented Oct 20, 2021

alalek commented Oct 21, 2021

asenyaev commented Oct 21, 2021 •

edited

cclauss Nov 26, 2021

Updated submodules to the release 4.5.4 and added python loader support #563

Updated submodules to the release 4.5.4 and added python loader support #563

Conversation

asenyaev commented Oct 15, 2021

alalek left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alalek Oct 15, 2021 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alalek Oct 15, 2021 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alalek Oct 15, 2021 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sergregory commented Oct 20, 2021

asenyaev commented Oct 20, 2021

alalek commented Oct 21, 2021

asenyaev commented Oct 21, 2021 • edited

Choose a reason for hiding this comment

alalek Oct 15, 2021 •

edited

alalek Oct 15, 2021 •

edited

alalek Oct 15, 2021 •

edited

asenyaev commented Oct 21, 2021 •

edited