[PyOV] Enable Python 3.10 on Azure CI (#12578)

* Fix mypy for numpy==1.21.6 * Apply CR changes * Allow Python 3.6 for open source * Attempt to silence mypy * Revert to 3.6 in setup.pys * Fix merge conflict * Add newline * Fix Python version assert * Test Python 3.10 CI * Try bumping up pybind to 2.10 * Link default python to python 3.10 * Add sudo to changing default python * Bump python version in setupvars * Fix exception in test * Do the same for compatibility * Apply same CI changes to other pipelines * Attempt to fix arm and windows CIs * Fix win py lib version and links? * Try to fix CI * Fix win python path * Update win path * Test disable PDPD * Disable TF for testing CI * Add sudo to rm * Test linux * Update requirements * RM sudo * Add debug step to CI on ARM * build py from source on linux * Remove debug step from ARM * Fix err * Fill num_proc * Fic py installation * Bump pdpd to 2.3.1 * Bump np vers * Try to upgrade onnx to 1.12.0 * Revert onnx upgrade, enable frontends * Build py shared * Export library * activate py env * Bump scipy for py3.10 * Another scipy bump * Fix Windows path, add nfs helper for mount * Attempt to fix win_cc * Batch of changes * Try to fix ngraph-onnx-lin * Install lsb-release for docker on ngraph-onnx-lin * Link lsb_release * Try bumping Ubuntu to 22.04 * Revert ubuntu change in dockerfile * Refactor tests for onnx1.12 * Fix ONNX compatibility and numpy warnings * Refactor tests for onnx1.12 * Fix ONNX compatibility and numpy warnings * Upgrade ONNX to 1.12 in cpp * Skip unsupported ops * Revert "Fix ONNX compatibility and numpy warnings" This reverts commit cd8a0e6. * Small changes * Remove unnecessary changes * Bump Python version in github workflows * Fix flake error * Another flake error * Even more flake errors * Xfail more tests * Fix mypy errors * Change onnx version in tests * Bump ONNX in MO * Fix onnx utils in comp * Minor changes * Import pathlib * Add xfails to compatibility tests * Skip segfaulting tests * Fix onnx nametuple * Fix flake * Try to fix py3.6 onnx * Switch to venv in windows * Fix model importer and reqs * Add __test__ * Debug print * Fix flake * Update linux.yml * Update windows.yml * Update linux.yml * Update windows.yml * Install python3.10 * Update windows.yml * Update windows.yml * Update windows.yml * Update windows.yml * Update windows.yml * Update windows.yml * Update windows.yml * Update windows.yml * Install py embedded * Fix dir * Fix curl * Add complete file * CP file * Bump py ver * Fix missing venv * Try to install * Disable venv * Bump OMZ commit * checkout omz * Revert omz * Bump omz * Revert pybind bump * Update .gitmodules * Change pybind to fork * change ssize_t to size_t * Install bz2 for py build * Bump omz * Bump pandas requirement version * Revert omz * Install missing clang * Bump omz * Bump versions on fork * Bump numpy in model zoo * Fix clang * Skip segfaulting pdpd test * Skip another pdpd test * Skip pdpd places.cpp * Bump py ver in setupvars.bat * Disable paddle tests * Use int instead of str * Fix requirements path * Enable lin-cc * Fixes in CI * lin-cc and ngrpah-onnx changes * Update num_proc * Update ngraph-onnx * Minor change * Fix env var * Change ubuntu to 20 * Get compiler on ngraph-onnx * Add sudo to apt-get * Add SSL * Download ssl * Different way of installing ssl * Minor changes * install zlib * Fix env var * Fix docker building error * Revert changes * Modify dockerfile * Change pychecks python version * Cleanup * Add LayerNorm to expanded ONNX functions * Revert previous commit * Add LayerNorm to expanded ONNX functions * Apply clang * Move LayerNorm tests to skip_segfault * Unskip pdpd tests * Enable pdpd tests on linux * Refactoring * Remove 3.6 from reqs * Refactoring * Minor changes * Minor changes * Relax MO reqs * Bump torch version * relax pytest * Specify pytest version * Fix pytest pip conflict * Bump pybind11 to v2.10.1 release tag * Specify separate pytest version for py3.10 * Bump attrs * Add py dependency * Refactor versions * Add venv activation to tf tests * Fix fdupes * Bump OMZ submodule * Remove find and replace * Change pybind to upstream fork * Set pybind to proper commit * Minor changes * Try to fix scipy * Updated exclude patterns for nlohmann json * [PyOV] Fix wheel requirements for new package versions * Specify CPU plugin versions * Bump OMZ version * Fix linter * Fix linter 2 * Fix linter 3 * Disable linter Co-authored-by: Mateusz <mateusz.mikolajczyk@intel.com> Co-authored-by: jiwaszki <jan.iwaszkiewicz@intel.com> Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
openvinotoolkit · Nov 8, 2022 · d2871fd · d2871fd
1 parent b210b19
commit d2871fd
Show file tree

Hide file tree

Showing 42 changed files with 293 additions and 119 deletions.
diff --git a/.ci/azure/android_arm64.yml b/.ci/azure/android_arm64.yml
@@ -63,6 +63,9 @@ jobs:
       curl -H Metadata:true --noproxy "*" "http://169.254.169.254/metadata/instance?api-version=2019-06-01"
       whoami
       uname -a
+      echo ls /usr/bin/python3.10
+      rm -rf /usr/bin/python3
+      sudo ln -s /usr/bin/python3.10 /usr/bin/python3
       echo Python3 info ; which python3 ; python3 --version
       echo Python info ; which python ; python --version
       echo Java info ; which java ; java -version

diff --git a/.ci/azure/linux.yml b/.ci/azure/linux.yml
@@ -62,7 +62,7 @@ jobs:
         CMAKE_CPACK_GENERATOR:
         SAMPLES_INSTALL_DIR: $(INSTALL_DIR)/samples
         PYTHON_SAMPLES_INSTALL_DIR: $(SAMPLES_INSTALL_DIR)/python
-        RUN_PREFIX: . $(SETUPVARS) -pyver 3.8 &&
+        RUN_PREFIX: . $(SETUPVARS) -pyver 3.10 &&
     maxParallel: 2
 
   # About 150% of total time
@@ -92,8 +92,33 @@ jobs:
     SHARE_DIR: /mount/cinfsshare/onnxtestdata
     CCACHE_DIR: $(SHARE_DIR)/ccache/master/linux
     CMAKE_VERSION: 3.24.0
+    BUILD_PYTHON: $(WORK_DIR)/build_python
+    INSTALL_PYTHON: $(INSTALL_OPENVINO)/extras/python
+    PY_VENV: $(WORK_DIR)/.venv
+    PYTHON_VERSION: 3.10.7
 
   steps:
+  - script: |
+      curl -H Metadata:true --noproxy "*" "http://169.254.169.254/metadata/instance?api-version=2019-06-01"
+      whoami
+      uname -a
+      echo Python3 info ; which python3 ; python3 --version
+      echo Java info ; which java ; java -version
+      echo gcc info ; which gcc ; gcc --version
+      echo cmake info ; which cmake ; cmake --version
+      lsb_release
+      env
+      cat /proc/cpuinfo
+      cat /proc/meminfo
+      cat /etc/fstab
+      vmstat -s
+      df
+      lsblk -o NAME,HCTL,SIZE,MOUNTPOINT | grep -i "sd"
+      free -h
+      echo TargetBranch: $(System.PullRequest.TargetBranch)
+      echo SourceBranch: $(Build.SourceBranch)
+    displayName: 'System info'
+
   - script: |
       set -e
       rm -rf $(WORK_DIR) ; mkdir $(WORK_DIR)
@@ -130,8 +155,21 @@ jobs:
       $(REPO_DIR)/install_build_dependencies.sh
       # Move jdk into contrib
       # 'clang' compiler is to check that samples can be built using it
-      sudo apt --assume-yes install openjdk-11-jdk clang
+      sudo apt --assume-yes install openjdk-11-jdk libbz2-dev clang
       # For opencv-python: python3-setuptools and pip upgrade
+      wget https://www.python.org/ftp/python/"$PYTHON_VERSION"/Python-"$PYTHON_VERSION".tgz
+      tar -xzf Python-"$PYTHON_VERSION".tgz
+      cd Python-"$PYTHON_VERSION" || exit
+      ./configure --enable-optimizations --prefix=/usr/local --enable-shared LDFLAGS="-Wl,-rpath /usr/local/lib"
+      make python
+      sudo make install
+      export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
+      /usr/local/bin/python3.10 -m venv $(PY_VENV)
+      . $(PY_VENV)/bin/activate
+      echo Python3 info ; which python3 ; python3 --version
+      python3 --version
+      python3.10 --version
+      curl https://bootstrap.pypa.io/get-pip.py | python3
       python3 -m pip install --upgrade pip
       python3 -m pip install -r $(REPO_DIR)/src/bindings/python/src/compatibility/openvino/requirements.txt
       python3 -m pip install -r $(REPO_DIR)/src/bindings/python/wheel/requirements-dev.txt
@@ -194,7 +232,7 @@ jobs:
         -DENABLE_PYTHON=ON
         -DBUILD_SHARED_LIBS=$(CMAKE_BUILD_SHARED_LIBS)
         -DENABLE_ONEDNN_FOR_GPU=$(CMAKE_BUILD_SHARED_LIBS)
-        -DPYTHON_EXECUTABLE=/usr/bin/python3.8
+        -DPYTHON_EXECUTABLE=$(PY_VENV)/bin/python3.10
         -DENABLE_TESTS=ON
         -DENABLE_OV_ONNX_FRONTEND=ON
         -DENABLE_FASTER_BUILD=ON
@@ -234,8 +272,8 @@ jobs:
     inputs:
       cmakeArgs: >
         -GNinja
-        -S $(REPO_DIR)/tests/layer_tests
-        -B $(BUILD_LAYER_TESTS_DIR)
+        $(REPO_DIR)/tests/layer_tests
+      workingDirectory: $(BUILD_LAYER_TESTS_DIR)
 
   - script: ninja -C $(BUILD_LAYER_TESTS_DIR)
     displayName: 'Build Layer Tests'
@@ -244,26 +282,24 @@ jobs:
     displayName: 'Remove debian dependencies'
     condition: eq(variables['CMAKE_CPACK_GENERATOR'], 'DEB')
     continueOnError: false
-
   - script: cmake -DCOMPONENT=python_wheels -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P $(BUILD_DIR)/cmake_install.cmake
     displayName: 'Install wheel packages'
-
   - script: cmake -DCOMPONENT=tests -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P $(BUILD_LAYER_TESTS_DIR)/cmake_install.cmake
     displayName: 'Install Layer Tests'
-
-  - script: python3 -m pip install openvino-dev --find-links=$(INSTALL_DIR)/tools
+  - script: |
+      . $(PY_VENV)/bin/activate
+      python3 -m pip install openvino-dev --find-links=$(INSTALL_DIR)/tools
     displayName: 'Install python wheels'
-
   - script: |
       set -e
       cmake -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -DCOMPONENT=tests -P $(BUILD_DIR)/cmake_install.cmake
     displayName: 'Install tests'
-
   - script: ls -alR $(INSTALL_DIR)
     displayName: 'List install test files'
 
     # Skip test_onnx/test_zoo_models and test_onnx/test_backend due to long execution time
   - script: |
+      . $(PY_VENV)/bin/activate
       export DATA_PATH=$(MODELS_PATH)
       export MODELS_PATH=$(MODELS_PATH)
       # because of static build libgna is needed for python binary
@@ -277,6 +313,7 @@ jobs:
 
     # Skip test_onnx/test_zoo_models and test_onnx/test_backend due to long execution time
   - script: |
+      . $(PY_VENV)/bin/activate
       export DATA_PATH=$(MODELS_PATH)
       export MODELS_PATH=$(MODELS_PATH)
       # because of static build libgna is needed for python binary and mock_py frontend library
@@ -294,6 +331,7 @@ jobs:
     continueOnError: false
 
   - script: |
+      . $(PY_VENV)/bin/activate
       # because of static build libgna is needed for python binary
       export LD_LIBRARY_PATH=$(REPO_DIR)/temp/gna_03.00.00.1815.1/linux/x64:$LD_LIBRARY_PATH
       python3 -m pytest -s $(INSTALL_TEST_DIR)/mo/unit_tests --junitxml=$(INSTALL_TEST_DIR)/TEST-ModelOptimizer.xml
@@ -443,17 +481,23 @@ jobs:
     inputs:
       cmakeArgs: >
         -GNinja
-        -S $(REPO_DIR)/tests/samples_tests
-        -B $(BUILD_SAMPLES_TESTS_DIR)
+        $(REPO_DIR)/tests/samples_tests
+      workingDirectory: $(BUILD_SAMPLES_TESTS_DIR)
 
-  - script: cmake -DCOMPONENT=tests -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P $(BUILD_SAMPLES_TESTS_DIR)/cmake_install.cmake
+  - script: cmake -DCOMPONENT=tests -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P cmake_install.cmake
+    workingDirectory: $(BUILD_SAMPLES_TESTS_DIR)
     displayName: 'Install Samples Tests'
 
-  - script: python3 -m pip install -r $(INSTALL_TEST_DIR)/smoke_tests/requirements.txt
+  - script: |
+      . $(PY_VENV)/bin/activate
+      python3 -m pip install -r $(INSTALL_TEST_DIR)/smoke_tests/requirements.txt
+    workingDirectory: $(INSTALL_TEST_DIR)
+
     displayName: 'Install dependencies for samples smoke tests'
     continueOnError: false
 
   - script: |
+      . $(PY_VENV)/bin/activate
       export PATH=$HOME/.local/bin:$PATH
       export IE_APP_PATH=$(INSTALL_DIR)/samples_bin
       export LD_LIBRARY_PATH=$IE_APP_PATH:$LD_LIBRARY_PATH
@@ -465,6 +509,7 @@ jobs:
     continueOnError: false
 
   - script: |
+      . $(PY_VENV)/bin/activate
       python3 -m pip install -r $(LAYER_TESTS_DIR)/requirements.txt
       export PYTHONPATH=$(REPO_DIR)/tools/mo/:$(LAYER_TESTS_DIR):$PYTHONPATH
       export OV_FRONTEND_PATH=$(INSTALL_DIR)/runtime/lib/intel64
@@ -474,6 +519,7 @@ jobs:
     continueOnError: false
 
   - script: |
+      . $(PY_VENV)/bin/activate
       python3 -m pip install -r $(LAYER_TESTS_DIR)/requirements.txt
       export PYTHONPATH=$(REPO_DIR)/tools/mo/:$(LAYER_TESTS_DIR):$PYTHONPATH
       export OV_FRONTEND_PATH=$(INSTALL_DIR)/runtime/lib/intel64
@@ -483,13 +529,15 @@ jobs:
     continueOnError: false
 
   - script: |
+      . $(PY_VENV)/bin/activate
       python3 -m pip install -r $(LAYER_TESTS_DIR)/requirements.txt
       export PYTHONPATH=$(LAYER_TESTS_DIR):$PYTHONPATH
       $(RUN_PREFIX) python3 -m pytest $(LAYER_TESTS_DIR)/tensorflow_tests/test_tf_Roll.py --ir_version=10 --junitxml=$(INSTALL_TEST_DIR)/TEST-tf_Roll.xmlTEST
     displayName: 'TensorFlow 1 Layer Tests - Legacy FE'
     continueOnError: false
 
   - script: |
+      . $(PY_VENV)/bin/activate
       python3 -m pip install -r $(LAYER_TESTS_DIR)/requirements.txt
       export PYTHONPATH=$(LAYER_TESTS_DIR):$PYTHONPATH
       export TEST_DEVICE=CPU

diff --git a/.ci/azure/linux_arm64.yml b/.ci/azure/linux_arm64.yml
@@ -46,8 +46,8 @@ jobs:
     system.debug: true
     VSTS_HTTP_RETRY: 5
     VSTS_HTTP_TIMEOUT: 200
-    PYTHON_ARM_VERSION: "3.8.12"
-    PYTHON_EXEC: "python3.8"
+    PYTHON_ARM_VERSION: "3.10.6"
+    PYTHON_EXEC: "python3.10"
     OPENVINO_ARCH: 'aarch64'
     NUM_PROC: 1
     BUILD_TYPE: Release
@@ -79,7 +79,9 @@ jobs:
       curl -H Metadata:true --noproxy "*" "http://169.254.169.254/metadata/instance?api-version=2019-06-01"
       whoami
       uname -a
-      echo Python3 info ; which python3 ; python3 --version
+      echo ls /usr/bin/python3.10
+      rm -rf /usr/bin/python3
+      sudo ln -s /usr/bin/python3.10 /usr/bin/python3      echo Python3 info ; which python3 ; python3 --version
       echo Python info ; which python ; python --version
       echo Java info ; which java ; java -version
       echo gcc info ; which gcc ; gcc --version
@@ -161,10 +163,10 @@ jobs:
         -DCMAKE_VERBOSE_MAKEFILE=ON
         -DCMAKE_COMPILE_WARNING_AS_ERROR=OFF
         -DOpenCV_DIR=$(INSTALL_OPENCV)/cmake
-        -DPYTHON_INCLUDE_DIRS=$(INSTALL_PYTHON)/include/python3.8
-        -DPYTHON_LIBRARY=$(INSTALL_PYTHON)/lib/libpython3.8.so
+        -DPYTHON_INCLUDE_DIRS=$(INSTALL_PYTHON)/include/python3.10
+        -DPYTHON_LIBRARY=$(INSTALL_PYTHON)/lib/libpython3.10.so
         -DENABLE_PYTHON=ON
-        -DPYTHON_MODULE_EXTENSION=".cpython-38-x86_64-linux-gnu.so"
+        -DPYTHON_MODULE_EXTENSION=".cpython-310-x86_64-linux-gnu.so"
         -DENABLE_TESTS=ON
         -DENABLE_DATA=OFF
         -DCMAKE_EXE_LINKER_FLAGS=-Wl,-rpath-link,$(INSTALL_OPENCV)/lib
@@ -207,10 +209,10 @@ jobs:
         -DOpenVINODeveloperPackage_DIR=$(BUILD_OPENVINO)
         -DENABLE_PYTHON=ON
         -DTHREADING=TBB
-        -DPYTHON_EXECUTABLE=$(INSTALL_PYTHON)/bin/python3.8
-        -DPYTHON_INCLUDE_DIRS=$(INSTALL_PYTHON)/include/python3.8
+        -DPYTHON_EXECUTABLE=$(INSTALL_PYTHON)/bin/python3.10
+        -DPYTHON_INCLUDE_DIRS=$(INSTALL_PYTHON)/include/python3.10
         -DPYTHON_LIBRARIES=$(INSTALL_PYTHON)/lib
-        -DPYTHON3_NUMPY_INCLUDE_DIRS=/usr/local/lib/python3.8/site-packages/numpy/core/include
+        -DPYTHON3_NUMPY_INCLUDE_DIRS=/usr/local/lib/python3.10/site-packages/numpy/core/include
         -DPYTHON_MODULE_EXTENSION=".so"
         -DPYBIND11_FINDPYTHON=OFF
         -DPYBIND11_NOPYTHON=OFF
@@ -248,8 +250,8 @@ jobs:
         -GNinja
         -DCMAKE_BUILD_TYPE=$(BUILD_TYPE)
         -DENABLE_PYTHON=ON
-        -DPYTHON_EXECUTABLE=/usr/local/bin/python3.8
-        -DPYTHON_INCLUDE_DIR=$(INSTALL_PYTHON)/include/python3.8
+        -DPYTHON_EXECUTABLE=/usr/local/bin/python3.10
+        -DPYTHON_INCLUDE_DIR=$(INSTALL_PYTHON)/include/python3.10
         -DPYTHON_LIBRARY=$(INSTALL_PYTHON)/lib
         -DCMAKE_TOOLCHAIN_FILE=$(OPENVINO_REPO_DIR)/cmake/arm64.toolchain.cmake
         -DOpenVINO_DIR=$(BUILD_OPENVINO)

diff --git a/.ci/azure/linux_conditional_compilation.yml b/.ci/azure/linux_conditional_compilation.yml
@@ -79,6 +79,19 @@ jobs:
   - script: |
       set -e
       $(REPO_DIR)/install_build_dependencies.sh
+      wget https://www.python.org/ftp/python/3.10.6/Python-3.10.6.tgz
+      tar -xzf Python-3.10.6.tgz
+      cd Python-3.10.6 || exit
+      ./configure --enable-optimizations --prefix=/usr/local --enable-shared LDFLAGS="-Wl,-rpath /usr/local/lib"
+      make -j 1 python
+      sudo make -j 1 install
+      export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
+      /usr/local/bin/python3.10 -m venv $(WORK_DIR)/.venv
+      . $(WORK_DIR)/.venv/bin/activate
+      echo Python3 info ; which python3 ; python3 --version
+      python3 --version
+      python3.10 --version
+      curl https://bootstrap.pypa.io/get-pip.py | python3
       python3 -m pip install -r $(REPO_DIR)/src/bindings/python/src/compatibility/openvino/requirements.txt
       # Speed up build
       wget https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-linux.zip
@@ -97,6 +110,7 @@ jobs:
         -DENABLE_FASTER_BUILD=ON
         -DENABLE_PROFILING_ITT=ON
         -DSELECTIVE_BUILD=COLLECT
+        -DPYTHON_EXECUTABLE=$(WORK_DIR)/.venv/bin/python3.10
         $(REPO_DIR)
       workingDirectory: $(BUILD_DIR)
 

diff --git a/.ci/azure/linux_coverity.yml b/.ci/azure/linux_coverity.yml
@@ -35,6 +35,9 @@ jobs:
       curl -H Metadata:true --noproxy "*" "http://169.254.169.254/metadata/instance?api-version=2019-06-01"
       whoami
       uname -a
+      echo ls /usr/bin/python3.10
+      rm -rf /usr/bin/python3
+      sudo ln -s /usr/bin/python3.10 /usr/bin/python3
       echo Python3 info ; which python3 ; python3 --version
       echo Python info ; which python ; python --version
       echo Java info ; which java ; java -version

diff --git a/.ci/azure/linux_ngraph_onnx.yml b/.ci/azure/linux_ngraph_onnx.yml
@@ -80,6 +80,7 @@ jobs:
       rm -rf $(WORK_DIR) ; mkdir $(WORK_DIR)
       sudo mkdir -p $(MODELS_DIR)
       sudo apt --assume-yes update && sudo apt --assume-yes install nfs-common
+      sudo apt install nfs-common -y
       sudo mount -vvv -t nfs cinfsshare.file.core.windows.net:/cinfsshare/onnxtestdata $(MODELS_DIR) -o vers=4,minorversion=1,sec=sys
       mkdir -p $(MODELS_DIR)/models_data
     displayName: 'Make dirs'
@@ -92,17 +93,36 @@ jobs:
 
   - script: |
       set -e
+      apt-get update && apt-get install -y lsb-release && apt-get clean all
+      sudo apt-get -y install libssl-dev zlib1g-dev
+      sudo apt-get -y install build-essential
+      wget https://www.python.org/ftp/python/3.10.7/Python-3.10.7.tgz
+      tar -xzf Python-3.10.7.tgz
+      cd Python-3.10.7 || exit
+      ./configure --enable-optimizations --prefix=/usr/local --enable-shared LDFLAGS="-Wl,-rpath /usr/local/lib"
+      make python
+      sudo make install
+      export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
+      /usr/local/bin/python3.10 -m venv $(WORK_DIR)/.venv
+      . $(WORK_DIR)/.venv/bin/activate
+      echo Python3 info ; which python3 ; python3 --version
+      python3 --version
+      python3.10 --version
+      curl https://bootstrap.pypa.io/get-pip.py | python3
       sudo apt --assume-yes install git-lfs uidmap
       curl -fsSL https://get.docker.com -o get-docker.sh
       sudo sh get-docker.sh
     workingDirectory: $(WORK_DIR)
     displayName: 'Install dependencies'
 
-  - script: src/bindings/python/tests/test_onnx/model_zoo_preprocess.sh -d $(MODELS_DIR)/models_data -o -s "$(ONNX_MODEL_ZOO_SHA)"
+  - script:
+      . $(WORK_DIR)/.venv/bin/activate
+      src/bindings/python/tests/test_onnx/model_zoo_preprocess.sh -d $(MODELS_DIR)/models_data -o -s "$(ONNX_MODEL_ZOO_SHA)"
     displayName: 'Update models'
     condition: ne(variables['BUILD_TYPE'], 'Debug')
 
-  - script: sudo docker build --tag=openvino-onnx-ci-image --file=.ci/openvino-onnx/Dockerfile --build-arg BUILD_TYPE=$(BUILD_TYPE) --build-arg PROTOBUF_LITE=$(PROTOBUF_LITE) .
+  - script: 
+      sudo docker build --tag=openvino-onnx-ci-image --file=.ci/openvino-onnx/Dockerfile --build-arg BUILD_TYPE=$(BUILD_TYPE) --build-arg PROTOBUF_LITE=$(PROTOBUF_LITE) .
     displayName: 'Docker build $(BUILD_TYPE)'
 
   - script: sudo fallocate -l 64G /swapfile ; sudo mkswap /swapfile ; sudo swapon /swapfile ; df ; free -h