`pip install -e .` but import onnx get `No module named 'onnx.onnx_pb'` #5691

yocox · 2023-10-20T06:57:09Z

yocox
Oct 20, 2023

Hi all. I follow the instruction to build ONNX from source and install it with

pip install -e .

it builds without problem. But when I tried to validate my installation and I got

$> python -c 'import onnx'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/yoco/projects/onnx/onnx/__init__.py", line 77, in <module>
    from onnx.external_data_helper import (
  File "/home/yoco/projects/onnx/onnx/external_data_helper.py", line 11, in <module>
    from onnx.onnx_pb import AttributeProto, GraphProto, ModelProto, TensorProto
ModuleNotFoundError: No module named 'onnx.onnx_pb'

It said can not found module onnx.onnx_pb while processing external_data_helper.py. I print the sys.path and found two possible related path

/home/yoco/.venv/myenv/lib/python3.8/site-packages   # My venv package 
/home/yoco/projects/onnx   # The ONNX source root

Both of them do not have the onnx_pb file. And I found the onnx_pb file is in the build folder, which is not in the sys.path

/home/yoco/projects/onnx/build/lib.linux-x86_64-cpython-38/onnx/onnx_pb.py

I believe the file is the Python system looking for. But I don't know why my pip install -e . did not setup the path or copy the file correctly. Is there are anything I did wrong? Thanks in advance.

xadupre · 2023-10-23T11:32:04Z

xadupre
Oct 23, 2023
Maintainer

Could you run pip install -e . -v to get more information about what is going wrong? Do you have an existing version of onnx installed?

1 reply

yocox Oct 28, 2023
Author

Thank you for the reply and the suggestion. I don't have an existing version of onnx installed. I run the installation in a virtual env.

I rerun the installation with -v, and get following result. It is long, I read the log but I am not be able to find out the problem. It says the installation is completed. I do find an ./lib/python3.8/site-packages/onnx.egg-link file under my virtual env folder, the content of the egg file point to the root dir of the onnx project. But when I try to python -c 'import onnx' it dose not found the onnx.onnx_pbwhich is under thebuild` folder.

I know the log is very long. I already remove 90% unrelated content. the rest 10% is still 150 lines. Thanks for the helping again.

Non-user install because user site-packages disabled
Created temporary directory: /tmp/pip-ephem-wheel-cache-6imi515f
Created temporary directory: /tmp/pip-req-tracker-qq5grt1w
Initialized build tracking at /tmp/pip-req-tracker-qq5grt1w
Created build tracker: /tmp/pip-req-tracker-qq5grt1w
Entered build tracker: /tmp/pip-req-tracker-qq5grt1w
Created temporary directory: /tmp/pip-install-6ujdbhnn
Obtaining file:///home/yoco/projects/onnx
  Added file:///home/yoco/projects/onnx to build tracker '/tmp/pip-req-tracker-qq5grt1w'
  Created temporary directory: /tmp/pip-build-env-32i6abz9
  Installing build dependencies: started
  Running command /home/yoco/.venv/testing_onnx/bin/python /home/yoco/.venv/testing_onnx/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-32i6abz9/overlay --no-warn-script-location -v --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=61'
  Non-user install by explicit request
  Created temporary directory: /tmp/pip-ephem-wheel-cache-rwbbb0kc
  Created build tracker: /tmp/pip-req-tracker-qq5grt1w
  Entered build tracker: /tmp/pip-req-tracker-qq5grt1w
  Created temporary directory: /tmp/pip-install-dzwthram
  1 location(s) to search for versions of setuptools:
  * https://pypi.org/simple/setuptools/
  Fetching project page and analyzing links: https://pypi.org/simple/setuptools/
  Getting page https://pypi.org/simple/setuptools/
  Found index url https://pypi.org/simple
  Looking up "https://pypi.org/simple/setuptools/" in the cache
  Request header has "max_age" as 0, cache bypassed
  Starting new HTTPS connection (1): pypi.org:443
  https://pypi.org:443 "GET /simple/setuptools/ HTTP/1.1" 304 0
    Skipping link: unsupported archive format: .egg: https://files.pythonhosted.org/packages/77/38/90cae8fd99c5dc402d25930e1c91f5fcf6de5c8e982fd152a5a061e3df56/setuptools-0.6b1-py2.3.egg#sha256=ae0a6ec6090a92d08fe7f3dbf9f1b2ce889bce2a3d7724b62322a29b92cf93f0 (from https://pypi.org/simple/setuptools/)
    Skipping link: unsupported archive format: .egg: https://files.pythonhosted.org/packages/1f/ab/1ed6131b5112d1d47cb89e1cbea08f783f0ed49d6fc795ed2ff63f84d20c/setuptools-0.6b1-py2.4.egg#sha256=377ff1ce6d8e24842fbc5f7fdf1c64875f7fe9a1c4db9e7b7479cbce1377da86 (from https://pypi.org/simple/setuptools/)
    Found link https://files.pythonhosted.org/packages/c9/3d/02a14af2b413d7abf856083f327744d286f4468365cddace393a43d9d540/wheel-0.41.1.tar.gz#sha256=12b911f083e876e10c595779709f8a88a59f45aacc646492a67fe9ef796c1b47 (from https://pypi.org/simple/wheel/) (requires-python:>=3.7), version: 0.41.1
    <... skipped ...>
    Found link https://files.pythonhosted.org/packages/b8/8b/31273bf66016be6ad22bb7345c37ff350276cfd46e389a0c2ac5da9d9073/wheel-0.41.2-py3-none-any.whl#sha256=75909db2664838d015e3d9139004ee16711748a52c8f336b52882266540215d8 (from https://pypi.org/simple/wheel/) (requires-python:>=3.7), version: 0.41.2
    Found link https://files.pythonhosted.org/packages/a4/99/78c4f3bd50619d772168bec6a0f34379b02c19c9cced0ed833ecd021fd0d/wheel-0.41.2.tar.gz#sha256=0c5ac5ff2afb79ac23ab82bab027a0be7b5dbcf2e54dc50efe4bf507de1f7985 (from https://pypi.org/simple/wheel/) (requires-python:>=3.7), version: 0.41.2
  Given no hashes to check 126 links for project 'wheel': discarding no candidates
  Using version 0.41.2 (newest of versions: 0.1, 0.2, 0.3, 0.4, 0.4.1, 0.4.2, 0.5, 0.6, 0.7, 0.8, 0.9, 0.9.1, 0.9.2, 0.9.3, 0.9.4, 0.9.5, 0.9.6, 0.9.7, 0.10.0, 0.10.1, 0.10.2, 0.10.3, 0.11.0, 0.12.0, 0.13.0, 0.14.0, 0.15.0, 0.16.0, 0.17.0, 0.18.0, 0.19.0, 0.21.0, 0.22.0, 0.23.0, 0.24.0, 0.25.0, 0.26.0, 0.27.0, 0.28.0, 0.29.0, 0.30.0, 0.31.0, 0.31.1, 0.32.0, 0.32.1, 0.32.2, 0.32.3, 0.33.0, 0.33.1, 0.33.4, 0.33.5, 0.33.6, 0.34.0, 0.34.1, 0.34.2, 0.35.0, 0.35.1, 0.36.0, 0.36.1, 0.36.2, 0.37.0, 0.37.1, 0.38.0, 0.38.1, 0.38.2, 0.38.3, 0.38.4, 0.40.0, 0.41.0, 0.41.1, 0.41.2)
  Collecting wheel
    Created temporary directory: /tmp/pip-unpack-8dmy3trz
    Looking up "https://files.pythonhosted.org/packages/b8/8b/31273bf66016be6ad22bb7345c37ff350276cfd46e389a0c2ac5da9d9073/wheel-0.41.2-py3-none-any.whl" in the cache
    Current age based on date: 732947
    Ignoring unknown cache-control directive: immutable
    Freshness lifetime from max-age: 365000000
    The response is "fresh", returning cached response
    365000000 > 732947
    Using cached wheel-0.41.2-py3-none-any.whl (64 kB)
    Added wheel from https://files.pythonhosted.org/packages/b8/8b/31273bf66016be6ad22bb7345c37ff350276cfd46e389a0c2ac5da9d9073/wheel-0.41.2-py3-none-any.whl#sha256=75909db2664838d015e3d9139004ee16711748a52c8f336b52882266540215d8 to build tracker '/tmp/pip-req-tracker-qq5grt1w'
    Removed wheel from https://files.pythonhosted.org/packages/b8/8b/31273bf66016be6ad22bb7345c37ff350276cfd46e389a0c2ac5da9d9073/wheel-0.41.2-py3-none-any.whl#sha256=75909db2664838d015e3d9139004ee16711748a52c8f336b52882266540215d8 from build tracker '/tmp/pip-req-tracker-qq5grt1w'
  Installing collected packages: wheel
    Created temporary directory: /tmp/pip-unpacked-wheel-x5ycx5m7

    Creating /tmp/pip-build-env-32i6abz9/normal/bin
    changing mode of /tmp/pip-build-env-32i6abz9/normal/bin/wheel to 775
  Successfully installed wheel-0.41.2
  Cleaning up...
  Removed build tracker: '/tmp/pip-req-tracker-qq5grt1w'
  Installing backend dependencies: finished with status 'done'
    Created temporary directory: /tmp/pip-modern-metadata-svs_lvwf
    Preparing wheel metadata: started
    Running command /home/yoco/.venv/testing_onnx/bin/python /tmp/tmpz9z8ba8i prepare_metadata_for_build_wheel /tmp/tmpzhquoey9
    running dist_info
    creating /tmp/pip-modern-metadata-svs_lvwf/onnx.egg-info
    writing /tmp/pip-modern-metadata-svs_lvwf/onnx.egg-info/PKG-INFO
    writing dependency_links to /tmp/pip-modern-metadata-svs_lvwf/onnx.egg-info/dependency_links.txt
    writing entry points to /tmp/pip-modern-metadata-svs_lvwf/onnx.egg-info/entry_points.txt
    writing requirements to /tmp/pip-modern-metadata-svs_lvwf/onnx.egg-info/requires.txt
    writing top-level names to /tmp/pip-modern-metadata-svs_lvwf/onnx.egg-info/top_level.txt
    writing manifest file '/tmp/pip-modern-metadata-svs_lvwf/onnx.egg-info/SOURCES.txt'
    reading manifest file '/tmp/pip-modern-metadata-svs_lvwf/onnx.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    adding license file 'LICENSE'
    writing manifest file '/tmp/pip-modern-metadata-svs_lvwf/onnx.egg-info/SOURCES.txt'
    creating '/tmp/pip-modern-metadata-svs_lvwf/onnx-1.15.0.dist-info'
    Preparing wheel metadata: finished with status 'done'
  Source in /home/yoco/projects/onnx has version 1.15.0, which satisfies requirement onnx==1.15.0 from file:///home/yoco/projects/onnx
  Removed onnx==1.15.0 from file:///home/yoco/projects/onnx from build tracker '/tmp/pip-req-tracker-qq5grt1w'
Requirement already satisfied: numpy in /home/yoco/.venv/testing_onnx/lib/python3.8/site-packages (from onnx==1.15.0) (1.24.4)
Requirement already satisfied: protobuf>=3.20.2 in /home/yoco/.venv/testing_onnx/lib/python3.8/site-packages (from onnx==1.15.0) (4.24.4)
Installing collected packages: onnx
  Running setup.py develop for onnx
    Running command /home/yoco/.venv/testing_onnx/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/yoco/projects/onnx/setup.py'"'"'; __file__='"'"'/home/yoco/projects/onnx/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
    running develop
    running egg_info
    writing onnx.egg-info/PKG-INFO
    writing dependency_links to onnx.egg-info/dependency_links.txt
    writing entry points to onnx.egg-info/entry_points.txt
    writing requirements to onnx.egg-info/requires.txt
    writing top-level names to onnx.egg-info/top_level.txt
    reading manifest file 'onnx.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    adding license file 'LICENSE'
    writing manifest file 'onnx.egg-info/SOURCES.txt'
    running build_ext
    running cmake_build
    Using cmake args: ['/usr/bin/cmake', '-DPYTHON_INCLUDE_DIR=/usr/include/python3.8', '-DPYTHON_EXECUTABLE=/home/yoco/.venv/testing_onnx/bin/python', '-DBUILD_ONNX_PYTHON=ON', '-DCMAKE_EXPORT_COMPILE_COMMANDS=ON', '-DONNX_NAMESPACE=onnx', '-DPY_EXT_SUFFIX=.cpython-38-x86_64-linux-gnu.so', '-DCMAKE_BUILD_TYPE=Release', '-DONNX_ML=1', '/home/yoco/projects/onnx']
    -- ONNX_PROTOC_EXECUTABLE: /usr/bin/protoc
    -- Protobuf_VERSION: 3.6.1
    Generated: /home/yoco/projects/onnx/.setuptools-cmake-build/onnx/onnx-ml.proto
    Generated: /home/yoco/projects/onnx/.setuptools-cmake-build/onnx/onnx-operators-ml.proto
    Generated: /home/yoco/projects/onnx/.setuptools-cmake-build/onnx/onnx-data.proto
    -- Could NOT find pybind11 (missing: pybind11_DIR)
    -- pybind11 v2.10.4
    --
    -- ******** Summary ********
    --   CMake version                     : 3.16.3
    --   CMake command                     : /usr/bin/cmake
    --   System                            : Linux
    --   C++ compiler                      : /usr/bin/c++
    --   C++ compiler version              : 9.4.0
    --   CXX flags                         :  -Wnon-virtual-dtor
    --   Build type                        : Release
    --   Compile definitions               : __STDC_FORMAT_MACROS
    --   CMAKE_PREFIX_PATH                 :
    --   CMAKE_INSTALL_PREFIX              : /usr/local
    --   CMAKE_MODULE_PATH                 :
    --
    --   ONNX version                      : 1.15.0
    --   ONNX NAMESPACE                    : onnx
    --   ONNX_USE_LITE_PROTO               : OFF
    --   USE_PROTOBUF_SHARED_LIBS          : ON
    --   Protobuf_USE_STATIC_LIBS          : OFF
    --   ONNX_DISABLE_EXCEPTIONS           : OFF
    --   ONNX_DISABLE_STATIC_REGISTRATION  : OFF
    --   ONNX_WERROR                       : OFF
    --   ONNX_BUILD_TESTS                  : OFF
    --   ONNX_BUILD_BENCHMARKS             : OFF
    --   ONNX_BUILD_SHARED_LIBS            :
    --   BUILD_SHARED_LIBS                 :
    --
    --   Protobuf compiler                 : /usr/bin/protoc
    --   Protobuf includes                 : /usr/include
    --   Protobuf libraries                : /usr/lib/x86_64-linux-gnu/libprotobuf.so;-lpthread
    --   BUILD_ONNX_PYTHON                 : ON
    --     Python version                :
    --     Python executable             : /home/yoco/.venv/testing_onnx/bin/python
    --     Python includes               : /usr/include/python3.8
    -- Configuring done
    -- Generating done
    -- Build files have been written to: /home/yoco/projects/onnx/.setuptools-cmake-build
    [  2%] Built target gen_onnx_proto
    [  8%] Built target gen_onnx_operators_proto
    [  8%] Built target gen_onnx_data_proto
    [ 22%] Built target onnx_proto
    [ 97%] Built target onnx
    [100%] Built target onnx_cpp2py_export
    Creating /home/yoco/.venv/testing_onnx/lib/python3.8/site-packages/onnx.egg-link (link to .)
    Adding onnx 1.15.0 to easy-install.pth file
    Installing backend-test-tools script to /home/yoco/.venv/testing_onnx/bin
    Installing check-model script to /home/yoco/.venv/testing_onnx/bin
    Installing check-node script to /home/yoco/.venv/testing_onnx/bin

    Installed /home/yoco/projects/onnx
    /tmp/pip-build-env-32i6abz9/overlay/lib/python3.8/site-packages/setuptools/command/develop.py:40: EasyInstallDeprecationWarning: easy_install command is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` and ``easy_install``.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.

            See https://github.com/pypa/setuptools/issues/917 for details.
            ********************************************************************************

    !!
      easy_install.initialize_options(self)
    /tmp/pip-build-env-32i6abz9/overlay/lib/python3.8/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` directly.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.

            See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
            ********************************************************************************

    !!
      self.initialize_options()
Successfully installed onnx
Cleaning up...
Removed build tracker: '/tmp/pip-req-tracker-qq5grt1w'

MaryRodina · 2023-11-09T08:46:55Z

MaryRodina
Nov 9, 2023

Hi everyone!
I have this error to after develop installation onnx from source too
pip install -e .
Got last version of main branch from repo.
System information:
Linux dl-server-2 5.4.0-164-generic 181-Ubuntu SMP Fri Sep 1 13:41:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

0 replies

xadupre · 2023-11-09T09:46:03Z

xadupre
Nov 9, 2023
Maintainer

The installation says Successfully installed onnx. I assume it was succesfully compiled. When you do import onnx, the import system looks into a subfolder onnx if it exists then looks into packages. Even if you don't have any other installation of onnx, do you have a subfolder onnx? It can be checked by doing python -c "import onnx;print(onnx.__file__)".

0 replies

MaryRodina · 2023-11-10T06:48:54Z

MaryRodina
Nov 10, 2023

i solved this problem copied all .py files from build directory /build/lib.linux-x86_64-cpython-38/onnx/ to folder projects/onnx/onnx/ .
This is strange that this didn't done by builder/installer .

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`pip install -e .` but import onnx get `No module named 'onnx.onnx_pb'` #5691

{{title}}

Replies: 4 comments 1 reply

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

pip install -e . but import onnx get No module named 'onnx.onnx_pb' #5691

yocox Oct 20, 2023

Replies: 4 comments · 1 reply

xadupre Oct 23, 2023 Maintainer

yocox Oct 28, 2023 Author

MaryRodina Nov 9, 2023

xadupre Nov 9, 2023 Maintainer

MaryRodina Nov 10, 2023

`pip install -e .` but import onnx get `No module named 'onnx.onnx_pb'` #5691

yocox
Oct 20, 2023

Replies: 4 comments 1 reply

xadupre
Oct 23, 2023
Maintainer

yocox Oct 28, 2023
Author

MaryRodina
Nov 9, 2023

xadupre
Nov 9, 2023
Maintainer

MaryRodina
Nov 10, 2023