Raspbian bookworm: undefined symbol EVP_idea_cbc

[tkundrat]

• Versions of Python, cryptography, cffi, pip, and setuptools
  you're using
  • cryptography: 42.0.5
  • cffi: 1.16.0
  • pip: 24.0
  • setuptools: 69.5.1
• How you installed cryptography
  • pip3 install homeassistant (version 2024.4.4), which has a dependency for cryptography==42.0.5
• Clear steps for reproducing your bug
  • Create venv with Python 3.12.2 (compiled locally from source as raspbian bookworm only ships 3.11.6)
  • source bin/activate inside the venv directory
  • pip3 install homeassistant
  • run hass inside the venv
  • Traceback:

Traceback (most recent call last):
  File "/srv/homeassistant/bin/hass", line 8, in <module>                                                                                                                
    sys.exit(main())                                                                
             ^^^^^^                                                                                                                                                      
  File "/srv/homeassistant/lib/python3.12/site-packages/homeassistant/__main__.py", line 174, in main
    args = get_arguments()                                                                                                                                               
           ^^^^^^^^^^^^^^^                                                          
  File "/srv/homeassistant/lib/python3.12/site-packages/homeassistant/__main__.py", line 82, in get_arguments 
    from . import config as config_util
  File "/srv/homeassistant/lib/python3.12/site-packages/homeassistant/config.py", line 27, in <module>
    from . import auth
  File "/srv/homeassistant/lib/python3.12/site-packages/homeassistant/auth/__init__.py", line 13, in <module>
    import jwt
  File "/srv/homeassistant/lib/python3.12/site-packages/jwt/__init__.py", line 1, in <module>
    from .api_jwk import PyJWK, PyJWKSet
  File "/srv/homeassistant/lib/python3.12/site-packages/jwt/api_jwk.py", line 7, in <module>
    from .algorithms import get_default_algorithms, has_crypto, requires_cryptography
  File "/srv/homeassistant/lib/python3.12/site-packages/jwt/algorithms.py", line 12, in <module>
    from .utils import (
  File "/srv/homeassistant/lib/python3.12/site-packages/jwt/utils.py", line 7, in <module>
    from cryptography.hazmat.primitives.asymmetric.ec import EllipticCurve
  File "/srv/homeassistant/lib/python3.12/site-packages/cryptography/hazmat/primitives/asymmetric/ec.py", line 11, in <module>
    from cryptography.hazmat._oid import ObjectIdentifier
  File "/srv/homeassistant/lib/python3.12/site-packages/cryptography/hazmat/_oid.py", line 7, in <module>
    from cryptography.hazmat.bindings._rust import (
ImportError: /srv/homeassistant/lib/python3.12/site-packages/cryptography/hazmat/bindings/_rust.cpython-312-arm-linux-gnueabihf.so: undefined symbol: EVP_idea_cbc, version OPENSSL_3.0.0

I'm using OpenSSL 3.0.11 19 Sep 2023 (Library: OpenSSL 3.0.11 19 Sep 2023).
Rust compiler:

stable-arm-unknown-linux-gnueabihf (default)
rustc 1.77.2 (25ef9e3d8 2024-04-09)

OS: Raspbian GNU/Linux 12 (bookworm) aarch64

It seems the mentioned .so file does not contain the symbol (I checked with nm -gD), so maybe the wrong openssl is used? I just installed the official packaged version via sudo apt install openssl.

[alex]

When you installed cryptography, where did the installation come from -- https://www.piwheels.org/ or did you build it yourself?

[tkundrat]

cryptography was compiled during pip install

[alex]

Hmm. Can you re-build cryptography with verbose logs and paste the output -- something like pip install -vvv cryptography --no-binary cryptography

[tkundrat]

I rebuilt the package using pip3 install --force-reinstall --no-cache-dir cryptography -vvv cryptography --no-binary cryptography, the output I piped into this file:
verbose.log

[alex]

Hmm, I'm a bit perplexed. I can see in the log that CRYPTOGRAPHY_OSSLCONF="OPENSSL_NO_IDEA" is being passed to our build, as expected. And we guard usage of IDEA with: #[cfg(not(CRYPTOGRAPHY_OSSLCONF = "OPENSSL_NO_IDEA"))]

So I'm not sure why it would be that it's trying to link this symbol.

[tkundrat]

On my currently running Home Assistant installation, I'm running the same cryptography version with Python 3.11.6 in a venv. There everything is working fine. The difference is only the cffi and setuptools version, could there be an issue?
I set up a new venv for this issue for testing under /home/pi/test-venv/ , but I did the same steps as for the actual installation under /srv/homeassistant/ , so there should not be an issue.
Is there something else I can try?

[alex]

cffi/setuptools version shouldn't impact this.

…

On Wed, Apr 24, 2024 at 4:08 PM Tristan Kundrat ***@***.***> wrote: On my currently running Home Assistant installation, I'm running the same cryptography version with Python 3.11.6 in a venv. There everything is working fine. The difference is only the cffi and setuptools version, could there be an issue? I set up a new venv for this issue for testing under /home/pi/test-venv/ , but I did the same steps as for the actual installation under /srv/homeassistant/ , so there should not be an issue. Is there something else I can try? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

-- All that is necessary for evil to succeed is for good people to do nothing.

[alex]

If you install from git (pip install git+https://github.com/pyca/cryptography) does this problem still occur?

[tkundrat]

There are some version constraints I had to violate:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency confl
icts.                                                                                                                                                                    
pyopenssl 24.1.0 requires cryptography<43,>=41.0.5, but you have cryptography 43.0.0.dev1 which is incompatible.                                                         
homeassistant 2024.4.4 requires cryptography==42.0.5, but you have cryptography 43.0.0.dev1 which is incompatible.

I now ran hass --skip-pip-packages cryptography, so cryptography 42.0.5 does not get installed automatically again, we also get a warning:

2024-04-25 11:18:45.845 WARNING (MainThread) [homeassistant.bootstrap] Skipping pip installation of required modules. This may cause issues

Home Assistant begins to start, but I get the error

ImportError: libmmal_core.so.0: cannot open shared object file: No such file or directory

Don't know if that is related to this issue.
But otherwise it seems to work, Home Assistant itself is starting up.

[alex]

Interesting. I do not see anything in the code that explains why there'd be a difference between main and 42.0.5...

…

On Thu, Apr 25, 2024 at 5:24 AM Tristan Kundrat ***@***.***> wrote: There are some version constraints I had to violate: ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency confl icts. pyopenssl 24.1.0 requires cryptography<43,>=41.0.5, but you have cryptography 43.0.0.dev1 which is incompatible. homeassistant 2024.4.4 requires cryptography==42.0.5, but you have cryptography 43.0.0.dev1 which is incompatible. I now ran hass --skip-pip-packages cryptography, so cryptography 24.0.5 does not get installed automatically again, we also get a warning: 2024-04-25 11:18:45.845 WARNING (MainThread) [homeassistant.bootstrap] Skipping pip installation of required modules. This may cause issues Home Assistant begins to start, but I get the error ImportError: libmmal_core.so.0: cannot open shared object file: No such file or directory Don't know if that is related to this issue. But otherwise it seems to work, Home Assistant itself is starting up. — Reply to this email directly, view it on GitHub <#10881 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAAGBALW7IOK7SHTPJHHGTY7DDUBAVCNFSM6AAAAABGWS3YS6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANZWG42DSMRSGU> . You are receiving this because you commented.Message ID: ***@***.***>

-- All that is necessary for evil to succeed is for good people to do nothing.

[github-actions]

This issue has been waiting for a reporter response for 3 days. It will be auto-closed if no activity occurs in the next 5 days.

[tkundrat]

I did not have any new idea to test. So I rebooted my pi again today and tried rebuilding the package as before inside my test-venv with pip3 install --force-reinstall --no-cache-dir cryptography -no-binary cryptography. Home Assistant now starts successfully, with the same error as with the 43.0.0 git version (libmmal_core.so.0 not found), but I don't think it's related to this.