Not finding libhdf5.so on ubuntu when building h5py from source.

[gl-yziquel]

I installed ubuntu libhdf5-dev package and it places libhdf5.so in /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so

Building h5py from git HEAD source with either python -m build or pip install . --user fails to locate libhdf5.so.

Is is a build bug in h5py ? Or is it ubuntu whose pkg-config does not behave properly ?

Contextual info:

mini-me@virtucon ~> uname -a
Linux virtucon 5.15.0-87-generic #97-Ubuntu SMP Mon Oct 2 21:09:21 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
mini-me@virtucon ~> python --version
Python 3.10.12

[aragilar]

Do you have pkg-config (or pkgconf) installed? What the error you see when the build fails?

[gl-yziquel]

pkg-config is indeed installed.

Here's what I get:

mini-me@virtucon ~/h/c/h5py (master)> python -m build                                                                   
* Creating venv isolated environment...                                                                                
* Installing packages in isolated environment... (Cython >=0.29.31,<4, oldest-supported-numpy, pkgconfig, setuptools >=61)
* Getting build dependencies for sdist...                                                                              
running egg_info                                                                                                       
writing h5py.egg-info/PKG-INFO                                                                                         
writing dependency_links to h5py.egg-info/dependency_links.txt                                                         
writing requirements to h5py.egg-info/requires.txt                                                                     
writing top-level names to h5py.egg-info/top_level.txt                                                                 
reading manifest file 'h5py.egg-info/SOURCES.txt'                                                                      
reading manifest template 'MANIFEST.in'                                                                                
warning: no files found matching 'AUTHORS'                                                                             
no previously-included directories found matching 'docs/_build'
...
warning: no previously-included files found matching 'rever.xsh'                                           
no previously-included directories found matching 'news'                                                               
adding license file 'LICENSE'                                                                                          
writing manifest file 'h5py.egg-info/SOURCES.txt'                                                                      
* Installing packages in isolated environment... (wheel)                                                               
* Building wheel...
running bdist_wheel                                                                                                    
running build                                                                                                          
running build_py                        
creating build                                                                                                         
creating build/lib.linux-x86_64-cpython-310
creating build/lib.linux-x86_64-cpython-310/h5py                                                                       
copying h5py/ipy_completer.py -> build/lib.linux-x86_64-cpython-310/h5py
...
creating build/lib.linux-x86_64-cpython-310/h5py/tests/test_vds
copying h5py/tests/test_vds/test_virtual_source.py -> build/lib.linux-x86_64-cpython-310/h5py/tests/test_vds
copying h5py/tests/test_vds/__init__.py -> build/lib.linux-x86_64-cpython-310/h5py/tests/test_vds
copying h5py/tests/test_vds/test_highlevel_vds.py -> build/lib.linux-x86_64-cpython-310/h5py/tests/test_vds
copying h5py/tests/test_vds/test_lowlevel_vds.py -> build/lib.linux-x86_64-cpython-310/h5py/tests/test_vds
copying h5py/tests/data_files/vlen_string_s390x.h5 -> build/lib.linux-x86_64-cpython-310/h5py/tests/data_files
copying h5py/tests/data_files/vlen_string_dset.h5 -> build/lib.linux-x86_64-cpython-310/h5py/tests/data_files
copying h5py/tests/data_files/vlen_string_dset_utc.h5 -> build/lib.linux-x86_64-cpython-310/h5py/tests/data_files
running build_ext
Loading library to get build settings and version: libhdf5.so
error: Unable to load dependency HDF5, make sure HDF5 is installed properly
Library dirs checked: ['/usr/local/lib']
error: libhdf5.so: cannot open shared object file: No such file or directory

ERROR Backend subprocess exited when trying to invoke build_wheel
mini-me@virtucon ~/h/c/h5py (master) [1]> 

[aragilar]

Can you post the output of pkg-config --libs hdf5? It looks like it's only looking '/usr/local/lib'.

[gl-yziquel]

Sure. Here goes:

mini-me@virtucon ~> pkg-config --libs hdf5
-L/usr/local/lib
mini-me@virtucon ~> 

It's a default ubuntu install. That's where ubuntu installed these packages:

mini-me@virtucon ~> apt-file search libhdf5.so
libhdf5-dev: /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so
libhdf5-mpich-dev: /usr/lib/x86_64-linux-gnu/hdf5/mpich/libhdf5.so
libhdf5-openmpi-dev: /usr/lib/x86_64-linux-gnu/hdf5/openmpi/libhdf5.so
mini-me@virtucon ~> 

I'd like to know if it is a hdf5 bug or a debian/ubuntu bug. Or just me raving on mushrooms.

[aragilar]

There is a pkg-config file pointing to /usr/local/lib, rather than the correct path. I'm not sure where it is, but

$ pkg-config --libs hdf5
-L/usr/lib/x86_64-linux-gnu/hdf5/serial -lhdf5

is what you should have.
Depending on what is providing pkg-config, you may be able to run:

$ pkg-config --path hdf5
/usr/lib/x86_64-linux-gnu/pkgconfig/hdf5.pc

which prints out the path to the config file.