Pypi wheel for tables 3.6.1 on Python 3.9 bundles (old) libraries

[nkeim]

Unlike the wheels for older versions of python, the current Linux wheel for python 3.9 includes prebuilt libraries for hdf5 etc. When imported, pytables uses the bundled hdf5 1.8.5patch1, instead of the newer system version (currently 1.10.4 on Ubuntu).

This breaks existing tests in soft-matter/trackpy#643 that pass for other versions of Python and other OS's.

This seems to be another kink to work out for python 3.9 along with #823 and #841 .

[avalentino]

Hi @ndawe looking at the wheels

tables-3.6.1-cp38-cp38-manylinux1_x86_64.whl
tables-3.6.1-cp39-cp39-manylinux2010_x86_64.whl

it seems that both include the HDF5 library even if different versions.
The wheel for Python 3.9 includes an older version and, if I understand correctly, this seems to be the cause of your problem.
Libraries included in the wheel for Python 3.9 are the ones that one can find or install on the manylinyx10 image (CentOS 6.x) and hence quite outdated.

The main activity regarding wheels is in PR #872.
Please add your comments to the PR if you want to be sure that future wheels are in line with your expectations.

Just for my curiosity: why your project prefers wheels over e.g. conda packages?
I mean, pytables, and scientific python sw in general, have dependencies form binary C/C++ libraries that are much better supported by a package manager like conda (or apt or rpm, etc.).
So what is, in your case, the advantage of using wheels?

[nkeim]

Thanks! Yes, that's correct—the Linux Python 3.9 wheel is built differently from all the others. It looks like #879 is trying to address this.

We have added a workaround to not test pytables when HDF5 is old, so this is not urgent for now.

conda is definitely the recommended way of installing our package, but not everyone uses it and we want to make life easier for those people.

[maxnoe]

We also just ran into this.

Using a fresh virtualenv for each python 3.8 and 3.9, I get:

❯ pyenv shell 3.8.7
❯ python -c 'import tables;print(tables.hdf5_version)'
1.10.4

and with 3.9:

❯ python -c 'import tables;print(tables.hdf5_version)'
1.8.5-patch1

this creates errors when openeing the same file twice:

❯ python -c 'import tables; tables.open_file("test.hdf5"); tables.open_file("test.hdf5")'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/maxnoe/test/3.9/lib/python3.9/site-packages/tables/file.py", line 288, in open_file
    raise ValueError(
ValueError: The file 'test.hdf5' is already opened.  Please close it before reopening.  HDF5 v.1.8.5-patch1, FILE_OPEN_POLICY = 'strict'
Closing remaining open files:test.hdf5...done

which do not happen with newer versions of libhdf5.

[maxnoe]

Could you please rebuild the wheels for python 3.9 using a recent libhdf5? This is turning into quite the blocker for us moving to 3.9. Please consider making a .post1 release for 3.6.1 with the updated wheels.

[alpae]

I second this point. We can not move forward to python 3.9. Let us know if we can somehow help bringing this forward.