Tables crashes python with a run time error

[sunilshah]

I am running python 3.9.6 and tables version 3.6.2.

Python was installed using homebrew on macOS 11.4 on M1 under Rosetta2.

hdf5 1.12.1. was installed using homebrew.

>>> h5file = tables.open_file("tutorial1.h5", mode="w", title="Test file")
Warning! ***HDF5 library version mismatched error***
The HDF5 header files used to compile this application do not match
the version used by the HDF5 library to which this application is linked.
Data corruption or segmentation faults may occur if the application continues.
This can happen when an application was compiled by one version of HDF5 but
linked with a different version of static or shared HDF5 library.
You should recompile the application or check your shared library related
settings such as 'LD_LIBRARY_PATH'.
You can, at your own risk, disable this warning by setting the environment
variable 'HDF5_DISABLE_VERSION_CHECK' to a value of '1'.
Setting it to 2 or higher will suppress the warning messages totally.
Headers are 1.12.0, library is 1.12.1
	    SUMMARY OF THE HDF5 CONFIGURATION
	    =================================

General Information:
-------------------
                   HDF5 Version: 1.12.1
                  Configured on: Mon Jul 12 08:05:03 BST 2021
                  Configured by: brew@BigSur
                    Host system: x86_64-apple-darwin20.4.0
              Uname information: Darwin BigSur 20.4.0 Darwin Kernel Version 20.4.0: Thu Apr 22 21:46:47 PDT 2021; root:xnu-7195.101.2~1/RELEASE_X86_64 x86_64
                       Byte sex: little-endian
             Installation point: /usr/local/Cellar/hdf5/1.12.1

Compiling Options:
------------------
                     Build Mode: production
              Debugging Symbols: no
                        Asserts: no
                      Profiling: no
             Optimization Level: high

Linking Options:
----------------
                      Libraries: static, shared
  Statically Linked Executables: 
                        LDFLAGS: 
                     H5_LDFLAGS:  -Wl,-commons,use_dylibs
                     AM_LDFLAGS:  -L/usr/local/opt/szip/lib
                Extra libraries: -lsz -lz -ldl -lm 
                       Archiver: ar
                       AR_FLAGS: cr
                         Ranlib: ranlib

Languages:
----------
                              C: yes
                     C Compiler: /usr/bin/clang ( Apple clang version 12.0.5 )
                       CPPFLAGS: 
                    H5_CPPFLAGS:   -DNDEBUG -UH5_DEBUG_API
                    AM_CPPFLAGS:  -I/usr/local/opt/szip/include
                        C Flags: 
                     H5 C Flags:  -std=c99  -Wall -Warray-bounds -Wcast-qual -Wconversion -Wdouble-promotion -Wextra -Wformat=2 -Wframe-larger-than=16384 -Wimplicit-fallthrough -Wno-c++-compat -Wno-format-nonliteral -Wnull-dereference -Wunused-const-variable -Wwrite-strings -Wpedantic -Wvolatile-register-var    -Wno-missing-noreturn  -Wbad-function-cast -Wimplicit-function-declaration -Wincompatible-pointer-types -Wmissing-declarations -Wpacked -Wshadow -Wswitch -Wno-error=incompatible-pointer-types-discards-qualifiers -Wunused-function -Wunused-variable -Wunused-parameter -Wcast-align -Wformat -O3
                     AM C Flags: 
               Shared C Library: yes
               Static C Library: yes


                        Fortran: yes
               Fortran Compiler: /usr/local/opt/gcc/bin/gfortran ( GNU Fortran (Homebrew GCC 11.1.0_1) 11.1.0)
                  Fortran Flags: 
               H5 Fortran Flags:  -std=f2008  -Waliasing -Wall -Wcharacter-truncation -Wextra -Wimplicit-interface -Wsurprising -Wunderflow -pedantic -Warray-temporaries -Wintrinsics-std -Wimplicit-procedure -Wreal-q-constant -Wfunction-elimination -Wrealloc-lhs -Wrealloc-lhs-all -Wno-c-binding-type -Wuse-without-only -Winteger-division -Wfrontend-loop-interchange  -fdiagnostics-urls=never -fno-diagnostics-color -s -O3
               AM Fortran Flags: 
         Shared Fortran Library: yes
         Static Fortran Library: yes

                            C++: yes
                   C++ Compiler: /usr/bin/clang++ ( Apple clang version 12.0.5 )
                      C++ Flags: 
                   H5 C++ Flags:   -Wall -Warray-bounds -Wcast-qual -Wconversion -Wdouble-promotion -Wextra -Wformat=2 -Wframe-larger-than=16384 -Wimplicit-fallthrough -Wno-c++-compat -Wno-format-nonliteral -Wnull-dereference -Wunused-const-variable -Wwrite-strings -Wpedantic -Wvolatile-register-var     -Wbad-function-cast -Wimplicit-function-declaration -Wincompatible-pointer-types -Wmissing-declarations -Wpacked -Wshadow -Wswitch -Wno-error=incompatible-pointer-types-discards-qualifiers -Wunused-function -Wunused-variable -Wunused-parameter -Wcast-align -Wformat -O3
                   AM C++ Flags:  -DOLD_HEADER_FILENAME -DHDF_NO_NAMESPACE -DNO_STATIC_CAST
             Shared C++ Library: yes
             Static C++ Library: yes

                           Java: no


Features:
---------
                   Parallel HDF5: no
Parallel Filtered Dataset Writes: no
              Large Parallel I/O: no
              High-level library: yes
                Build HDF5 Tests: yes
                Build HDF5 Tools: yes
                    Threadsafety: no (recursive RW locks: no)
             Default API mapping: v112
  With deprecated public symbols: yes
          I/O filters (external): deflate(zlib),szip(no encoder)
                             MPE: no
                   Map (H5M) API: no
                      Direct VFD: no
                      Mirror VFD: no
              (Read-Only) S3 VFD: no
            (Read-Only) HDFS VFD: no
                         dmalloc: no
  Packages w/ extra debug output: none
                     API tracing: no
            Using memory checker: no
 Memory allocation sanity checks: no
          Function stack tracing: no
                Use file locking: best-effort
       Strict file format checks: no
    Optimization instrumentation: no
Bye...
zsh: abort      python3

This runs fine with hdf5 version 1.12.0_4 and tables version 3.6.1, under both Rosetta2 and in native mode. I am not sure if this is an issue with hdf5 install under homebrew or with tables.

[avalentino]

The message clearly states that you are using at runtime a version of the HDF5 library that is different form the one that you have used to build pytables.
You should fix it.
If you have multiple versions of HDF5 installed I would recommend to remove all the ones that you do not actually intend to use.
Also please make sure that DYLD_LIBRARY_PATH is properly set to point to the correct version of the HDF5 dylib at runtime.

[sunilshah]

Thanks @avalenito.

I did understand the message. However, I am not sure if tables is installing a version of hdf5 with version for 3.6.1 in the path for runtime library. Since everything works fine with the same set of packages (both for homebrew and pip) for earlier versions, I suspect something has changed in some install of the newer versions. Does pypi tables version 3.61, compiled with hdf5 1.12.0 version headers? If yes, a pip install -U --build_from_source tables should fix the problem, right?

How do I check for path for the runtime used by tables? I could then try to find the offending libraries and the cause of the conflicting libraries. I noticed that when upgrading hdf5 using homebrew, homebrew installed a number of other packages--octave and its required packages. IIRC after the homebrew upgrades, pip upgrade of tables (---build_from_source) did not resolve the problem.

[avalentino]

@sunilshah my comment was referred to an installation of PyTables from sources.
That kind of installation never includes HDF5 which is expected to be properly installed on the hast system.
In the User Manual you can find instructions to control the search paths for the relevant libraries at build time.

Does pypi tables version 3.61, compiled with hdf5 1.12.0 version headers?

if you refer to wheels I honestly don't remember, I should check, but I doubt.
Please consider the PyTables 3.6.1 is not new, it has been released in 2019!

If yes, a pip install -U --build_from_source tables should fix the problem, right?

Not sure to understand your reasoning, if you build form sources it is totally irrelevant how wheels on PyPI have been built. The source package is not tied to a specific HDF5 version.

How do I check for path for the runtime used by tables?

This is very system dependent, on OS X you could try to use something like otool -L to check what libraries are used by a program or a dynamic lib.

I noticed that when upgrading hdf5 using homebrew, homebrew installed a number of other packages--octave and its required packages. IIRC after the homebrew upgrades, pip upgrade of tables (---build_from_source) did not resolve the problem.

Sorry I don't use homebrew but, yes, my guess is that you need to clean a little bit your environment.

[sunilshah]

Sorry I don't use homebrew but, yes, my guess is that you need to clean a little bit your environment.

Perhaps homebrew's choice of different paths to support Apple Silicon may be playing a role here. I noticed that in setup.py, choice of prefix could be a problem.

def add_from_flags(envname, flag_key, dirs):
        dirs.extend(
            Path(flag[len(flag_key) :])
            for flag in os.environ.get(envname, "").split()
            if flag.startswith(flag_key)
        )

    if os.name == "posix":
        prefixes = ("/usr/local", "/sw", "/opt", "/opt/local", "/usr", "/")
        prefix_paths = [Path(x) for x in prefixes]

default_header_dirs = []
        add_from_path("CPATH", default_header_dirs)
        add_from_path("C_INCLUDE_PATH", default_header_dirs)
        add_from_flags("CPPFLAGS", "-I", default_header_dirs)
        add_from_flags("CFLAGS", "-I", default_header_dirs)
        default_header_dirs.extend(_tree / "include" for _tree in prefix_paths)

        default_library_dirs = []
        add_from_flags("LDFLAGS", "-L", default_library_dirs)
        default_library_dirs.extend(
            _tree / _arch
            for _tree in prefix_paths
            for _arch in ("lib64", "lib")
        )
        default_runtime_dirs = default_library_dirs

Homebrew has standardized all installs on Apple Silicon for x86_64-Rosetta2 under /usr/local/, while for arm64 under /opt/homebrew/.

Homebrew 3.0.0, Mike McQuaid, 05 February 2021

If my conjecture is true, all users who maintain both Rosetta2 and arm64 could have problems.

[jbrockmendel]

xref pandas-dev/pandas#42755

[sunilshah]

xref #887

[jbrockmendel]

FWIW im not on apple silicon

[sunilshah]

@jbrockmendel

Which machine and OS are you using? Could you please attach output of brew config?

Are you using pip to install pandas?

[jbrockmendel]

macOS 11.2.3, the problem started occurring after i did brew update a couple days ago. pandas built from source

[sunilshah]

@jbrockmendel

That is exactly how I discovered the problem on my machine.

Unfortunately, it has made my python code base brittle to brew / pandas upgrades. I have to resolve the problem or find a workaround to use of tables--it is only used for df.to_hdf() and df.read_hdf() in a few places.

[jbrockmendel]

I just re-installed using pip3 install tables --ignore-installed --no-binary tables and that tentatively solved the problem

[sunilshah]

@jbrockmendel

Can you please attach output of brew config ?

What version of hdf5 is your brew install show? I have problem moving from hdf5 version 1.12.0_4 to 1.12.1 for x86_64 (Rosetta 2) tables version 3.6.1. I also have same problems for tables version 3.6.2 (dev) for native M1 (arm64).

[jbrockmendel]

% brew config
HOMEBREW_VERSION: 3.2.6-100-gcd900ef
ORIGIN: https://github.com/Homebrew/brew
HEAD: cd900ef71eeecb621faba88aabc641fe93325fe6
Last commit: 25 hours ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 9216a4df62bd5c08b495be4c5fc909d98ed5aab3
Core tap last commit: 21 hours ago
Core tap branch: master
HOMEBREW_PREFIX: /usr/local
HOMEBREW_CASK_OPTS: []
HOMEBREW_MAKE_JOBS: 16
Homebrew Ruby: 2.6.3 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
CPU: 16-core 64-bit kabylake
Clang: 12.0.5 build 1205
Git: 2.32.0 => /usr/local/bin/git
Curl: 7.64.1 => /usr/bin/curl
macOS: 11.2.3-x86_64
CLT: 12.5.0.0.1.1617976050
Xcode: 12.5.1

What version of hdf5 is your brew install show?

1.12.1

Did you try the --no-binary thing above? That should compile pytables against the installed hdf5, which is what the warning is complaining about

[sunilshah]

@jbrockmendel thanks.

I see that you are current on homebrew and Xcode but using an older version of Mac OS (I am on 11.5.1). What version of python3 are you running? I am on 3.9.6. What about pip3 version?

pip3 install -U --no-binary --ignore-installed tables would force compilation rather than use available binary (.whl). It would also overwrite existing hdf5 install. Because I use brew installed hdf5, numpy and scipy natively (-arch arm64) on Apple Silicon, it would break many more packages that currently work using the brew installed packages.

I do have an Intel based Mac (though on OS 11.5.1). It fails the same way as on M1.

pip3 version 21.2.4 does not take the option --no-binary without an argument.

[jbrockmendel]

does not take the option --no-binary without an argument

The command I suggested does have an argument "tables". Note that "tables" shows up twice in the command above.

pip3 install -U --no-binary --ignore-installed tables would force compilation rather than use available binary (.whl).

Yes, that is the idea. IIUC the wheel was built using an older hdf5 than what is now installed on our machines, so we specifically need to not use that wheel.

It would also overwrite existing hdf5 install

I don't think this is correct.

[sunilshah]

pip3 install tables --ignore-installed --no-binary tables fails in compile step on my Intel Mac (OS 11.5.2) with an error.

I also verified that brew hdf5 installation functions in other applications that use it-- octave 6.3.0 save my.hdf5 -hdf5 a b x and load my.hdf5 -hdf5 work just fine. h5dump also works fine on the saved file on the Intel / Mac OS 11.5.2 machine.

Now I suspect that the problem is NOT with hdf5 install under homebrew but with pytables install / run-time. Between documentation of pytables install and assumptions in the code base, the problem with pytables keeps recurring over the last seven+ months across multiple OS updates, MAC hardware, brew updates and python3.9 versions.

When I get version information in python on the Intel Mac,

tables.print_versions()

it reports hdf5 version 1.12.0!

Yet, homebrew only shows hdf5 version 1.12.1. Where does pytbales pickup hdf5 version 1.12.0? How do I debug this?

[sunilshah]

I did work around the use of pytables in my code base by using x.to_pickle( ) and pd.read_pickle( ).

[sunilshah]

Tables does crash under Apple M1 in native mode. It has the same error message about library mismatch. It can not be fixed by compiling from source.

[avalentino]

@sunilshah what is the status of this issue.
I'm still convinced that it is a problem with your environment.
Have you tried to run the installation process in a truly clean environment?

[sunilshah]

@sunilshah what is the status of this issue.

The problem remains at run time on M1 in native mode.

Each issue mentioned above in this thread, including failure to build from source properly are still present.

brew config
HOMEBREW_VERSION: 3.3.8
ORIGIN: https://github.com/Homebrew/brew
HEAD: 3f0b412951996a675b8a48037e9a978f0ccd8363
Last commit: 10 days ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 1cd38e99a1fdb0019bd0b6daa4a662e177866228
Core tap last commit: 6 days ago
Core tap branch: master
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_CORE_GIT_REMOTE: https://github.com/Homebrew/homebrew-core
HOMEBREW_DISPLAY: /private/tmp/com.apple.launchd.fK18466pfs/org.xquartz:0
HOMEBREW_MAKE_JOBS: 8
Homebrew Ruby: 2.6.8 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
CPU: octa-core 64-bit arm_firestorm_icestorm
Clang: 12.0.5 build 1205
Git: 2.30.1 => /Library/Developer/CommandLineTools/usr/bin/git
Curl: 7.77.0 => /usr/bin/curl
macOS: 12.1-arm64
CLT: 12.5.0.22.9
Xcode: 13.2.1
Rosetta 2: false

I'm still convinced that it is a problem with your environment.
Have you tried to run the installation process in a truly clean environment?

I do not use python environments on M1. So it may be a significant effort to create "clean environment". As I mentioned earlier, I no longer use dataFrame.to_hdf() in my code. I did not find the documentation for PyTables installation on PyTables.org very clear for Mac OS.

I was able to build from source and run h5py on my M1 configuration. So, I am not sure the issue is with my environment per se.

By the way, another problem with multiple versions of run-time libraries is in issue
(scipy/scipy#14688 (comment))

also in (scipy/scipy#15050 (comment))

The kernel panic was due to a timeout bug in Mac OS (fixed in 12.0.1) but the performance problem was related to multiple BLAS libraries and their tuning for M1.

[avalentino]

As mentioned in #887 (comment) c-blosc at the moment does not support M1 native so it is impossible to have PyTables as well on that platform.
We need to wait the next Mayor release.

I did not find the documentation for PyTables installation on PyTables.org very clear for Mac OS.

If you have some experience to share please fee free to submit a PR to improve the PyTables documentation, it would be very appreciated.

[sunilshah]

As mentioned in #887 (comment) c-blosc at the moment does not support M1 native so it is impossible to have PyTables as well on that platform. We need to wait the next Mayor release.

I did not find the documentation for PyTables installation on PyTables.org very clear for Mac OS.

If you have some experience to share please fee free to submit a PR to improve the PyTables documentation, it would be very appreciated.

On my M1, c-blosc version 1.21.1 is installed and running under Rosetta2 and in native mode using homebrew. It was installed on October 15, 2021.

[avalentino]

On my M1, c-blosc version 1.21.1 is installed and running under Rosetta2 and in native mode using homebrew. It was installed on October 15, 2021.

Sorry, you are right #930 (comment)

[sunilshah]

I did not find the documentation for PyTables installation on PyTables.org very clear for Mac OS.

If you have some experience to share please fee free to submit a PR to improve the PyTables documentation, it would be very appreciated.

Sorry, my experience is only as a user of PyTables through pandas DataFrame.to_hdf(). I can point out what was confusing as a user trying to make the code run on M1.