NotImplementedError: unable to open file: libtensorflow_io_golang.so

[Conchylicultor]

I'm trying to use tensorflow_io in tensorflow/datasets#1113. However, our tests fails for both Python 2 and 3 when using TF 1.15.0, with the error:

NotImplementedError: unable to open file: libtensorflow_io_golang.so, from paths: ['/home/kbuilder/virtualenv/tfds_env/local/lib/python2.7/site-packages/tensorflow_io/core/python/ops/libtensorflow_io_golang.so'] 
caused by: ['dlopen: cannot load any more object with static TLS']

From tensorflow/tensorflow#19010, it may be related to the order in which we import the dependencies.

/home/kbuilder/virtualenv/tfds_env/local/lib/python2.7/site-packages/tensorflow_io/__init__.py:21: in <module>
    from tensorflow_io.core.python.ops.io_info import version as __version__
/home/kbuilder/virtualenv/tfds_env/local/lib/python2.7/site-packages/tensorflow_io/core/python/ops/__init__.py:69: in <module>
    core_golang_ops = _load_library('libtensorflow_io_golang.so')
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
      raise NotImplementedError(
          "unable to open file: " +
>         "{}, from paths: {}\ncaused by: {}".format(filename, filenames, errs))
E     NotImplementedError: unable to open file: libtensorflow_io_golang.so, from paths: ['/home/kbuilder/virtualenv/tfds_env/local/lib/python2.7/site-packages/tensorflow_io/core/python/ops/libtensorflow_io_golang.so']
E     caused by: ['dlopen: cannot load any more object with static TLS']

/home/kbuilder/virtualenv/tfds_env/local/lib/python2.7/site-packages/tensorflow_io/core/python/ops/__init__.py:67: NotImplementedError

Did you ever encountered this and or knew how to fix it ?

[yongtang]

@Conchylicultor tensorflow-io 0.8.0 is compatible with TF 1.15 and tensorflow-io 0.9.0 is compatible with TF 2.0. Due to some name space changes in TensorFlow's C++ API, the two version does not work if exchanged.

Can you give it a try to pin to tensorflow-io==0.8.0 for 1.15, and see if the issue still exist?

[yongtang]

The namespace change in 1.15 was mentioned in #472 (comment)

TF 2.0 revert back the behavior.

[yongtang]

@Conchylicultor Actually, I might be jump to conclusion too soon. Depending on which version is installed:

• tensorflow-io 0.8.0 + tensorflow 1.15 -- should work
• tensorflow-io 0.9.0 + tensorflow 2.0 -- should work
• tensorflow-io 0.8.0 + tensorflow 2.0 -- should not work due to namespace mismatch
• tensorflow-io 0.9.0 + tensorflow 1.15 -- should not work due to namespace mismatch

If the issue with TF 1.15 is happening with tensorflow-io 0.8.0 + tensorflow 1.15, then there might be some other factors. I can take a look and see if there are possible reasons for that.

[vlasenkoalexey]

One thing worth mentioning is that tensorflow-io might not work with manually built tensorflow if different compile flags were used, see #161 (comment)

[Conchylicultor]

It looked like installing tensorflow_io 0.9.0. Maybe this was due to us using tf-nightly.
It worked for both 1.15 and 2.0. Not sure why it is compatible with 1.15 though.

Thanks for your help