Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid error if plugins .so module is not available #1302

Merged
merged 1 commit into from
Feb 8, 2021

Conversation

yongtang
Copy link
Member

@yongtang yongtang commented Feb 5, 2021

This PR raises a warning instead of an error in case plugins .so module is not available, so that tensorflow-io package can be at least partially used with python-only functions.

This PR is related to #1298

Signed-off-by: Yong Tang yong.tang.github@outlook.com

This PR raises a warning instead of an error in case
plugins .so module is not available, so that tensorflow-io
package can be at least partially used with python-only
functions.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
@pietmlr pietmlr mentioned this pull request Feb 6, 2021
plugin_ops = _load_library("libtensorflow_io.so", "fs")
try:
core_ops = _load_library("libtensorflow_io.so")
plugin_ops = _load_library("libtensorflow_io.so", "fs")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we reload plugin_ops = _load_library("libtensorflow_io.so", "fs") only?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@burgerkingeater This part is specific to CentOS 7. In CentOS 7 we used static linking, that prompt us to combine libtensorflow_io.so and libtensorflow_io_plugins.so into one libtensorflow_io.so as static linking does not work with two .so. Notice the comment about:

Note: load libtensorflow_io.so imperatively in case of statically linking

When we have one libtensorflow_io.so file to load, due to tensorflow's limitations, it needs to be loaded with full symbols before file system symbols. So we use core_ops = _load_library("libtensorflow_io.so") before fs load.
This also means tensorflow-io on CentOS 7 can not have api backward-compatibility like other linux dist. This is a limitation that may only be resolved in CentOS 8 or higher.

@yongtang yongtang merged commit 1b39aab into tensorflow:master Feb 8, 2021
@yongtang yongtang deleted the python-only branch February 13, 2021 14:15
i-ony pushed a commit to i-ony/io that referenced this pull request Mar 8, 2021
This PR raises a warning instead of an error in case
plugins .so module is not available, so that tensorflow-io
package can be at least partially used with python-only
functions.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
michaelbanfield pushed a commit to michaelbanfield/io that referenced this pull request Mar 30, 2021
This PR raises a warning instead of an error in case
plugins .so module is not available, so that tensorflow-io
package can be at least partially used with python-only
functions.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants