v2.7 tensorboard works with s3

[tswangdi]

When I run the following command:
tensorboard --logdir s3://bucket/path

I get File system scheme 's3' not implemented error.

Complete Information is：

Exception in thread Reloader:
Traceback (most recent call last):
  File "/Users/user/.pyenv/versions/3.8.7/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/Users/user/.pyenv/versions/3.8.7/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/user/.pyenv/versions/3.8.7/lib/python3.8/site-packages/tensorboard/backend/event_processing/data_ingester.py", line 98, in _reload
    self._multiplexer.AddRunsFromDirectory(path, name)
  File "/Users/user/.pyenv/versions/3.8.7/lib/python3.8/site-packages/tensorboard/backend/event_processing/plugin_event_multiplexer.py", line 198, in AddRunsFromDirectory
    for subdir in io_wrapper.GetLogdirSubdirectories(path):
  File "/Users/user/.pyenv/versions/3.8.7/lib/python3.8/site-packages/tensorboard/backend/event_processing/io_wrapper.py", line 195, in GetLogdirSubdirectories
    if not tf.io.gfile.exists(path):
  File "/Users/user/.pyenv/versions/3.8.7/lib/python3.8/site-packages/tensorflow/python/lib/io/file_io.py", line 292, in file_exists_v2
    _pywrap_file_io.FileExists(compat.path_to_bytes(path))
tensorflow.python.framework.errors_impl.UnimplementedError: File system scheme 's3' not implemented (file: 's3://bucket/path')
Serving TensorBoard on localhost; to expose to the network, use a proxy or pass --bind_all
TensorBoard 2.7.0 at http://localhost:6006/ (Press CTRL+C to quit)

I want to know how it works with s3 in tensorboard v2.7.

[bileschi]

Verified that I get the same error with 2.7, as well as building at HEAD.

There were some recent changes to how remote file systems are supported. tensorboard used to rely on tensorflow. It could be a dependency issue. Routing to someone who has recently(ish) worked on our s3 support.

@yatbear can you take a look? Possibly related to #5288 or the underlying cause there?

[yatbear]

#5288 just moved the function IsCloudPath that checks for s3 file system prefix from tensorboard/backend/event_processing/io_wrapper.py to tensorboard/util/io_util.py, there shouldn't be any logical changes to the s3 support.

[yatbear]

Hi @tswangdi, can you try pip install tensorflow_io to see if that fixes the error?

More information: https://giters.com/tensorflow/tensorflow/issues/51583

[tswangdi]

@yatbear I've tried, but it didn't work.

The versions of the package:

boto3                           1.20.26
botocore                        1.23.26
tensorboard                     2.7.0
tensorboard-data-server         0.6.1
tensorboard-plugin-wit          1.8.0
tensorflow                      2.7.0
tensorflow-estimator            2.7.0
tensorflow-io                   0.23.1
tensorflow-io-gcs-filesystem    0.23.1

$ python --version
Python 3.8.7

The following code works (from https://giters.com/tensorflow/tensorflow/issues/51583):

import tensorflow as tf
import tensorflow_io as tfio

v = tf.io.read_file('s3://1234567')

[yatbear]

@tswangdi, apologies for missing the instruction to import tensorflow_io in the code in my comment above. Closing the issue since you are able to access the s3 bucket now, please feel free to reopen if there is any other issue :)

[tswangdi]

@yatbear But I cannot access the s3 bucket in tensorboard yet.

[spolloni]

@yatbear we are experiencing the same issue.

[yatbear]

@tswangdi, re #5480 (comment), did you mean tf.io.read_file from s3 worked, but tensorboard --logdir s3://bucket/path failed? If so is the error message still the same after installing tensorflow-io?

[spolloni]

@yatbear see also #5359

[tswangdi]

@yatbear yes, the error message is the same.

[yatbear]

Hi, @tswangdi the fix to conditionally import tensorflow_io module should be available in yesterday's release, thanks for your patience!

[tswangdi]

@yatbear OK, thank you for your efforts!

[crypdick]

Excellent @yatbear ! This is such a convenience!

[ngohoanganh96]

I am get the same error when using tensorboard --logdir s3://zenml-minio-store/logs/...
I used version as below;
tensorflow=2.8.0, tensorboard=2.8.0, tensorflow-io=0.24.0
I have tried to update to tensorflow=2.12.0, tensorboard=2.12.3, tensorflow-io= 0.33.0, but i doesn't work

[dhouib-akram]

Try to create an env without tensorflow (tensorboard doesn't need tensorflow to work ) it should work