File system scheme 's3' not implemented

[greper]

Click to expand!

Issue Type

Bug

Source

binary

Tensorflow Version

2.10.0

Custom Code

No

OS Platform and Distribution

Windows11

Mobile device

No response

Python version

3.8.10

Bazel version

No response

GCC/Compiler version

No response

CUDA/cuDNN version

No response

GPU model and memory

No response

Current Behaviour?

An error occurred when I ran tensorboard using s3 as logdir

tensorboard --logdir=s3://mnist/log --port=6123

I find this method is the cause of the problem

tf.io.gfile.exists("s3://minst/log/xxxxxx") 

### Standalone code to reproduce the issue


import os

os.environ.setdefault("S3_ENDPOINT", "https://xxxx")
os.environ.setdefault("AWS_ACCESS_KEY_ID", "xxx")
os.environ.setdefault("AWS_SECRET_ACCESS_KEY", "xxxx")
os.environ.setdefault("AWS_REGION", "us-east-1")

import tensorflow as tf
import tensorflow_io as tfio

gfile = tf.io.gfile.GFile("s3://minst/log/xxxxxx")
print(gfile)
# ↑ ↑ ↑ ---------it's ok

v = tf.io.read_file('s3://minst/log/xxxxx')
print(v) 
# ↑ ↑ ↑ ---------it's ok

tf.io.gfile.exists("s3://minst/log/xxxxxx")
# ↑ ↑ ↑ ---------error

tensorboard.run_main()
# ↑ ↑ ↑ ---------error

os.system("tensorboard --logdir=s3://mnist/log --port=6123")
# ↑ ↑ ↑ ---------error

# will got same error

### Relevant log output

Traceback (most recent call last):
  File "D:\Users\xiaojunnuo\Desktop\tensorboard\venv\lib\site-packages\tensorflow\python\lib\io\file_io.py", line 288, in file_exists_v2
    _pywrap_file_io.FileExists(compat.path_to_bytes(path))
tensorflow.python.framework.errors_impl.UnimplementedError: File system scheme 's3' not implemented (file: 's3://minst/log/xxxxxxxxxxxxxxxx')

[bhack]

tf.io.gfile.exists("s3://minst/log/xxxxxx")

tensorflow/tensorflow/python/lib/io/file_io.py

Lines 270 to 272 in 359c3cd

	This currently returns `True` for existing directories but don't rely on this
	behavior, especially if you are using cloud filesystems (e.g., GCS, S3,
	Hadoop):

os.system("tensorboard --logdir=s3://mnist/log --port=6123")

What TB version are you using? Check tensorflow/tensorboard#5491 as it is not too old.

[greper]

tensorboard is the latest

[greper]

Looks like it should be a problem with tensorboard, it shouldn't use tf.io.gfile.exists, I'll go to the tensorboar to report this problem.

thanks .

[bhack]

I am asking about the version cause it seems similar to tensorflow/tensorboard#5480

[greper]

My tensorboard version is 2.10.1

I see that the tensorboard source code is actually using tf.io.gfile.exists

https://github.com/tensorflow/tensorboard/blob/53e6d9ba3c2b1fb749725acb02e154d5387d9217/tensorboard/backend/event_processing/io_wrapper.py#L195

[bhack]

Do you have also an updated tensorflow-io version installed in the venv?

[greper]

tensorflow_io==0.27.0 , The problem is still there

[greper]

My s3 backend is minio, I don't know if it matters

[bhack]

I remember we had a thread on the forum:

https://discuss.tensorflow.org/t/writing-tensorboard-logs-to-s3-bucket-using-keras-callbacks/9960

[optiluca]

Same issue over here. It seems to me that S3 tensorboard support has never worked in recent history, courtesy of the issue @greper describes.

[bhack]

If it is a TB issue I think you could post your case at tensorflow/tensorboard#4255

[greper]

If 'tf.io.gfile.exists' does not support s3, then what interface can determine whether s3 files and directories exist

[bhack]

If 'tf.io.gfile.exists' does not support s3, then what interface can determine whether s3 files and directories exist

S3 is implemented in the IO repo so it is better that you ask there or on the forum

[greper]

I have submitted an issue to io , thank you for your reply

[sushreebarsa]

@greper Thank you for the update!
Could you move this issue to closed status as it will be addressed in the IO repo. Thank you!

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No