[Question] Refresh logdir subdirectories

[Krasner]

I'm unclear if this behavior is intended or not: it seems that Tensorboard will monitor and refresh events after the initial logdir is set, but if new subdirectories show up in logdir those events will not show up.

The subdirectory iterator in uploader/logdir_loader.py in function synchronize_runs() which calls io_wrapper.GetLogdirSubdirectories is only called once upon creation of TensorBoardUploader

for subdir in io_wrapper.GetLogdirSubdirectories(self._logdir):

Therefore if new subdirectories in logdir appear while Tensorboard is already running, io_wrapper.GetLogdirSubdirectories will not be called again and thus those new subdirectories are not detected.

Is this correct? If so, can a logdir subdirectory monitor be added?

[bmd3k]

Hi, in fact TensorBoard should discover new subdirectories for as long as the tensorboard process is running.

If you are experiencing different behavior then we will need additional information to help you troubleshoot.

[Krasner]

@bmd3k this is related to issue #6349 where I am trying to access an aws s3 folder as logdir.
This is a consistent problem I am seeing with the subdirectories not being updated.

All the diagnostics are in that issue.
Thanks!

[Krasner]

@bmd3k
We can close this - in the issue above #6349 I implemented s3fs walk to iterate through my s3 bucket since tf.io.gfile was very slow. apparently s3.walk doesn't automatically reindex the bucket if called again unless refresh flag is set explicitly:
for dir_path, _, filenames in s3.walk(top, topdown=True, refresh=True):

With this flag the subdirectories are updated. But the issue #6349 still stands and still needs addressing.