Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tar support #234

Open
moskomule opened this issue Oct 29, 2021 · 1 comment
Open

tar support #234

moskomule opened this issue Oct 29, 2021 · 1 comment

Comments

@moskomule
Copy link

moskomule commented Oct 29, 2021

Hi, the document says

PFIO supports various container file formats to aggregate many small files into single large file with metadata mapping, e.g. HDF5, ZIP and Tar (and more in future)

but does PFIO currently support tar files at the moment or plan to add its support in the future?

When I quickly try it as follows, pfio did not recognize a tar file:

with pfio.v2.from_url('file:///absolute_path/to/tar_file.tar') as f:
    # raises ValueError: /absolute_path/to/tar_file.tar must be a directory
    ...
@kuenishi
Copy link
Member

I have been hesitating to support tar so far, because the format of tar needs seeking every head of the file entry to create directory tree map inside the tarball. The expected implementation of tar container will be very inefficient at the very beginning of opening the tarball as a PFIO container. Supporting tar format aiming for small tarballs for utility won't be bad, but it's in lower priority for us. Absolutely, contribution will be welcomed.
See also: https://superuser.com/a/538798

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants