Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

s3fs rm fails silently #838

Open
dhirschfeld opened this issue Dec 13, 2023 · 1 comment
Open

s3fs rm fails silently #838

dhirschfeld opened this issue Dec 13, 2023 · 1 comment

Comments

@dhirschfeld
Copy link

I'm observing that rm is swallowing underlying errors and not deleting the files it was requested to delete:

>>> fs.rm(path)
[]
>>> fs.rm_file(path)
<snip>
File C:\python\envs\dev-py310\lib\site-packages\s3fs\core.py:1910, in S3FileSystem._rm_file(self, path, **kwargs)
   1907 self.invalidate_cache(path)
   1909 try:
-> 1910     await self._call_s3("delete_object", Bucket=bucket, Key=key)
   1911 except ClientError as e:
   1912     raise translate_boto_error(e)
  
PermissionError: Access Denied

i.e. rm is failing silently.

My expectation when I call rm is that only one of 2 things will happen - either:

  1. All files were deleted successfully
  2. Or an error is raised

I'm fine with some of the files having been deleted and some not so long as an exception is raised telling me that not all of the files/folders I requested to be deleted were able to be deleted.

@martindurant
Copy link
Member

rm() expands its inputs (globs and maybe recursive) and returns the list of deleted files. If no file were found, it should raise FileNotFound I suppose.

In _bulk_delete there is a comment to extract errors from files found but not deleted. This should be filled out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants