s3 fake directory fix

[GeorgeSabu]

• Have fixed the fake s3 directory bug
• Have fixed the test cases
• Added some content to the file in the assets folder in test so that file size is not zero
• Also fixed some warning that are coming while running test cases

[GeorgeSabu]

@pjbull I have updated cloudpathlib/client.py and cloudpathlib/cloudpath.py as they caused warning while running test cases.

[GeorgeSabu]

@pjbull There is a attr.exceptions.FrozenInstanceError that is coming while running test because of which tests are failing which I am not able to reproduce locally.

[jayqi]

@GeorgeSabu the error is from #203

[GeorgeSabu]

@jayqi oops didnt know about it. I did an empty commit to rerun the test to make sure its not some other thing. Thanks for the info.

[jayqi]

Another thing to note here is that we've made some changes to _list_dir in #202 so there are going to be some merge conflicts.

---

Additionally, some comments about the changes:

• I'm kind of confused about how this fix solves the problem exactly. I believe that this is related to Recursion error on "fake" s3 directories #198? Where is the recursion error coming from, and why is _list_dir the underlying cause? I'd like to see a test that reproduces the recursion error so I can understand why this change would fix it. I think perhaps Recursion error on "fake" s3 directories #198 is not fully explained.
  • Comparing the discussion from Ceph compatibility - is_dir() fails for directories when "/" included at end of path #148 that motivated Recursion error on "fake" s3 directories #198 to the recent discussion in fix: download_to function tried to download path files #204, it kind of seems like this isn't maybe a problem with fake directories and actually a general bug with trying to apply download_to() on a directory due to how iterdir() works. In that case, I think the change proposed in this PR is the wrong fix. Re-implement glob logic #202 should be fixing the underlying cause and we expect to merge it soon.
• I'm also in general not sure the proposed change is exactly the desired behavior. _list_dir should yield directories if there are any. However, I think this is perhaps pointing out a weakness in Re-implement glob logic #202 where we don't identify fake directories as directories.

[GeorgeSabu]

@jayqi As far as I understood the code the function iterdir which wraps the _list_dir. And the iterdir is been used as an iterator in download_to as well as copytree function. The reason it goes to recursion is that when S3 creates a fake directory what I understands it does is making a PUT request to "Key" test-folder/.

import boto3

client = boto3.client('s3')

response = client.put_object(
        Bucket='my-top-level-bucket',
        Body='',
        Key='test-folder/'
        )

Now 'test-folder/' behaves likes an object than a prefix. And as copytree calls copytree internally for its subdirectory causes for it to go into recursion. And same is the case with download_to function.

By checking the Size > 0 I am making sure that this recursion does not happen. I hope this clears.