Natively implement 'move' operation for S3 objects

[fabiog1901]

Hello, I hope I'm in the right place to raise a generic observation about the S3 API, and a possible solution to be implemented and made available in the Go SDK, and all other sdk's.

Currently, to move an object from one S3 bucket to another, I see that we have to take these actions (I quickly tested using the cli, with aws s3 mv --debug):

1. Check the object exists.
2. Copy the object to the target bucket.
3. Delete the object from the source bucket.

These actions require my user to have the S3 Policy DeleteObject, which I rather not grant otherwise.

A possible solution, and hence my question, is: can the move action be implemented server side instead? We could have a Policy called MoveObject, and while under the hood is still a get+copy+delete, the server side implementation will not required the user to have the DeleteObject policy as, for all intents and purposes, no object gets deleted, just "moved".

[RanVaknin]

Hi @fabiog1901 ,

for all intents and purposes, no object gets deleted, just "moved".

I'm not sure what you mean by just "move", Sounds like you just want to copy the object over, for that you can use CopyObject.

If you are referring to a move implementation similar to what a file system does by changing the metadata for the parent directory, this is not how S3 works. S3 uses a distributed architecture that is fundamentally different from a traditional file system. In S3, data is stored as blobs within buckets, and these objects are distributed across multiple computers in different data centers. Because of its distributed nature, all references to an object have to be copied over to a new location, and then deleted from the old one.

can the move action be implemented server side instead?

Even if the action is implemented service side, ie S3 publishes a MoveObject and that action will delete an object you will still need a DeleteObject in your policy, the same way you need a write permission to use CopyObject.

Thanks,
Ran~

[fabiog1901]

Thanks Ran for the answer, but I am not sure why the client would need DeleteObject and CopyObject in the presence of an hypothetical MoveObject policy. True, server side there will be a copy+delete, but that's transparent to the client.

I realize this repo is not the right venue to ask as this is a server side feature request, not SDK bound. What would be a great place to ask? Thank you 🙇

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.