Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3 Signature Error Performing List With Prefix Containing Spaces #2800

Closed
tustvold opened this issue Sep 28, 2022 · 1 comment · Fixed by #2801
Closed

S3 Signature Error Performing List With Prefix Containing Spaces #2800

tustvold opened this issue Sep 28, 2022 · 1 comment · Fixed by #2801
Labels
bug object-store Object Store Interface

Comments

@tustvold
Copy link
Contributor

Describe the bug

Attempting to perform a list request using a prefix containing a space results in a signature error.

<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method
...

This is because IAM requires always percent encoding all spaces - https://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html

Percent-encode all other characters with %XY, where X and Y are hexadecimal characters (0-9 and uppercase A-F). For example, the space character must be encoded as %20 (not using '+', as some encoding schemes do)

This is normally the case as we explicitly percent encode paths - https://github.com/apache/arrow-rs/blob/master/object_store/src/aws/client.rs#L471

However, we delegate to reqwest for percent encoding of query parameters, and it uses + instead of %20.

To Reproduce

Expected behavior

Additional context

@tustvold tustvold added the bug label Sep 28, 2022
tustvold added a commit to tustvold/arrow-rs that referenced this issue Sep 28, 2022
tustvold added a commit that referenced this issue Sep 29, 2022
* Fix S3 query canonicalization (#2800)

* Disable listing with spaces on azurite and localstack
@alamb
Copy link
Contributor

alamb commented Sep 30, 2022

label_issue.py automatically added labels {'object-store'} from #2801

@alamb alamb added the object-store Object Store Interface label Sep 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug object-store Object Store Interface
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants