Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remote_storage: update S3 SDK BehaviorVersion #7665

Open
problame opened this issue May 8, 2024 · 0 comments
Open

remote_storage: update S3 SDK BehaviorVersion #7665

problame opened this issue May 8, 2024 · 0 comments
Assignees
Labels
a/tech_debt Area: related to tech debt c/storage/pageserver Component: storage: pageserver

Comments

@problame
Copy link
Contributor

problame commented May 8, 2024

PR #7664 upgraded AWS SDKs but didn't dare to change the BehaviorVersion.

To be done in the week after releasing the #7664

@problame problame added the c/storage/pageserver Component: storage: pageserver label May 8, 2024
@problame problame self-assigned this May 8, 2024
@problame problame added the a/tech_debt Area: related to tech debt label May 8, 2024
problame added a commit that referenced this issue May 9, 2024
…(updates AWS SDKs) (#7664)

Before this PR, using the AWS SDK profile feature for running against
minio didn't work because
* our SDK versions were too old and didn't include
  awslabs/aws-sdk-rust#1060 and 
* we didn't massage the s3 client config builder correctly.

This PR
* udpates all the AWS SDKs we use to, respectively, the latest version I
could find on crates.io (Is there a better process?)
* changes the way remote_storage constructs the S3 client, and
* documents how to run the test suite against real S3 & local minio.

Regarding the changes to `remote_storage`: if one reads the SDK docs, it
is clear that the recommended way is to use `aws_config::from_env`, then
customize.
What we were doing instead is to use the `aws_sdk_s3` builder directly.

To get the `local-minio` in the added docs working, I needed to update
both the SDKs and make the changes to the `remote_storage`. See the
commit history in this PR for details.

Refs:
* byproduct: smithy-lang/smithy-rs#3633
* follow-up on deprecation:
#7665
* follow-up for scrubber S3 setup:
#7667
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a/tech_debt Area: related to tech debt c/storage/pageserver Component: storage: pageserver
Projects
None yet
Development

No branches or pull requests

1 participant