Can't find anything about Endpoint::immutable after v.56.1

[makorne]

Hi! Thank you for your great crate!

But what to use instead Endpoint::immutable ?

Thanks!

[aajtodd]

What are you trying to do?

If you just want to change the base URL the SDK uses see the developer guide for some examples.

If you want to wholesale replace an endpoint to always be something specific you can override the endpoint resolver to just always return that endpoint:

    // a type that implements `ResolveEndpoint`, see docs
    // https://docs.rs/aws-sdk-s3/latest/aws_sdk_s3/config/struct.Builder.html#method.endpoint_resolver
    let endpoint_resolver = MyCustomResolver::new(); 

    aws_sdk_s3::Config::new(&config)
        .to_builder()
        // override endpoint resolution completely and return our endpoint
        .endpoint_resolver(endpoint_resolver)
        .build();

[makorne]

I try to put a file in local garage
My old code used aws-sdk-s3="1.28" with Endpoint::immutable and all worked.

But there is no Endpoint::immutable anymore.
And EndpointFuture::ready(Ok(Endpoint::builder().url("http://127.0.0.1:3900/").build())) throws an error:

ServiceError { source: Unhandled(Unhandled { source: ErrorMetadata { code: Some("InvalidRequest"), message: Some("Bad request: Invalid create bucket XML query"), extras: None },
 meta: ErrorMetadata { code: Some("InvalidRequest"), 
 message: Some("Bad request: Invalid create bucket XML query"), extras: None } }),
  raw: Response { status: StatusCode(400),
  headers: Headers { headers: {"content-type": HeaderValue { _private: H0("application/xml") },
  "content-length": HeaderValue { _private: H0("204") }, 
  "date": HeaderValue { _private: H0("Mon, 13 May 2024 08:00:43 GMT") }} }, 
  body: SdkBody { inner: Once(Some(b"<?xml version=\"1.0\" encoding=\"UTF-8\"?><Error>
  <Code>InvalidRequest</Code>
  <Message>Bad request: Invalid create bucket XML query</Message>
  <Resource>/test_file_XhHytm</Resource>
  <Region>garage</Region>
  </Error>")), retryable: true }, 
  extensions: Extensions { extensions_02x: Extensions, extensions_1x: Extensions } } } 
   name: test_file_XhHytm input: 2048

garage log:

INFO garage_api::generic_server: [::ffff:127.0.0.1]:54170 PUT /test_file_wh0xzw?x-id=PutObject
INFO garage_api::generic_server: Response: error 400 Bad Request, Bad request: Invalid create bucket XML query

If I try like in LOCALSTACK_ENDPOINT example

    let mut shared_config = aws_config::defaults(BehaviorVersion::latest());
    shared_config = shared_config.endpoint_url("http://127.0.0.1:3900/");   
    let shared_config = shared_config.load().await;
    let s3_config_builder = aws_sdk_s3::config::Builder::from(&shared_config);
    aws_sdk_s3::Client::from_conf(s3_config_builder.build())

I get an error: dispatch failure

Any ideas?

[aajtodd]

As a general note we don't guarantee compatibility with third-party tools. I'm happy to try and point you in the right direction though or give you some things to try.

---

Can you share more of the logs for the dispatch failure you get when setting endpoint_url?

More than likely when setting endpoint_url you also need to set force_path_style to true or else the default endpoint resolver for S3 is going to hoist the bucket name into the hostname of the URI.

let config = aws_config::defaults(BehaviorVersion::latest())
    .endpoint_url("http://127.0.0.1:3900/")
    .load()
    .await;

let s3_config = aws_sdk_s3::config::Builder::from(&config)
    .force_path_style(true)
    .build();

let s3 = aws_sdk_s3::Client::from_conf(s3_config);

---

[makorne]

Can you share more of the logs for the dispatch failure you get when setting endpoint_url?

With force_path_style(true) detailed logs:

garage_api::generic_server: Request { method: PUT, uri: /test_file_fTK9Ti?x-id=PutObject, version: HTTP/1.1, headers: {"content-type": "application/octet-stream", "content-length": "2048", "user-agent": "aws-sdk-rust/1.2.1 os/linux lang/rust/1.77.2", "x-amz-user-agent": "aws-sdk-rust/1.2.1 api/s3/1.28.0 os/linux lang/rust/1.77.2", "x-amz-date": "20240515T095349Z", "authorization": "AWS4-HMAC-SHA256 Credential=GKc0ed675144fd7ed5bbd79bc6/20240515/garage/s3/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-user-agent, Signature=71be79aadcaafba5bee1e33b0a4d4793e9bc5cfdbb3cbeb14ecb5af83383d39a", "x-amz-content-sha256": "10fc3c51a152e90e5b90319b601d92ccf37290ef53c35ff92507687d8a911a08", "amz-sdk-request": "attempt=1; max=1", "amz-sdk-invocation-id": "0a668aea-c053-41f2-a996-14f73b91edb0", "host": "127.0.0.1:3900"}, body: Body(Streaming) }
2024-05-15T09:53:49.027402Z DEBUG garage_api::s3::router: Received an unknown query parameter: 'x-id'
2024-05-15T09:53:49.027412Z DEBUG garage_api::generic_server: Endpoint: CreateBucket
2024-05-15T09:53:49.027806Z  INFO garage_api::generic_server: Response: error 400 Bad Request, Bad request: Invalid create bucket XML query

Looks like the error due to Received an unknown query parameter: 'x-id'

[aajtodd]

This seems like an issue with the third-party tool. Does your request to an actual S3 bucket work? If so I'd recommend opening an issue with garage.