Empty Content-Encoding header with S3 CRT client

[bertbaron]

Describe the bug

When performing a PutObjectRequest using S3 CRT client, an empty Content-Encoding header (system-defined metadata) is set to the object.
While this header in itself may not be much of a problem, it completely broke our sites served from CloudFront with the Manged-CachingOptimized behavior. The reason is that, when CloudFront decides to send the content compressed, it adds a header like Content-Encoding: br but also includes the empty Content-Encoding: header. The result is that the browser may assume the content is uncompressed.

Note that this behavior seems to be very similar to this issue from 2017 on the .NET client: aws/aws-sdk-net#559. The issues in combination with CloudFront is also mentioned in a comment there.

Expected Behavior

When performing a PutObjectRequest using S3 CRT client, without explicitly specifying contentEncoding, no Content-Encoding header should be set

Current Behavior

When performing a PutObjectRequest using S3 CRT client, without explicitly specifying contentEncoding, an empty Content-Encoding header is set

Reproduction Steps

To reproduce the following code snippet can be used

    public void putObject(String bucket, String key) {
        S3AsyncClient asyncClient = S3AsyncClient.crtBuilder()
                .region(Region.of(System.getenv(SdkSystemSetting.AWS_REGION.environmentVariable())))
                .credentialsProvider(DefaultCredentialsProvider.create())
                .build();
        AsyncRequestBody requestBody = AsyncRequestBody.fromString("content");
        asyncClient.putObject(b -> b.bucket(bucket)
                .key(key)
                .serverSideEncryption("AES256").build(), requestBody).join();
    }

Possible Solution

Our current workaround is to not use the CRT client (replacing S3AsyncClient.crtBuilder() with S3AsyncClient.builder())

Additional Information/Context

Version 0.29.11 of aws-crt was used

AWS Java SDK version used

2.25.1

JDK version used

openjdk 21.0.2 2024-01-16 LTS

Operating System and version

any

[bhoradc]

Hi @bertbaron,

Thank you for reporting the issue. I am able to reproduce the behaviour which you have highlighted in the issue.

Using the code snippet S3AsyncClient.crtBuilder() you provided, I am able to see the empty Content-Encoding key getting uploaded in S3.

Though I am inclined to relate this problem to the cross-sdk issue aws/aws-sdk#498, in your example I do not see the content-encoding=aws-chunked getting passed from Java SDK.

Will look further to understand why the empty Content-Encoding header/metadata still gets added in S3.

I will check and keep you posted on it.

Regards,
Chaitanya

[bhoradc]

Hi @bertbaron,

When I enabled the CRT debug logs, I can see that Java SDK is sending the content-encoding header value as aws-chunked. But S3 is storing the resulting object without the aws-chunked encoding.

The cross-SDK issue that I linked earlier aws/aws-sdk#498 is the right place to track it, as it impacts the other SDKs as well.

This issue has been reported internally to the S3 team, who has a backlog item to track it further.

Therefore, I will go ahead and close this one. We'll post further updates for it on the cross-SDK issue.

Regards,
Chaitanya

[bertbaron]

Thanks for the update, I will track the other issue.

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.