LogRequestWithBody breaks S3 CopyObject

[SmilyOrg]

Documentation

• I've gone though the API reference
• I've checked AWS Forums and StackOverflow for answers
• I've searched for previous similar issues and didn't find any solution

Describe the bug

Enabling LogRequestWithBody client log mode breaks S3 CopyObject functionality with S3 returning 501 Not Implemented. Using LogRequest instead or just removing logging restores CopyObject functionality.

Expected behavior

The object/file should be copied successfully with a HTTP 200 response and CopyObject should return no error.

These are the expected request/response logs.

Request

PUT /TARGET_KEY?x-id=CopyObject HTTP/1.1
Host: BUCKET.s3.eu-central-1.amazonaws.com
User-Agent: aws-sdk-go-v2/1.13.0 os/windows lang/go/1.17.5 md/GOOS/windows md/GOARCH/amd64 api/s3/1.24.1
Content-Length: 0
Accept-Encoding: identity
Amz-Sdk-Invocation-Id: [redacted]
Amz-Sdk-Request: attempt=1; max=3
Authorization: [redacted]
X-Amz-Content-Sha256: [redacted]
X-Amz-Copy-Source: BUCKET/SOURCE_KEY
X-Amz-Date: [redacted]
X-Amz-Security-Token: [redacted]

Response

HTTP/1.1 200 OK
Content-Length: [redacted]
Content-Type: application/xml
Date: [redacted]
Server: AmazonS3
X-Amz-Copy-Source-Version-Id: [redacted]
X-Amz-Expiration: [redacted]
X-Amz-Id-2: [redacted]
X-Amz-Request-Id: [redacted]
X-Amz-Version-Id: [redacted]

<?xml version="1.0" encoding="UTF-8"?>
<CopyObjectResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <LastModified>[redacted]</LastModified>
  <ETag>[redacted]</ETag>
</CopyObjectResult>

Current behavior

The object/file is not copied and CopyObject returns the following error

operation error S3: CopyObject, https response error StatusCode: 501, RequestID: [redacted], HostID: [redacted], api error NotImplemented: A header you provided implies functionality that is not implemented

Request

PUT /TARGET_KEY?x-id=CopyObject HTTP/1.1
Host: BUCKET.s3.eu-central-1.amazonaws.com
User-Agent: aws-sdk-go-v2/1.13.0 os/windows lang/go/1.17.5 md/GOOS/windows md/GOARCH/amd64 api/s3/1.24.1
Content-Length: 0
Accept-Encoding: identity
Amz-Sdk-Invocation-Id: [redacted]
Amz-Sdk-Request: attempt=1; max=3
Authorization: [redacted]
X-Amz-Content-Sha256: [redacted]
X-Amz-Copy-Source: BUCKET/SOURCE_KEY
X-Amz-Date: [redacted]
X-Amz-Security-Token: [redacted]

Response

HTTP/1.1 501 Not Implemented
Connection: close
Transfer-Encoding: chunked
Content-Type: application/xml
Date: [redacted]
Server: AmazonS3
X-Amz-Id-2: [redacted]
X-Amz-Request-Id: [redacted]

<Error>
  <Code>NotImplemented</Code>
  <Message>A header you provided implies functionality that is not implemented</Message>
  <Header>Transfer-Encoding</Header>
  <RequestId>[redacted]</RequestId>
  <HostId>[redacted]</HostId>
</Error>

Steps to Reproduce

go run s3_log_copyobject_bug.go

s3_log_copyobject_bug.go

package main

import (
	"context"
	"fmt"

	"github.com/aws/aws-sdk-go-v2/aws"
	"github.com/aws/aws-sdk-go-v2/config"
	"github.com/aws/aws-sdk-go-v2/service/s3"
)

func main() {

	region := "eu-central-1"
	sourceBucket := "BUCKET"
	sourceKey := "SOURCE_KEY"
	targetBucket := "BUCKET"
	targetKey := "TARGET_KEY"

	cfg, err := config.LoadDefaultConfig(
		context.TODO(),
		config.WithRegion(region),
		// Enabling `LogRequestWithBody` breaks CopyObject below with a
		// HTTP 501 Not Implemented
		// If you comment out the line below, CopyObject should start
		// working and the code should print "CopyObject success"
		config.WithClientLogMode(aws.LogRequestWithBody),
	)
	if err != nil {
		panic(err.Error())
	}

	client := s3.NewFromConfig(cfg)

	source := fmt.Sprintf("%s/%s", sourceBucket, sourceKey)
	_, err = client.CopyObject(context.TODO(), &s3.CopyObjectInput{
		CopySource: aws.String(source),
		Key:        aws.String(targetKey),
		Bucket:     aws.String(targetBucket),
	})
	if err != nil {
		panic(err.Error())
	}

	println("CopyObject success")
}

Possible Solution

The current hypothesis is that when LogRequestWithBody reads the request body, it corrupts its state in the case of empty/no body CopyObject operation. The returned error is a bit strange as it refers to a Transfer-Encoding header, which doesn't even seem to be set in the request itself? Perhaps the SDK erroneously sends a non-empty body for which S3 expects that header to be set.

AWS Go SDK version used

v1.13.0

Compiler and Version used

go version go1.17.5 windows/amd64

Operating System and version

Windows 10 Version 20H2

[jasdel]

Thanks for reaching out @SmilyOrg this issue was fixed in v1.26.0 of the s3 module. This was fixed via the SDK's handling of HTTP request body in aws/smithy-go#356.

After updating to the latest version of the S3 module, are you still experiencing this issue?

[SmilyOrg]

Hi @jasdel!

I can't believe I missed both my version being out of date and the existing issue referencing NotImplemented. Fix seems to work for me as well!

Thank you.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.