Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to github.com/aws/aws-sdk-go-v2/service/s3 v1.25.0: plenty of warnings related to missing checksums #1606

Closed
3 tasks done
pommedeterresautee opened this issue Mar 1, 2022 · 2 comments · Fixed by #1607
Labels
bug This issue is a bug.

Comments

@pommedeterresautee
Copy link

Documentation

Describe the bug

When I update the github.com/aws/aws-sdk-go-v2/service/s3 dependency to v1.25.0 I get plenty of warnings:

...
SDK 2022/03/01 13:45:05 WARN Response has no supported checksum. Not validating response payload.
SDK 2022/03/01 13:45:05 WARN Response has no supported checksum. Not validating response payload.
SDK 2022/03/01 13:45:05 WARN Response has no supported checksum. Not validating response payload.
SDK 2022/03/01 13:45:05 WARN Response has no supported checksum. Not validating response payload.
...

In precedent version there is no such things.
There is no easy way to enable / disable checksum check

Expected behavior

I can download files from S3 bucket without having thousands of warnings.

Current behavior

When I try to download a file, I have thousands of warnings telling me there is no supported checksums to check.

...
SDK 2022/03/01 13:45:05 WARN Response has no supported checksum. Not validating response payload.
SDK 2022/03/01 13:45:05 WARN Response has no supported checksum. Not validating response payload.
SDK 2022/03/01 13:45:05 WARN Response has no supported checksum. Not validating response payload.
SDK 2022/03/01 13:45:05 WARN Response has no supported checksum. Not validating response payload.
...

Steps to Reproduce

My code to download a file from S3 bucket:

// ...
		obj := s3.GetObjectInput{
			Bucket: &bucket,
			Key:    &path,
		}
		ctx, cancel := context.WithTimeout(mainContext, requestTimeout)
		defer cancel()
		rawContent, err := client.GetObject(ctx, &obj)
		if err != nil {
			return nil, fmt.Errorf("error during AWS s3 object download: %v", err)
		}
		if ctx.Err() != nil {
			return nil, fmt.Errorf("context error (may be timeout related): %v", ctx.Err())
		}
		buf := new(bytes.Buffer)
		_, err = buf.ReadFrom(rawContent.Body)
		if err != nil {
			return nil, fmt.Errorf("error during AWS s3 object content read: %v", err)
		}
		return buf.Bytes(), nil

I tried below code but it didn't change anything:

		obj := s3.GetObjectInput{
			Bucket: &bucket,
			Key:    &path,
			ChecksumMode: types.ChecksumModeEnabled,
		}

Possible Solution

This is related to PR #1600 from

AWS Go SDK version used

1.25.0

Compiler and Version used

Go 1.16

Operating System and version

Ubuntu 21.04

@pommedeterresautee pommedeterresautee added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Mar 1, 2022
jasdel added a commit to jasdel/aws-sdk-go-v2 that referenced this issue Mar 1, 2022
Updates the SDK's checksum validation logic to require opt-in to output
response payload validation. The SDK was always preforming output
response payload checksum validation, not respecting the output
validation model option.

Fixes aws#1606
@jasdel jasdel removed the needs-triage This issue or PR still needs to be triaged. label Mar 1, 2022
@jasdel
Copy link
Contributor

jasdel commented Mar 1, 2022

Thanks for reporting this issue @pommedeterresautee. I was able to reproduce this issue, and put out PR #1607 that addresses this bug in the SDK's handling of automatic validation of response payload checksums.

@jasdel jasdel assigned skmcgrail and unassigned skmcgrail Mar 1, 2022
jasdel added a commit that referenced this issue Mar 7, 2022
…1607)

Updates the SDK's checksum validation logic to require opt-in to output
response payload validation. The SDK was always preforming output
response payload checksum validation, not respecting the output
validation model option.

Fixes #1606
@github-actions
Copy link

github-actions bot commented Mar 7, 2022

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants