You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Exceeding Upload.MAX_PARTS doesn't throw an exception completely out
SDK version number
@aws-sdk/client-s3@3.549.0
Which JavaScript Runtime is this issue in?
Node.js
Details of the browser/Node.js/ReactNative version
v20.12.0
Reproduction Steps
import{S3}from'@aws-sdk/client-s3'import{Upload}from'@aws-sdk/lib-storage'consts3=newS3()constbucket='bucket'// Update this to your bucket name// The upload must be more than 5242880000 bytes (5 MiB x 10000 parts)// To pass this via stdin, you can use the following command:// $ dd if=/dev/urandom bs=5242880 count=11000 | node repro.mjs// If you have pipeviewer installed, you can use the following command to see the progress:// $ dd if=/dev/urandom bs=5242880 count=11000 | pv -s $((5242880 * 11000)) | node repro.mjsconstupload=newUpload({params: {Bucket: bucket,Key: 'bigfile.txt',Body: process.stdin,},client: s3,queueSize: 50,})awaitupload.done()
The object size is actually exactly 5 MiB x 10,000 parts: 52428800000 bytes. Notably, it also correctly completes the multi-part upload, so it really does look like it completes.
I also tested with await upload.done().catch(err => {throw err}) in case of promise being rejected; it has the same problem.
Thanks for your patience. Im able to reproduce the reported behavior. It's not clear to me at this point why an error is not thrown by the SDK. Will review this and let you know.
Thanks,
Ran~
RanVaknin
added
needs-review
This issue/pr needs review from an internal developer.
p2
This is a standard priority issue
investigating
Issue is being investigated and/or work is in progress to resolve the issue.
and removed
needs-triage
This issue or PR still needs to be triaged.
needs-review
This issue/pr needs review from an internal developer.
labels
Apr 22, 2024
kuhe
added
queued
This issues is on the AWS team's backlog
and removed
investigating
Issue is being investigated and/or work is in progress to resolve the issue.
labels
May 7, 2024
Checkboxes for prior research
Describe the bug
Exceeding
Upload.MAX_PARTS
doesn't throw an exception completely outSDK version number
@aws-sdk/client-s3@3.549.0
Which JavaScript Runtime is this issue in?
Node.js
Details of the browser/Node.js/ReactNative version
v20.12.0
Reproduction Steps
Observed Behavior
The object size is actually exactly 5 MiB x 10,000 parts: 52428800000 bytes. Notably, it also correctly completes the multi-part upload, so it really does look like it completes.
I also tested with
await upload.done().catch(err => {throw err})
in case of promise being rejected; it has the same problem.Expected Behavior
https://github.com/aws/aws-sdk-js-v3/blob/v3.549.0/lib/lib-storage/src/Upload.ts#L195 should throw an exception out when the parts max is exceeded.
Possible Solution
I'm not a node expert, but I suspect it has something to do with throwing an error inside an async function.
Additional Information/Context
No response
The text was updated successfully, but these errors were encountered: