Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Draft] Fix resumable upload #845

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

barathsriram
Copy link
Contributor

@barathsriram barathsriram commented Jun 29, 2022

This is a work-in-progress PR that intends to fix #532 and #623

Changes:

  1. Fix the Resumable upload URL to match Google's API URL
  2. Retrieve upload_id from query parameter

TODO:

  • Fix any existing test failures
  • Add new tests for the changes

@agates4
Copy link

agates4 commented Jun 29, 2022

Awesome, thanks for putting this together. I'm looking forward to it!

@fsouza
Copy link
Owner

fsouza commented Jul 15, 2022

Hey @barathsriram, thanks for contributing! Do you plan to continue working on this?

@barathsriram
Copy link
Contributor Author

Hey @fsouza apologies for the delay in response. Yes, I am planning to start the work again on this PR soon.

@wwwjn
Copy link

wwwjn commented Nov 6, 2022

Hey @barathsriram thanks for such great work! Is there any progress on this PR?

@ciekawy
Copy link

ciekawy commented Mar 26, 2023

considering node library @google-cloud/storage does seem to use resumable uploads only - am I correct this feature is correctly broken?

@rockettomatooo
Copy link

I am also using the node @google-cloud/storage package. Would love to see this PR merged! 🤩

@fsouza
Copy link
Owner

fsouza commented May 3, 2023

considering node library @google-cloud/storage does seem to use resumable uploads only - am I correct this feature is correctly broken?

I think there are multiple endpoints for resumable uploads and the node library uses one that's not supported by fake-gcs-server. I don't know exactly whether this PR fixes that issue as I haven't encountered it myself, but I do plan to pick it back some time before the end of May if @barathsriram doesn't beat me to it.

@odinho
Copy link

odinho commented Oct 11, 2023

To make this turn up on search, I believe this is the error we're getting when we try to use @google-cloud/storage nodejs package:

172.30.0.15 - - [11/Oct/2023:09:12:31 +0000] \"PATCH /b/bucket/o/folder%2file.json? HTTP/1.1\" 405 0\n"

Error: Call  falied  [cause]: ApiError: Method Not Allowed   at <my code>
   at new ApiError (/cloud-function/node_modules/@google-cloud/storage/build/src/nodejs-common/util.js:79:15)
      at Util.parseHttpRespMessage (/cloud-function/node_modules/@google-cloud/storage/build/src/nodejs-common/util.js:181:41)
      at Util.handleResp (/cloud-function/node_modules/@google-cloud/storage/build/src/nodejs-common/util.js:155:76)
      at /cloud-function/node_modules/@google-cloud/storage/build/src/nodejs-common/util.js:537:22
      at onResponse (/cloud-function/node_modules/retry-request/index.js:240:7)
      at /cloud-function/node_modules/teeny-request/build/src/index.js:226:13
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
    code: 405,
    errors: [],
    response: PassThrough {
      _readableState: [ReadableState],
      _events: [Object: null prototype],
      _eventsCount: 4,
      _maxListeners: undefined,
      _writableState: [WritableState],
      allowHalfOpen: true,
      statusCode: 405,
      statusMessage: 'Method Not Allowed',
      request: [Object],
      body: '',
      headers: [Object],
      toJSON: [Function: toJSON],
      [Symbol(kCapture)]: false,
      [Symbol(kCallback)]: null
    }
  }

Basically, the PATCH request returns a 405 Method Not Allowed in some cases of the resumable upload.


Seeing as this new PR doesn't really start supporting PATCH at all, it's something else. This code was using file.setMetadata({metadata: { changeCount: 1 }}), so that probably sends a PATCH.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Resumable uploads don't set content-type
7 participants