Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

multipart/form-data fails to upload file when using FormData #4885

Open
colesiegel opened this issue Jul 26, 2022 · 11 comments
Open

multipart/form-data fails to upload file when using FormData #4885

colesiegel opened this issue Jul 26, 2022 · 11 comments

Comments

@colesiegel
Copy link

Hi,

I'm trying to use FormData to POST a file using multipart/form-data

I noticed for very small files this works no problem but as soon as the length gets above about 10KB I start to see errors Error [ERR_UNHANDLED_ERROR]: Unhandled error. ('[AxiosError: maxContentLength size of -1 exceeded]

After debugging the solution I have noticed that if I put a breakpoint just before the call to axios and wait a few seconds, then I don't see this failure. Is there some promise I am supposed to await that I am missing?

Service class

import axios from 'axios';
import FormData from 'form-data';

public postFile = async (
    filename: string,
    fileContent: Buffer
  ) => {
    const formData = new FormData();
    // filename seems to be required for the specific API I am posting to
    formData.append('file', fileContent, filename);
    formData.append('report_format', 'json');

    try {
      const result = await axios.post(
        'ENDPOINT',
        formData,
        {
          headers: {
            ...formData.getHeaders(),
            'Authorization': 'Bearer xxx',
          },
        }
      );
      return result.data;
    } catch (e) {
      logger.error('failed');
      throw e;
    }
  };

Usage

const staticfileAnalysis = await postFile(
  'abcd.txt',
  Buffer.from('abcd')
);

Only issue I could find with the same error is #4806

@colesiegel
Copy link
Author

colesiegel commented Jul 26, 2022

Here is a log for reference, it seems like actual reason for error is a 400 status code response, but I have no idea why sometimes this would come back when usually the request works:

Error:
Error [ERR_UNHANDLED_ERROR]: Unhandled error. ('[AxiosError: maxContentLength size of -1 exceeded] {\n' +
  "  code: 'ERR_BAD_RESPONSE',\n" +
  '  config: {\n' +
  '    transitional: {\n' +
  '      silentJSONParsing: true,\n' +
  '      forcedJSONParsing: true,\n' +
  '      clarifyTimeoutError: false\n' +
  '    },\n' +
  '    adapter: [Function: httpAdapter],\n' +
  '    transformRequest: [ [Function: transformRequest] ],\n' +
  '    transformResponse: [ [Function: transformResponse] ],\n' +
  '    timeout: 0,\n' +
  "    xsrfCookieName: 'XSRF-TOKEN',\n" +
  "    xsrfHeaderName: 'X-XSRF-TOKEN',\n" +
  '    maxContentLength: -1,\n' +
  '    maxBodyLength: -1,\n' +
  '    env: { FormData: [Function] },\n' +
  '    validateStatus: [Function: validateStatus],\n' +
  '    headers: {\n' +
  "      Accept: 'application/json, text/plain, */*',\n" +
  "      'Content-Type': 'multipart/form-data; boundary=--------------------------084245449142421267498946',\n" +
  "      Authorization: 'xxx',\n" +
  "      'User-Agent': 'axios/0.27.2',\n" +
  "      'content-type': 'multipart/form-data; boundary=--------------------------084245449142421267498946'\n" +
  '    },\n' +
  "    method: 'post',\n" +
  "    url: 'https://test-endpoint.com/file',\n" +
  '    data: FormData {\n' +
  '      _overheadLength: 304,\n' +
  '      _valueLength: 22317,\n' +
  '      _valuesToMeasure: [],\n' +
  '      writable: false,\n' +
  '      readable: true,\n' +
  '      dataSize: 0,\n' +
  '      maxDataSize: 2097152,\n' +
  '      pauseStreams: true,\n' +
  '      _released: true,\n' +
  '      _streams: [],\n' +
  '      _currentStream: null,\n' +
  '      _insideLoop: false,\n' +
  '      _pendingNext: false,\n' +
  "      _boundary: '--------------------------084245449142421267498946',\n" +
  '      _events: [Object: null prototype],\n' +
  '      _eventsCount: 1\n' +
  '    }\n' +
  '  },\n' +
  '  request: <ref *1> ClientRequest {\n' +
  '    _events: [Object: null prototype] {\n' +
  '      abort: [Function (anonymous)],\n' +
  '      aborted: [Function (anonymous)],\n' +
  '      connect: [Function (anonymous)],\n' +
  '      error: [Function (anonymous)],\n' +
  '      socket: [Function (anonymous)],\n' +
  '      timeout: [Function (anonymous)],\n' +
  '      prefinish: [Function: requestOnPrefinish]\n' +
  '    },\n' +
  '    _eventsCount: 7,\n' +
  '    _maxListeners: undefined,\n' +
  '    outputData: [],\n' +
  '    outputSize: 0,\n' +
  '    writable: true,\n' +
  '    destroyed: true,\n' +
  '    _last: true,\n' +
  '    chunkedEncoding: true,\n' +
  '    shouldKeepAlive: false,\n' +
  '    _defaultKeepAlive: true,\n' +
  '    useChunkedEncodingByDefault: true,\n' +
  '    sendDate: false,\n' +
  '    _removedConnection: false,\n' +
  '    _removedContLen: false,\n' +
  '    _removedTE: false,\n' +
  '    _contentLength: null,\n' +
  '    _hasBody: true,\n' +
  "    _trailer: '',\n" +
  '    finished: true,\n' +
  '    _headerSent: true,\n' +
  '    socket: TLSSocket {\n' +
  '      _tlsOptions: [Object],\n' +
  '      _secureEstablished: true,\n' +
  '      _securePending: false,\n' +
  '      _newSessionPending: false,\n' +
  '      _controlReleased: true,\n' +
  '      secureConnecting: false,\n' +
  '      _SNICallback: null,\n' +
  "      servername: 'test',\n" +
  '      alpnProtocol: false,\n' +
  '      authorized: true,\n' +
  '      authorizationError: null,\n' +
  '      encrypted: true,\n' +
  '      _events: [Object: null prototype],\n' +
  '      _eventsCount: 12,\n' +
  '      connecting: false,\n' +
  '      _hadError: false,\n' +
  '      _parent: null,\n' +
  "      _host: 'test',\n" +
  '      _readableState: [ReadableState],\n' +
  '      _maxListeners: undefined,\n' +
  '      _writableState: [WritableState],\n' +
  '      allowHalfOpen: false,\n' +
  '      _sockname: null,\n' +
  '      _pendingData: null,\n' +
  "      _pendingEncoding: '',\n" +
  '      server: undefined,\n' +
  '      _server: null,\n' +
  '      ssl: null,\n' +
  '      _requestCert: true,\n' +
  '      _rejectUnauthorized: true,\n' +
  '      parser: null,\n' +
  '      _httpMessage: [Circular *1],\n' +
  '      write: [Function: writeAfterFIN],\n' +
  '      [Symbol(res)]: null,\n' +
  '      [Symbol(verified)]: true,\n' +
  '      [Symbol(pendingSession)]: null,\n' +
  '      [Symbol(async_id_symbol)]: 3141,\n' +
  '      [Symbol(kHandle)]: null,\n' +
  '      [Symbol(kSetNoDelay)]: false,\n' +
  '      [Symbol(lastWriteQueueSize)]: 0,\n' +
  '      [Symbol(timeout)]: null,\n' +
  '      [Symbol(kBuffer)]: null,\n' +
  '      [Symbol(kBufferCb)]: null,\n' +
  '      [Symbol(kBufferGen)]: null,\n' +
  '      [Symbol(kCapture)]: false,\n' +
  '      [Symbol(kBytesRead)]: 980,\n' +
  '      [Symbol(kBytesWritten)]: 23927,\n' +
  '      [Symbol(connect-options)]: [Object],\n' +
  '      [Symbol(RequestTimeout)]: undefined\n' +
  '    },\n' +
  "    _header: 'POST /test HTTP/1.1\\r\\n' +\n" +
  "      'Accept: application/json, text/plain, */*\\r\\n' +\n" +
  "      'content-type: multipart/form-data; boundary=--------------------------084245449142421267498946\\r\\n' +\n" +
  "      'Authorization: xxx\\r\\n' +\n" +
  "      'User-Agent: axios/0.27.2\\r\\n' +\n" +
  "      'Host: test\\r\\n' +\n" +
  "      'Connection: close\\r\\n' +\n" +
  "      'Transfer-Encoding: chunked\\r\\n' +\n" +
  "      '\\r\\n',\n" +
  '    _keepAliveTimeout: 0,\n' +
  '    _onPendingData: [Function: noopPendingOutput],\n' +
  '    agent: Agent {\n' +
  '      _events: [Object: null prototype],\n' +
  '      _eventsCount: 2,\n' +
  '      _maxListeners: undefined,\n' +
  '      defaultPort: 443,\n' +
  "      protocol: 'https:',\n" +
  '      options: [Object],\n' +
  '      requests: {},\n' +
  '      sockets: {},\n' +
  '      freeSockets: {},\n' +
  '      keepAliveMsecs: 1000,\n' +
  '      keepAlive: false,\n' +
  '      maxSockets: Infinity,\n' +
  '      maxFreeSockets: 256,\n' +
  "      scheduling: 'lifo',\n" +
  '      maxTotalSockets: Infinity,\n' +
  '      totalSocketCount: 0,\n' +
  '      maxCachedSessions: 100,\n' +
  '      _sessionCache: [Object],\n' +
  '      [Symbol(kCapture)]: false\n' +
  '    },\n' +
  '    socketPath: undefined,\n' +
  "    method: 'POST',\n" +
  '    maxHeaderSize: undefined,\n' +
  '    insecureHTTPParser: undefined,\n' +
  "    path: '/test',\n" +
  '    _ended: false,\n' +
  '    res: IncomingMessage {\n' +
  '      _readableState: [ReadableState],\n' +
  '      _events: [Object: null prototype],\n' +
  '      _eventsCount: 4,\n' +
  '      _maxListeners: undefined,\n' +
  '      socket: [TLSSocket],\n' +
  '      httpVersionMajor: 1,\n' +
  '      httpVersionMinor: 1,\n' +
  "      httpVersion: '1.1',\n" +
  '      complete: false,\n' +
  '      headers: [Object],\n' +
  '      rawHeaders: [Array],\n' +
  '      trailers: {},\n' +
  '      rawTrailers: [],\n' +
  '      aborted: true,\n' +
  '      upgrade: false,\n' +
  "      url: '',\n" +
  '      method: null,\n' +
  '      statusCode: 400,\n' +
  "      statusMessage: 'BAD REQUEST',\n" +
  '      client: [TLSSocket],\n' +
  '      _consuming: true,\n' +
  '      _dumped: false,\n' +
  '      req: [Circular *1],\n' +
  "      responseUrl: 'https://test-endpoint.com/file',\n" +
  '      redirects: [],\n' +
  '      [Symbol(kCapture)]: false,\n' +
  '      [Symbol(RequestTimeout)]: undefined\n' +
  '    },\n' +
  '    aborted: false,\n' +
  '    timeoutCb: null,\n' +
  '    upgradeOrConnect: false,\n' +
  '    parser: null,\n' +
  '    maxHeadersCount: null,\n' +
  '    reusedSocket: false,\n' +
  "    host: 'test',\n" +
  "    protocol: 'https:',\n" +
  '    _redirectable: Writable {\n' +
  '      _writableState: [WritableState],\n' +
  '      _events: [Object: null prototype],\n' +
  '      _eventsCount: 3,\n' +
  '      _maxListeners: undefined,\n' +
  '      _options: [Object],\n' +
  '      _ended: true,\n' +
  '      _ending: true,\n' +
  '      _redirectCount: 0,\n' +
  '      _redirects: [],\n' +
  '      _requestBodyLength: 22677,\n' +
  '      _requestBodyBuffers: [],\n' +
  '      _onNativeResponse: [Function (anonymous)],\n' +
  '      _currentRequest: [Circular *1],\n' +
  "      _currentUrl: 'https://test-endpoint.com/file',\n" +
  '      [Symbol(kCapture)]: false\n' +
  '    },\n' +
  '    [Symbol(kCapture)]: false,\n' +
  '    [Symbol(kNeedDrain)]: true,\n' +
  '    [Symbol(corked)]: 0,\n' +
  '    [Symbol(kOutHeaders)]: [Object: null prototype] {\n' +
  '      accept: [Array],\n' +
  "      'content-type': [Array],\n" +
  '      authorization: [Array],\n' +
  "      'user-agent': [Array],\n" +
  '      host: [Array]\n' +
  '    }\n' +
  '  }\n' +
  '}')
    at new NodeError (internal/errors.js:322:7)
    at Worker.emit (events.js:389:17)
    at Worker.emit (domain.js:475:12)
    at Worker.[kOnErrorMessage] (internal/worker.js:289:10)
    at Worker.[kOnMessage] (internal/worker.js:300:37)
    at MessagePort.<anonymous> (internal/worker.js:206:57)
    at MessagePort.[nodejs.internal.kHybridDispatch] (internal/event_target.js:399:24)
    at MessagePort.exports.emitMessage (internal/per_context/messageport.js:18:26)
    at MessagePort.callbackTrampoline (internal/async_hooks.js:130:17)
Waiting for the debugger to disconnect...

I think maybe the issue is related to request headers not being set properly (content-type and content-length) on all requests especially when file is large

@impro5
Copy link

impro5 commented Aug 4, 2022

yes ,is not work when the data type is formdata. There is a code in /axios/lib/adapters/xhr.js file that doesn't work。
if (utils.isFormData(requestData) && utils.isStandardBrowserEnv()) { delete requestHeaders['Content-Type']; // Let the browser set it }
If the data format is formdata, it will be converted to other data types before this code. So this code is invalid now. The converted location is in the /axios/lib/core/dispatchRequest.js file。
The specific code is as follows
config.data = transformData.call( config, config.data, config.headers, config.transformRequest );
If you want the top code to take effect, you need to adjust this code as follows
if(!utils.isFormData(config.data)){ config.data = transformData.call( config, config.data, config.headers, config.transformRequest ); }
@colesiegel

@moseskamau338
Copy link

if(!utils.isFormData(config.data)){ config.data = transformData.call( config, config.data, config.headers, config.transformRequest ); }

How do I use this without touching the core axios stuff?

@colesiegel
Copy link
Author

I am wondering the same. I'm not about to change internal implementation of an HTTP client, just to get the correct status code back.

@moseskamau338
Copy link

Aparently, this solution

axios.post(url, formData, {
  headers: { 'Content-Type': 'multipart/form-data' },
  transformRequest: formData => formData,
})

Specifically this line:
transformRequest: formData => formData,
worked for me.

It makes sense since it assumes data transformation but then returns the same value🤔

@moseskamau338
Copy link

Actually, it's sending [object object]... I don't think that is a file

@moseskamau338
Copy link

So I managed to get this working. I had to create a totally new instance like so:

axios.post(
               '/endpoint/here',
               formData, /*this has to be the exact formData object. DO NOT DE-STRUCTURE*/
               {
                   headers: {
                       'Authorization': `Bearer ${'token'}`, 
                       'Content-Type': 'multipart/form-data'
                   },
                   ...options
               }
           )

Am not sure why this works and not the pre-instantiated one

@mhesham93
Copy link

mhesham93 commented Aug 22, 2022

Been trying to make this work for over 5 hours now and still stuck. not sure what I'm missing.

The only difference is that I convert the a buffer to blob first.

I get this before the request

FormData { [Symbol(state)]: [ { name: 'file', value: [File] } ] }

and that's what I get after the error

data: FormData { [Symbol(state)]: [Array] }

Not sure why this happens.

@rcbevans
Copy link

rcbevans commented Oct 7, 2022

So I managed to get this working. I had to create a totally new instance like so:

axios.post(
               '/endpoint/here',
               formData, /*this has to be the exact formData object. DO NOT DE-STRUCTURE*/
               {
                   headers: {
                       'Authorization': `Bearer ${'token'}`, 
                       'Content-Type': 'multipart/form-data'
                   },
                   ...options
               }
           )

Am not sure why this works and not the pre-instantiated one

After upgrading to axios ^1.1.0 from ^0.27.2 in the browser I am seeing multipart/form-data uploads that previously worked breaking with my backend returning HTTP 413 Payload too large.

On inspection of the request headers, in the working version, the body was being sent as multipart/form-data as expected, but version 1.1.0 was changing the content-type to application/x-www-form-urlencoded for some reason.

export const FileClient = axios.create({
  baseURL: '/api',
  headers: {
    'Content-type': 'multipart/form-data',
  },
});

...

const formData = new FormData();
formData.append('file', file, file.name);
const result = await FileClient.post<string>(url, formData, {
  headers: {
    Authorization: `Bearer ${token}`,
  },
});

For whatever reason, using a non-precreated instance works...

import axios from 'axios';

...

const formData = new FormData();
formData.append('file', file, file.name);
const result = await axios.post<string>(url, formData, {
  baseURL: '/api',
  headers: {
    Authorization: `Bearer ${token}`,
  },
});

@rcbevans
Copy link

rcbevans commented Oct 7, 2022

Fixing capitalization of the 'Content-Type' header the request seems to work as expected.

export const FileClient = axios.create({
  baseURL: '/api',
  headers: {
    'Content-Type': 'multipart/form-data',
  },
});

@dreamingofcows
Copy link

dreamingofcows commented Aug 14, 2023

I am using Axios 1.4 (also tried with 1.3.4) and the work around still does not work for me:

If I look at the actual request payload it still just sends the string "[object Object]" instead of the actual data... help...

axiosConfig
      .post(
        `/url`,
        applicationFormData,
        {
          transformRequest: () => applicationFormData,
        }
      );
      

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants