Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Preallocate buffer size when reading data #239

Merged

Commits on Jan 17, 2022

  1. Preallocate buffer size when reading data

    When small chunks are received from a large payload, i.e > 10MBs, performance
    gets impacted because due to the buffer allocation, for payloads of 10MBs with
    chunks of 2KBs it can take ~  20 seconds.
    
    We add an exponencial growth of the buffer providing enough room or subsequent
    chunks until a maximum threshold of 1MB.
    
    With that change we can see a x50 improvement with the trade off of having to
    allocate a little extra space, worst case scenario 1MB.
    pfreixes committed Jan 17, 2022
    Copy the full SHA
    86dc983 View commit details
    Browse the repository at this point in the history

Commits on Jan 19, 2022

  1. Change the threshold for 256KB

    Beyond that number there is no gain in the performance
    pfreixes committed Jan 19, 2022
    Copy the full SHA
    a9edd80 View commit details
    Browse the repository at this point in the history

Commits on Jan 20, 2022

  1. Copy the full SHA
    99714d5 View commit details
    Browse the repository at this point in the history
  2. Copy the full SHA
    dbd7d13 View commit details
    Browse the repository at this point in the history
  3. Copy the full SHA
    bab41d9 View commit details
    Browse the repository at this point in the history
  4. Less aggressive timeout

    pfreixes committed Jan 20, 2022
    Copy the full SHA
    ef3a4ee View commit details
    Browse the repository at this point in the history