Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve SFTPStorage #1194

Merged
merged 4 commits into from Nov 9, 2022
Merged

Improve SFTPStorage #1194

merged 4 commits into from Nov 9, 2022

Commits on Nov 9, 2022

  1. Move seekability test to a separate helper function 

    Having a third copy of this code would feel silly.
    @vainu-arto
    vainu-arto committed Nov 9, 2022
    Configuration menu
    Copy the full SHA
    75c0063 View commit details
    Browse the repository at this point in the history

  2. Stop calling content.open() from SFTPStorage._save 

    The open method isn't available in all file-like objects one might want to
save using the storage framework, so instead of relying on it calling
seek(0) for us do it ourselves (conditionally).
    @vainu-arto
    vainu-arto committed Nov 9, 2022
    Configuration menu
    Copy the full SHA
    af38ba5 View commit details
    Browse the repository at this point in the history

  3. Require at least version 1.10.0 of paramiko 

    This version introduced the SFTPClient.putfo() method. It was released in
early 2013, so this shouldn't be a great hardship.
    @vainu-arto
    vainu-arto committed Nov 9, 2022
    Configuration menu
    Copy the full SHA
    094240f View commit details
    Browse the repository at this point in the history

  4. Use SFTPClient.putfo() in SFTPStorage._save() 

    Instead of using SFTPClient.open() to get a file-like object, reading the
entire content into memory and then calling .write() on that.

The measured performance difference is stunning, and the uploaded file data
no longer needs to fit into memory.
    @vainu-arto
    vainu-arto committed Nov 9, 2022
    Configuration menu
    Copy the full SHA
    a348add View commit details
    Browse the repository at this point in the history