serve sftp: upload fails (“Illegal seek”, SSH_FX_FAILURE)

[thomae]

The associated forum post URL from https://forum.rclone.org

https://forum.rclone.org/t/serve-sftp-upload-fails-illegal-seek-ssh-fx-failure/27238

What is the problem you are having with rclone?

Uploading a file using the rclone SFTP backend to the rclone serve sftp server fails with "Illegal seek" (SSH_FX_FAILURE).
As a workaround you can either add --sftp-disable-concurrent-writes to the client or --vfs-cache-mode writes to the server.
As discussed in the forum it is suspected that a rewrite of the file upload code of the upstream sftp library broke it as it worked with v1.55.1.

What is your rclone version (output from rclone version)

v1.56.2 or v1.57.0-beta.5823.da8f9be84

Which OS you are using and how many bits (e.g. Windows 7, 64 bit)

Debian 10 Buster, 64 bit

Which cloud storage system are you using? (e.g. Google Drive)

Local Filesystem

The command you were trying to run (e.g. rclone copy /tmp remote:tmp)

Server & Client

wget https://downloads.rclone.org/v1.56.2/rclone-v1.56.2-linux-amd64.zip
unzip rclone-v1.56.2-linux-amd64.zip
cd rclone-v1.56.2-linux-amd64/

Server only

mkdir server-folder
./rclone serve sftp server-folder --no-auth --log-file rclone-server.log -vvv

Client only

mkdir client-folder
fallocate -l 100MB client-folder/100MB.test
./rclone copy --sftp-host 127.0.0.1 --sftp-port 2022 --sftp-ask-password client-folder :sftp: --log-file rclone-client.log -vvv

For the beta version I updated with

./rclone selfupdate --beta

and then repeated the commands above.

A log from the command with the -vvv flag (e.g. output from rclone -vv copy /tmp remote:tmp)

• stable (v1.56.2, client & server log)
• beta (v1.57.0-beta.5823.da8f9be84, client & server log)

How to use GitHub

• Please use the 👍 reaction to show that you are affected by the same issue.
• Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
• Subscribe to receive notifications on status change and new comments.

[ncw]

This problem is easy to replicate.

The problem is in the client rather than the server it is easy to work out and a bit of git bisect shows that this is the problem commit

commit 0537791
Author: @ncw
Date: Mon Apr 5 10:32:20 2021 +0100

sftp: Fix performance regression by re-enabling concurrent writes #5197

Disabling the feature introduced by that commit with --sftp-disable-concurrent-writes also fixes the problem as @thomae noted.

The root cause of this problem is in the github.com/pkg/sftp library which gets the packets out of order when transmitting. I'm going to see if I can fix this!

[ncw]

This has now been fixed upstream in pkg/sftp#482

I've pulled that into rclone now - can you give it a go and see if it works for you @thomae ?

v1.58.0-beta.5935.f1b9f020c.fix-5806-serve-sftp on branch fix-5806-serve-sftp (uploaded in 15-30 mins)

[ncw]

I've merged this to master now which means it will be in the latest beta in 15-30 minutes and released in v1.58