Posts Tagged ‘interrupted’

GNU / Linux: Resuming failed / interrupted – scp, sftp, ftp)file or directory upload with rsync

Saturday, October 20th, 2012

Rsync SSH Logo Linux Continue failed upload logo

You probably wondered how and if it is possible to Continue / Resume interrupted SFTP / FTP, SCP commands file transfer ?

Continuing a failed Upload is something, very useful especially for people like me who use Linux over wireless and there are constant failures with Internet, or just have to move quickly from a Wireless location to another one while sftp user@host upload was in progress. SFTP protocol was not planned with a continues upload in mind, just like web HTTP proto (POST) file upload was not. Thus there is no direct way to continue file upload using some embedded SCP / FTP protocol feature.
This is really bad especially, if you’re uploading some enormous 10 Gigabytes files and upload interrupts on 95% percentage 😉

Thanksfully though default protos does not support continues downloads rsync does!

there is a possibility to continue SCP failed uploads!
Using rsync, you can continue uploading any failed upload nomatter the protocol supports resume or not 🙂

To continue an (interrupted) SCP, SFTP (SSH) or (FTP) proto transmission with rsync

# rsync --rsh='ssh' -av --progress --partial interrupted_file_name_to_be_uploaded.rar root@UPLOAD-HOST.COM:/root

If you need to resume failed upload to SSH server running on unusual port number use, let’s say SSH listening for connections on port 2202:

# rsync --rsh='ssh -p 2202' -av --progress --partial interrupted_file_name_to_be_uploaded.rar root@UPLOAD-HOST.COM:/root

For efficiency of upload for lage files it is also useful to use rsync’s file compression capabilities with -z switch:

# rsync --rsh='ssh -p 2202' -avz --progress --partial interrupted_file_name_to_be_uploaded.rar root@UPLOAD-HOST.COM:/root

Sometimes its necessery to resume a failed ( upload ) transfer of a directory with some sub directory structure to do so:

# rsync --rsh='ssh -p 22' -avztrlpog --progress --partial interrupted_file_name_to_be_uploaded.rar

Here is short explained each rsync switches you see above:

-e ssh rsync will use ssh client instead of rsh
-z compress file transfer
-t preserve time (other attributes as owner or permissions are also possible)
-l copy symlinks as symlinks
-r recursive into subdirectories
-p preserve permissions
-o preserve owner
-g preserve group
-v verbose

-tplog key switches are very useful as they will keep file creation and modification times, exact permissions, owner and group permissions and copy symlinks correctly.