Advanced Scheduled Synchronization with TurboFTP

1. Concepts

Source and Destination

The source and destination folders are defined based on the synchronization direction. For example if the sync direction is Download, the remote folder is the source folder, and the local folder is the destination folder.

 


2. Synchronization Schemes




Synchronization Schemes define how TurboFTP chooses files to transfer after the file comparision is done. We will explain each of the available Synchronization Schemes in TurboFTP by examples.


2.1 Add only new files to destination

Only transfer new source files.

EXAMPLE 1: Only transfer new source files. No file time stamp comparison is involved.


2.2 Update only existing files at destination


Transfer files that overwrite old copies in destination folder.

EXAMPLE 2: Under this scheme TurboFTP will compare local and remote file time stamps. In this example, the local file root.txt exists at local and at remote and the local copy is newer. After the synchronization, the local copy of root.txt is uploaded while test.php is not.

direction

2.3 Update and add new files to destination

Transfer new source files and updated source files that overwrite old copies in destination folder.

EXAMPLE 3: the local copy of root.txt is newer and after synchronization it is uploaded. In the local folder has test.php which the remote destination folder doesn't have, therefore test.php is uploaded as new file.

Before action:

After action:


2.4 Update, add new and remove unique files at destination


Transfer new source files and updated source files that overwrite old copies in destination folder and remove orphan files in destination folder. Unique/orphan files are files that exist at destination but not at source.

EXAMPLE 4: Note that 'have.rar' at remote doesn't exist at local as is seen as a unique file. Therefore it is removed after the action.

Before action:

direction


After action:



2.5 Transfer files added or updated since last sync

Compare time stamps of source files with last run time of the task, if it is newer then transfer them to destination. With this scheme, file size comparison is not applicable. The first time the task runs, all source files will be transferred since there is no valid 'last runtime'.

EXAMPLE 5:

After the 1st excution of task:

direction

Then root.txt is modified and the task is executed again. This time the file 'test.php' is uploaded because its time stamp is newer than the last run time.


2.6 Remove duplicate files at destination


Find out files that exist at both local and remote and remove the copies at destination.

Before action:

direction

After action:

2.7 Transfer all source files

Transfer all files in source folder overwriting any existing files at destination.

EXAMPLE 6: as illustrated below, all files are transferred regardless whether the destination folder has them or not.

Before action:

direction

After action:

2.8 Transfer all source files and remove unique files at destination

Transfer all files in source folder overwriting any existing files at destination. Remove orphan files in destination folder. This is an option for 100% mirroring without caring about the result of file comparisons, in terms of either file time or file size, between both sides.

EXAMPLE 7:

Before action: the orphan file 'have_no.rar' at destination is removed after the action.

direction

After action:


3. Other Options of Synchronization


Explanation of the above options:

3.1 Include subfolders: whether synchronize subfolders as well.

3.2 Ignore file name case: when comparing local and remote files, ignore file name case difference.

3.3 Compare file size rather than file time: by default, TurboFTP compares file time stamps during synchronization. Due to the limitation of regular FTP, where time in remote file listing is within the accuracy of minute, you might want to decide which files to transfer based on the file size difference in bytes. Note that is this option is selected and the source file size is different from the destination file size, TurboFTP will consider the source file is 'newer'.

3.4 The last two options only relate to visual comparison of Folder Synchronization in TurboFTP and will be ignored when doing automated synchronizations.


4. Execute external programs during synchronization


TurboFTP allows you to interface your own workflow processes with its synchronization action. External programs can be executed before task, after task and against each downloaded file(locally) or uploaded file.

4.1 Execute Programs before Task

EXAMPLE 8: Pack local files into an archive, and upload the archive to remote server.

We create a batch file to do archiving with a command line archiver like RAR.

archive.bat
"C:\Program Files\WinRAR\Rar.exe" a "C:\123\test\test.rar" "C:\123\test\test.php"

As illustrated below, we pecify archive.bat to be executed before task:

As we only want to upload archives, we set up an inclusion file filter:



Before the task runs, both the local and the remote folders have the same set of files:


After it runs, the local files are packed into test.rar and uploaded:

4.2 Run program after task

EXAMPLE 9: Download remote archive and extract it in a local folder.

expand.bat
"C:\Program Files\WinRAR\UnRAR.exe" e "C:\123\test\test.rar" "C:\123\test\"

We specify expand.bat to be run after task.

Before action, the local destination folder is empty. We use a sync task to download archives to the local folder, then extract all files from archive test.rar.


After action:

 

4.3 Execute external program agains each downloaded file

EXAMPLE 10: Add each downloaded file to archivse

We have an archive back.rar and will add each downloaded file to it.

addarchive.bat
"C:\Program Files\WinRAR\Rar.exe" a "C:\backup\backup.rar" %1

Before task runs:


When the task done, files are downloaded and added to the archive:

You can also delete or move each downloaded or uploaded file.

Please note under After Download the path we move files to is a remote folder path.

4.4 Execute program against each uploaded file

EXAMPLE 11: Extract a backup archive and uploaded extracted files to remote folder.

We use the following script to extract a backup archive in a temporary folder.

expand.bat
"C:\Program Files\WinRAR\UnRAR.exe" e "C:\backup\backup.rar" "C:\123\test\"

While the task runs we delete each uploaded local file.

del.bat
del %1

Before action: the local and remote folders are empty.


After action:

You can also use TurboFTP's build funtion to delete/move uploaded files. This is preferable because the task log shows whether the external program completed successfully.