rsync stands for remote sync.
rsync is used to perform the backup operation in UNIX / Linux.
rsync utility is used to synchronize the files and directories from one location to another in an effective way. Backup location could be on local server or on remote server
1.0 Important features of rsync
- Speed: First time, rsync replicates the whole content between the source and destination directories. Next time, rsync transfers only the changed blocks or bytes to the destination location, which makes the transfer really fast.
- Security: rsync allows encryption of data using ssh protocol during transfer.
- Less Bandwidth: rsync uses compression and decompression of data block by block at the sending and receiving end respectively. So the bandwidth used by rsync will be always less compared to other file transfer protocols.
- Privileges: No special privileges are required to install and execute rsync
$ rsync options source destination
Source and destination could be either local or remote. In case of remote, specify the login name, remote server name and location.
Example 1. Synchronize Two Directories in a Local Server
To sync two directories in a local computer, use the following rsync -zvr command.
$ rsync -zvr /var/opt/installation/inventory/ /root/tempIn the above rsync example:
- -z is to enable compression
- -v verbose
- -r indicates recursive
Example 2. Preserve timestamps during Sync using rsync –a
rsync option -a indicates archive mode. -a option does the following,
- Recursive mode
- Preserves symbolic links
- Preserves permissions
- Preserves timestamp
- Preserves owner and group
$ rsync -azv /var/opt/installation/inventory/ /root/temp/
Example 3. Synchronize Only One File
To copy only one file, specify the file name to rsync command, as shown below.
$ rsync -v /var/lib/rpm/Pubkeys /root/temp/
Example 4. Synchronize Files From Local to Remote
rsync allows you to synchronize files/directories between the local and remote system.
$ rsync -avz /root/temp/ firstname.lastname@example.org:/home/thegeekstuff/temp/
Example 5. Synchronize Files From Remote to Local
When you want to synchronize files from remote to local, specify remote path in source and local path in target as shown below.
$ rsync -avz email@example.com:/var/lib/rpm /root/temp
Example 6. Remote shell for Synchronizationrsync allows you to specify the remote shell which you want to use. You can use rsync ssh to enable the secured remote connection.
Use rsync -e ssh to specify which remote shell to use. In this case, rsync will use ssh.
$ rsync -avz -e ssh firstname.lastname@example.org:/var/lib/rpm /root/temp
Example 7. Do Not Overwrite the Modified Files at the DestinationIn a typical sync situation, if a file is modified at the destination, we might not want to overwrite the file with the old file from the source.
Use rsync -u option to do exactly that. (i.e do not overwrite a file at the destination, if it is modified). In the following example, the file called Basenames is already modified at the destination. So, it will not be overwritten with rsync -u.
$ rsync -avzu email@example.com:/var/lib/rpm /root/temp
Example 8. Synchronize only the Directory Tree Structure (not the files)
Use rsync -d option to synchronize only directory tree from source to the destination. The below example, synchronize only directory tree in recursive manner, not the files in the directories.
$ rsync -v -d firstname.lastname@example.org:/var/lib/ .
Example 9. View the rsync Progress during TransferWhen you use rsync for backup, you might want to know the progress of the backup. i.e how many files are copies, at what rate it is copying the file, etc.
rsync –progress option displays detailed progress of rsync execution as shown below.
$ rsync -avz --progress email@example.com:/var/lib/rpm/ /root/temp/
Example 10. Delete the Files Created at the TargetIf a file is not present at the source, but present at the target, you might want to delete the file at the target during rsync.
In that case, use –delete option as shown below. rsync delete option deletes files that are not there in source directory.
# Source and target are in sync. Now creating new file at the target.
$ > new-file.txt
Example 11. Do not Create New File at the TargetIf you like, you can update (Sync) only the existing files at the target. In case source has new files, which is not there at the target, you can avoid creating these new files at the target. If you want this feature, use –existing option with rsync command.
First, add a new-file.txt at the source.
[/var/lib/rpm ]$ > new-file.txt
$ rsync -avz --existing firstname.lastname@example.org:/var/lib/rpm/ .
Example 12. View the Changes Between Source and DestinationThis option is useful to view the difference in the files or directories between source and destination.
At the source:
$ ls -l /var/lib/rpm
-rw-r--r-- 1 root root 5357568 2010-06-24 08:57 Basenames
-rw-r--r-- 1 root root 12288 2008-05-28 22:03 Conflictname
-rw-r--r-- 1 root root 1179648 2010-06-24 08:57 DirnamesAt the destination:
$ ls -l /root/temp
-rw-r--r-- 1 root root 12288 May 28 2008 Conflictname
-rw-r--r-- 1 bin bin 1179648 Jun 24 05:27 Dirnames
-rw-r--r-- 1 root root 0 Sep 3 06:39 BasenamesIn the above example, between the source and destination, there are two differences. First, owner and group of the file Dirname differs. Next, size differs for the file Basenames.
Now let us see how rsync displays this difference. -i option displays the item changes.
$ rsync -avzi email@example.com:/var/lib/rpm/ /root/temp/
receiving file list ... done
sent 48 bytes received 2182544 bytes 291012.27 bytes/sec
total size is 45305958 speedup is 20.76In the output it displays some 9 letters in front of the file name or directory name indicating the changes.
In our example, the letters in front of the Basenames (and Dirnames) says the following:
> specifies that a file is being transferred to the local host.
f represents that it is a file.
s represents size changes are there.
t represents timestamp changes are there.
o owner changed
g group changed.
Example 13. Include and Exclude Pattern during File Transferrsync allows you to give the pattern you want to include and exclude files or directories while doing synchronization.
$ rsync -avz --include 'P*' --exclude '*' firstname.lastname@example.org:/var/lib/rpm/ /root/temp/In the above example, it includes only the files or directories starting with ‘P’ (using rsync include) and excludes all other files. (using rsync exclude ‘*’ )
Example 14. Do Not Transfer Large Files
You can tell rsync not to transfer files that are greater than a specific size using rsync –max-size option.
$ rsync -avz --max-size='100K' email@example.com:/var/lib/rpm/ /root/temp/
max-size=100K makes rsync to transfer only the files that are less than or equal to 100K. You can indicate M for megabytes and G for gigabytes.
Example 15. Transfer the Whole FileOne of the main feature of rsync is that it transfers only the changed block to the destination, instead of sending the whole file.
If network bandwidth is not an issue for you (but CPU is), you can transfer the whole file, using rsync -W option. This will speed-up the rsync process, as it doesn’t have to perform the checksum at the source and destination.
# rsync -avzW firstname.lastname@example.org:/var/lib/rpm/ /root/temp
# rsync -av --no-R --files-from=/tmp/a.txt / root@ip:/home/
--no-R = No relative path