Tutorials

rsync HowTo

How To Backup Your Data The Easy Way

Here is how to use rsync as an easy and automatic way to backup your data on a server (Unix or Mac).

First some prerequisites:

  • You have rsync and tcsh installed.
  • Supposed your data is in “/srv/www” on the server.
  • Supposed your data should get backuped into a “backup” destination directory on an external drive that is mounted as “/Volumes/Drive”.

Then the following tcsh command will make an incremental backup of the data with rsync.
It will store deleted and changed files in a backup directory with a suffix that corresponds to the backup date: 

rsync -abv --delete --ignore-errors /srv/www/ /Volumes/Drive/backup --backup-dir=/Volumes/Drive/backup-`date '+%y%m%d'`

Be careful about specifiying the source and destination directories: Mixing up the directories can delete your data. Better give it a dry run on some test data before checking it out on your real data (using the option —dry-run).

Now we put that rsync command into a shell script, for example “backup.sh”: 

#!/bin/tcsh -f

set SRC="/srv/www" # list of directories to be included in the backup
set DST="/Volumes/Drive/backup"

set DATE=`date '+%Y%m%d'`

rsync -abv --delete --ignore-errors $SRC $DST --backup-dir=$DST-$DATE >> $DST-$DATE.log

Do not forget to set the executable bit via “chmod +x backup.sh”.

Hint: You can put multiple source directories (separated by spaces) into the SRC shell variable so that they are backuped in the directory given by the DST shell variable.

Next, we execute that shell script daily as a so called cron job. For that we need a so called “crontab”. An example crontab that is starting the backup script every night at 3am is the following: 

#min	hour	mday	month	wday	command
0	3	*	*	*	/etc/backup.sh

You can list your actual installed crontab with the following command: 

crontab -l

Now ew replace the crontab of the super user with the one above by editing the crontab of the root user: 

sudo crontab -e -u root

In case the EDITOR variable of sudo is not set to an editor you are comfortable with, we replace the editor, for example with nano: 

sudo sh -c 'EDITOR=nano; crontab -e -u root'

Finally, we move the backup shell script into /etc/ so that it can be executed by the cron daemon: 

sudo cp backup.sh /etc/

This way you get a safe copy of the actual data in the backup dir every day and all elder versions of deleted or updated files in a backup dir with the corresponding date suffix.

In order to disable the indexing of the backup drive:

sudo mdutil -i off /Volumes/Drive

In order to disable the indexing completely:

sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist

That’s it! Enjoy your backup! :)

In case of a data emergency, we use the following script to restore the backup data: 

#!/bin/tcsh -f

set SRC="/Volumes/Drive/backup"
set DST="/srv/www/" # list of directories to be restored"

echo the following commands are required for restoration:
foreach DIR ($DST)
   echo rsync -abv --delete --ignore-errors $SRC/$DIR:t $DIR:h --exclude-from=/Volumes/schorsch/data/backup.ex
end

How To Mirror A Disk The Easy Way

Another application of rsync is mirroring a file system.

For example, in a server setup we usually have two disks. One disk contains the OS and the second should contain a copy of the latest functional version of the operating system, so that a disk failure can be undone just by switching to the mirror disk, without the need to reinstall the server completely from scratch.

We use the following script for that purpose: 

#!/bin/tcsh -f

set SRC="/"
set DST="/Volumes/Mirror"

sudo rsync -av --exclude /Volumes --exclude /private/var/tmp --exclude /dev --delete --ignore-errors $SRC $DST

The script mirrors the root file system “/” to the mirror volume “/Volumes/Mirror”.

Options: