2011-10-03

Smart Backup on Linux

About two years ago after setting up a Linux server I was asked to create a backup of the last seven days. Reason for this was the idea that if something gets deleted or corrupted it might not be noticed immediately.

The problem was: The hard disk did not have enough free space to hold 7 times the data and at the moment there was no external hard disk available.

I searched the Internet and found a really smart solution - you can find details here on mikerubel.org (rsync snapshots). My final script looks like this:

 #!/bin/bash  
 rm -rf /data/autobackup/backup.7  
 mv /data/autobackup/backup.6 /data/autobackup/backup.7  
 mv /data/autobackup/backup.5 /data/autobackup/backup.6  
 mv /data/autobackup/backup.4 /data/autobackup/backup.5  
 mv /data/autobackup/backup.3 /data/autobackup/backup.4  
 mv /data/autobackup/backup.2 /data/autobackup/backup.3  
 mv /data/autobackup/backup.1 /data/autobackup/backup.2  
 cp -alv /data/autobackup/backup.0 /data/autobackup/backup.1 > /data/autobackup/0to1.log  
 rsync -av --delete /data/live/ /data/autobackup/backup.0/ > /data/autobackup/last-rsync.log  


If you want to go more into detail - Michael Jakl has posted a variant here (rsync time machine).

Currently in my use case the live part takes 149GB and the whole backup of 7 days takes 161 GB (instead of more than a TB!)

I know, disk space is quite cheap, but the rising amount of data also requires a lot! - In my case:
  1. I already have several external drives (2 HDs where HD images of my work notebook are saved alternating).
  2. Two separate external drives where data only is saved with rsync (to one disk more often to other about once in two months).
  3. I have a 500GB external HD where data is outsourced that I do not need that much or simply consumes too much of my primary HD. That one was not saved in the early times. In the meantime I do sync it to a second one from time to time (people have a lot of external HDs these days but I am not sure if they backup those too...).
Guess what, if I would decide to keep snapshots of several days - way too much for a normal copy x7!