Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

Use a shell script for incremental backups UNIX
This shell script is a modification of a script I've been using on Unix systems since the late 80's. I recently updated it for use on 10.3, and it should work on 10.4 as well. Here's a reformatted version of the instructions, which are commented in the script:

Description: UNIX Incremental Backup script

This script will back up any files newer then a touch file located in /, and will log each file it backs up. Log files are overwritten once a week.
  1. To use this script, create the following directories: /usr/local/bin and /usr/local/logs.
  2. Then execute the command touch /.inc_mark.
  3. Create a full backup of the dirs you want to do incremental backups for, using a command similar to this: find /Users -print | cpio -pdumv /Volumes/Backup
  4. Schedule the script using crontab, launchd, or something similar.
[robg adds: While I haven't tested the full script, I did test the cpio command to see how it handles files with resource forks (there's no mention of such files in its man pages). Happily, it seemed to handle them just fine, as I was able to open a selection of "resource forked" files that I copied across drives via the cpio command. Please read the script and modify as necessary -- there are some hard-coded paths you'll want to replace. You'll also need to use sudo to create the above-mentioned directories.]
    •    
  • Currently 2.00 / 5
  You rated: 1 / 5 (4 votes cast)
 
[15,627 views]  

Use a shell script for incremental backups | 10 comments | Create New Account
Click here to return to the 'Use a shell script for incremental backups' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
Use a shell script for incremental backups
Authored by: boredzo on Oct 07, '05 07:00:37AM

UNIX utilities like cpio only handle resource forks on 10.4 and later.



[ Reply to This | # ]
Incomplete backups
Authored by: Mike F. on Oct 07, '05 08:02:52AM

This script has a chance of missing modified files.

The reference date (/.inc_mark) is set after the whole backup is completed. So some files modified while the backup is taking place will not be backed up the next time the script runs (unless they are modified again of course).

I haven't tried this but a possible solution might be to note the reference time before the backup starts in a second file, do the backup and finally set the real reference file to have the modified date noted before. That's still not perfect as some files might be backed up too often but it's better than failing to backup files.



[ Reply to This | # ]
Incomplete backups
Authored by: plmc on Oct 13, '05 06:43:29PM

Valid point... but I generally schedule this for 2:45 am... Very rarely do I do any work at that time, so the chances of missing the files that are created during the script execution is low.



[ Reply to This | # ]
A quicker way?
Authored by: jgl24 on Oct 07, '05 08:39:16AM

I really like the idea of this, but unless I'm mistaken, there's no compression going on here, so... it would be a lot easier to use rsync.

rsync -vrut $SRCDIR $BACKUPDIR >> $LOGFILE



[ Reply to This | # ]
Use a shell script for incremental backups
Authored by: vocaro on Oct 07, '05 09:57:37AM
There is also a script called rdiff-backup with more features than this one, especially when it comes to restoring files.

[ Reply to This | # ]
Use a shell script for incremental backups
Authored by: bluehz on Oct 07, '05 03:19:23PM

One warning about rdiff-backup. I tried it this week and was a little concerned with the fact that all the files that are backed up have their names changed. This is the mechanism that rdiff-backup uses to indicate the date/version of the file in backup. While that is probably fine - it means you can't just open up a backup and drag a file off, unless you want to go about renaming the files to their original names. I can only assume that rdiff-backup would actually rename the files for you back to their original names if you went through the actual rdiff-backup process of restoring.



[ Reply to This | # ]
Use a shell script for incremental backups
Authored by: sjk on Oct 07, '05 07:37:56PM
Use a shell script for incremental backups
Authored by: paulsomm on Oct 10, '05 08:46:34AM

Are you sure? I use rdiff-backup and regularly pull individual files back by simply copying them from the backup folder to my harddrive.

There is a funky named folder underwhich are files with changed names, but this is the diff folder. There is a root folder that is an exact copy of the source. In this other folder are the diffs for each file in that root. If you need to get back file versions other than what's at the root, you need to run rdiff-backup in restore mode to apply the diffs to the file as it saves it elsewhere.



[ Reply to This | # ]
cpio ignores resource forks
Authored by: grikdog on Oct 10, '05 10:57:56AM

At least for Mac OS X 10.3 and prior, cpio does NOT copy resource forks. It creates zero-length directory entries on the destination volume, instead of copying Classic applications. Forget cpio; if you must use this method, use hfstar, found at http://www.metaobject.com/downloads/macos-x/



[ Reply to This | # ]
cpio ignores resource forks
Authored by: raster on Oct 10, '05 02:53:28PM

Can't you just use ditto?



[ Reply to This | # ]