Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

Create daily backups on an encrypted disk image System
With all of the recent news about new trojan programs that can delete a user's home folder, I thought I might share my backup routine, especially since many of the pieces I have assembeled came from knowledge picked up here. I run the following script when I go to sleep. It backs up a number of pieces of system data, like my Apache2 document root and my MySQL databases, to my home folder, and then archives my home folder to an encrypted disk image and stores it on my external FireWire drive. It then will continue and archive other specified user accounts to their own encrypted disks.

I've tried to provide as much commenting through the script as I could; if you copy and paste, make sure that you get the line breaks correct (three lines have been split here for easier reading). It has to be run as root or with sudo privledges, or as I have done, attached to the periodic entry for daily, /etc -> periodic -> daily -> 500.daily with the line sh /Users/User1/bin/DailyMaintenance. It also contains passwords, so it should be chmodded to 700 and stored somewhere safe.

I'm curious what other vital pieces of data that others might suggest backing up (/etc folder, /var/logs, etc.), and if anyone has any ideas on how to make this do its thing faster. It takes about three to four hours to back up the two home directories on my PowerBook, which are only about 5 gigabytes total.

Read the rest of the hint for the script ... it's also available on my website [4KB download].

#!/bin/sh

# User1
# Make archive directory in /Users/User1 to hold system data
echo "Deleting and recreating Daily Archive folder"
rm -rf /Users/User1/Archives/
mkdir /Users/User1/Archives/

# Remove any .DS_Store files and backup Document Root
echo "Removing .DS_Store files"
find /Library/Apache2/htdocs/. -iname ".DS_Store" -delete
echo "Archiving /Library/Apache2/htdocs"
tar czf /Users/User1/Archives/ApacheDocRoot.tar.gz /Library/Apache2/htdocs
echo "Archiving /Library/Apache2/conf"
tar czf /Users/User1/Archives/ApacheConfig.tar.gz /Library/Apache2/conf
echo "Archiving /Library/Apache2/logs"
tar czf /Users/User1/Archives/ApacheLogs.tar.gz /Library/Apache2/logs

# Dump and Zip up SQL Data
echo "Dumping Database Data"
# NEXT TWO LINES ARE ONE LONG LINE (use a space between the two parts)!
/Library/MySQL/bin/mysqldump --user=XXXXXXXX --password=XXXXXXXX -A >
  /Users/User1/Archives/SQLDump.txt
echo "Zipping up Database Data"
# NEXT TWO LINES ARE ONE LONG LINE (use a space between the two parts)!
tar czf /Users/User1/Archives/SQLDump.txt.tar.gz
  /Users/User1/Archives/SQLDump.txt
echo "Removing Dump File"
rm /Users/User1/Archives/SQLDump.txt

# Creating the backup disk image file and volume name
echo "Creating archived .dmg from /Users/User1"
SOURCE='/Users/User1'
FILEDEST='/Volumes/Backups/Archives'
VOLUMENAME=`date +%Y-%m-%d`_User1
IMAGENAME=$FILEDEST/$VOLUMENAME.dmg


# NEXT TWO LINES ARE ONE LONG LINE (use a space between the two parts)!
hdiutil create -srcfolder $SOURCE -encryption -passphrase XXXXXXXX
  -fs HFS+ -volname $VOLUMENAME $IMAGENAME

########################################

# User2
echo "Creating archived .dmg from /Users/User2"
SOURCE='/Users/User2'
FILEDEST='/Volumes/Backups/Archives'
VOLUMENAME=`date +%Y-%m-%d`_User2
IMAGENAME=$FILEDEST/$VOLUMENAME.dmg

# NEXT TWO LINES ARE ONE LONG LINE (use a space between the two parts)!
hdiutil create -srcfolder $SOURCE -encryption -passphrase XXXXXXXX
  -fs HFS+ -volname $VOLUMENAME $IMAGENAME

echo "Daily Backups Completed"
[robg adds: I haven't tested this one...]
    •    
  • Currently 1.00 / 5
  • 1
  • 2
  • 3
  • 4
  • 5
  (1 vote cast)
 
[16,431 views]  

Create daily backups on an encrypted disk image | 21 comments | Create New Account
Click here to return to the 'Create daily backups on an encrypted disk image' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
Create daily backups on an encrypted disk image
Authored by: roncross@cox.net on May 17, '04 12:23:02PM

Gee, I just use my .mac account and use the backup.app software that comes with it. I can save all of the important stuff that I need either to be saved to the idisk, cd, dvd, hard drive, external hard drive, etc...

No script for me in this case. Sure it is $99/year, but it is well worth the cost in my mind. What I like about the software is that my stuff is not backed up on the same computer that I use unless I specifically want to do that.

Besides that, I use the .mac account for so much more such as publishing my calendar, posting pictures and movies of events to my friends and family, and retrieving my emails from anywhere there is a computer connected to the web.

Not an issue for me. In this case, Apple got it right.
thx
RLC

---
rlc



[ Reply to This | # ]
Create daily backups on an encrypted disk image
Authored by: rhowell on May 17, '04 12:29:43PM

The only thing I don't like about Backup is that it won't copy any of my .files (dot-files), which is a real bummer if you use a lot of unix apps. SilverKeeper and others do, however, but of course they don't have any integration with my .Mac account.



[ Reply to This | # ]
Create daily backups on an encrypted disk image
Authored by: cclifton on May 17, '04 12:44:33PM

Gee, the original poster is backing up 5GB of data. .Mac will provide 1GB for $350 a year. Assuming they would even sell 5GB of storage, and that prices would scale linearly, that's $1750 a year.

I use .Mac for backing up some data; it's handy for what it is designed for. However, the original poster's script is a very useful contribution. Why do you feel the need to be a troll?

---
Curt



[ Reply to This | # ]
Create daily backups on an encrypted disk image
Authored by: roncross@cox.net on May 18, '04 02:45:05AM

Again if you have GBytes of information, you do not have to backup to an iDisk and pay the high cost for storage. You can also use backup to save to an external disk, dvd, etc...

I am not recommending that people pay a thousand dollars to back up their critical information. The main thing about backing up is to preserve file permission, ownership, modification date.

From what I understand, backing up HFS files and older MAC files using a unix script may not preserve file permission, ownership, modification dates.

Maybe the author of the script can speak to this issue of file permission.

thx
RLC

---
rlc



[ Reply to This | # ]
Create daily backups on an encrypted disk image
Authored by: eagle on May 17, '04 01:43:16PM

Nice hint. I was just this weekend beginning to look for a multi-user backup solution (preferably to a DVD/CD, but I can modify this to do that) and this hint will be a good starting point. Thanks.



[ Reply to This | # ]
Create daily backups on an encrypted disk image
Authored by: aqsalter on May 17, '04 09:04:15PM

How are you going to Backup to CD?
If you can provide this as a hint we may have a really great solution to the Backup dillemma (judging by the number of hints on backup solutions).



[ Reply to This | # ]
Create daily backups on an encrypted disk image
Authored by: eagle on May 18, '04 07:30:28AM
I'll probably do something like this.

[ Reply to This | # ]
Other solution
Authored by: germ on May 17, '04 03:40:03PM

I have a similar solution, but simpler (IMHO).
First, I do not want my backup stored in a compressed format. I want it exactly as my home folder looks like.
Second, I do not want to copy the whole home folder each time (yes, my home folder is big).
I use psync, which does the job nicely. I have written a very simple shell script that calls psync for each folder I want to backup and execute it at night through a cron job. In only takes a few minutes every night because I backup on a second hard drive (very fast and convenient).



[ Reply to This | # ]
Create daily backups on an encrypted disk image
Authored by: rhowell on May 17, '04 05:26:21PM

This is a great backup technique if you have filevault turned on (well, for one user that is). This way your backup is as secure as your home directory. Previous techniques mentioned won't preserve this security.



[ Reply to This | # ]
Create daily backups on an encrypted disk image
Authored by: Spades on May 17, '04 05:51:38PM
Actually, if you have filevault turned on and can take the time, then all you have to do is log out, log in as a different user, and copy the username.sparse file from your home directory to somewhere else. It'll take a lot less time that than creating a whole new image from your home directory. You just can't be logged in while making the copy.

[ Reply to This | # ]
Create daily backups on an encrypted disk image
Authored by: rhowell on May 17, '04 09:35:59PM

Brilliant! The image is already created for you when filevault is turned on!



[ Reply to This | # ]
Create daily backups on an encrypted disk image
Authored by: Spades on May 21, '04 10:43:13AM
Oh by the way, you can make an even better backup by converting to a compressed image. It will be smaller, and it may take less time than a straight copy. You still have to be logged out, but rather than the straight copy use the command:

hdiutil convert username.sparseimage -format UDZO -o somefilename

Add the -encryption flag if you want to keep the image encrypted. To restore the backup, follow the last half of this hint:

http://www.macosxhints.com/article.php?story=20031104223355900

[ Reply to This | # ]
CCC!
Authored by: astrodawg on May 17, '04 05:55:34PM

I use CarbonCopyCloner to duplicate back up my entire drive each morning. I have two 125gb drives. One is for use, one is for backup. Each morning CCC determines the differences between the two drives and makes the backup drive match the primary. CCC also repairs permissions before the backup so the drive is always in good repair.

Very easy. Cheap too!



[ Reply to This | # ]
CCC!
Authored by: Cameroon on May 17, '04 07:11:27PM

Well, the only flaw with this is that you're only "safe" for one day. If something goes wrong and isn't caught before the next morning (say you deleted a file that 2 days later you realize you shouldn't have), you're out of luck.



[ Reply to This | # ]
CCC!
Authored by: omnivector on May 17, '04 09:52:58PM

i was just about to give CCC a try, until i found out it doesn't backup to NFS volumes :/ screw that.

---
- Tristan



[ Reply to This | # ]
rsyncx
Authored by: pmccann on May 17, '04 09:38:36PM

This is just the sort of job that something like rsyncx will excel at. If you use the latest version of rsyncx you can easily set up a rotating set of backups (rolling through, say, the seven days of the week), with the following noteworthy feature: the common files among the rolling backups are hard links. That is, the data is only stored once, but you still have seven different "live" copies of your home directory (or your whole drive if you so desire). So it's like an incremental in terms of the storage required, but to the user it's exactly as if there were seven independent copies available.

*Very, very nice*. In addition, rsyncx (and psync or ditto for that matter) will preserve resource forks that might be crucial in some legacy apps/documents, and additional HFS+-specific file info that will be lost through the use of tar and cp.

The only downside to rsyncx is the relative paucity of the documentation: there is, however, a really nice video presentation on macosxlabs.org that shows how to use the various assistants and tools. It really deserves to be a whole lot better known than it seems to be. (Hmmm, maybe a good candidate for a hint or a "Pick of the Week"!)



[ Reply to This | # ]
rsyncx
Authored by: roncross@cox.net on May 18, '04 02:56:25AM

The first priority in backing up files is about preserving critical information first. After that, you want to preserve file permission, ownership, and modification date. Unix script are not very good at doing this for HFS+ file system and older Mac application files.

It seems that for unix configuration, system and dot files, this script maybe the way to go. But for HFS+ file system and some older Mac application files backup.app from apple is the way to go.

Apple has been aware of this problem for a long time now and are looking for solutions to fix it.

---
rlc



[ Reply to This | # ]
rsyncx
Authored by: pmccann on May 18, '04 11:26:48PM

Umm, I'm not sure how this relates to the parent comment (of mine): rsyncx is *perfectly* aware of all the hfs+ stuff, and permissions, and resource forks etc etc, and is orders of magnitude more flexible than the backup software apple distributes.

Paul



[ Reply to This | # ]
rsyncx
Authored by: sjk on May 19, '04 01:04:05AM

Anyone know whether or not rsyncx preserves Finder info? AFAIK ditto is the only utility that comes with OS X that does.



[ Reply to This | # ]
using backup.app from .MAC to backup dot-files
Authored by: roncross@cox.net on May 18, '04 05:12:18AM

(Authored by: rhowell on Mon, May 17 '04 at 12:29PM

The only thing I don't like about Backup is that it won't copy any of my .files (dot-files), which is a real bummer if you use a lot of unix apps. SilverKeeper and others do, however, but of course they don't have any integration with my .Mac account.)

This is not true, I was able to use backup.app to backup .files (dot-files) without any problems. The only thing that I wasn't allow to back up is the .Trash. But I personally don't see any reason to backup the trash right now.

You do this by going to find in finder and doing a search that queries for the items whose visibility is true and whose name starts with "."

Once you see the dot-file, then simply drag it over to the backup window and backup.app will back it up without any problems. If you look for it, you will not find it since all of the files that are being backed up are compressed and have a .gz extension in the backup folder.

thx
RLC

---
rlc



[ Reply to This | # ]
Create daily backups on an encrypted disk image
Authored by: White Fire on Jun 18, '04 11:34:20PM
It might be wise to replace the gzip lines to bzip2 to increase the compression rate if you have a bit more grunt and/or time. This will also mean you have to change a the standard tar extensions to tar cjf /path/to/filename.tbz . Note the .tar.gz file extension has been replaced with the .tbz file extension, and the z flag with the j flag. Please note, however, that the f flag must go last due to an oddity with the way tar likes its options.

Additionally, the next few lines of code can be replaced:


tar czf /Users/User1/Archives/SQLDump.txt.tar.gz
  /Users/User1/Archives/SQLDump.txt
echo "Removing Dump File"
rm /Users/User1/Archives/SQLDump.txt
Even after we change it like I said above, it still could use some pruning. Thusly:

bzip2 --best /Users/User1/Archives/SQLDump.txt
I think the second one has a certain elegance about it.

---
huwr@evula.org

[ Reply to This | # ]