Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!


Click here to return to the 'How should Time Machine be setup?' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
How should Time Machine be setup?
Authored by: cmcd on Nov 06, '09 01:25:14PM

First, thanks for this - definitely an improvement.

I followed the instructions and have the new Entourage sparsebundle in ~/Documents. My Time Machine backups are picking it up. In Time Machine preferences, the mounted sparsebundle is excluded from Time Machine backups. Is this correct or should the mounted sparsebundle be included.

And, how does a restore work? If I need to recover a previous version, do I just restore the entire sparsebundle file, replacing the current one in ~/Documents? Or can I somehow restore certain files within the sparsebundle file?

Regarding the risk of corruption, I've decided to run Time Machine manually, maybe twice a day, preceded by shutting down all Microsoft apps as recommended here http://blog.entourage.mvps.org/2009/02/backup_entourage.html#more



[ Reply to This | # ]
How should Time Machine be setup?
Authored by: cmcd on Nov 10, '09 02:13:49PM

I've followed the instructions and have Entourage Identities within a sparsebundle image file - but, my Time Machine backups show a full backup of the Identities file, not just the incremental changes.

After manually running Time Machine a few times per day - instead of hourly - I mounted the Time Machine backup file from the Time Capsule (where we backup to) and explored the backups via Finder (each Time Machine backup shows its own subfolder within the Time Machine sparsebundle with a date-time stamp in the folder name). As I drilldown into each backup and Show Package Contents for each of the Entourage Identities sparsebundle files within, each shows 830 band files each of 8 mb (totaling about 6gb), which is the size of my Identities file. This is repeated in each of the Time Machine backups at different times that I've checked.

I was expecting to see just a number of modified bands in each backup - not all the bands! What's the deal?



[ Reply to This | # ]
How should Time Machine be setup?
Authored by: cmcd on Nov 10, '09 02:26:03PM

Note: of the 830 8mb band files included in each backup, only a relatively small number show a recent modified date (i.e., same day as the backup) - many haven't been modified in several days. But, all 830 files are included in each of the backups, requiring lots of disk space.



[ Reply to This | # ]
How should Time Machine be setup?
Authored by: everkleer80 on Nov 11, '09 06:32:04AM

The way TM works is it creates hard links to any files that haven't changed since they were last backed up. If you're not familiar with hard links, they are basically file entries pointing to data on your hard drive. Technically I think every file on your drive is a hard link and once another hard link to a preexisting file has been created, they are identical and there is no way to tell them apart. So if, for example, you have an 8MB file on your drive and you create another hard link to that file, the new file will both A. look exactly like the original, B. contain the exact same data (if the data changes then both files will be changed), C. report a size of 8MB, but the data will only be on your drive once so no matter how many hard links there are to it, the total amount of disk space used by all of them combined will be 8MB. I don't know if this is a good explanation (if you need more info you can look it up - Wikipedia comes to mind) but basically the files you are seeing that do not have a 'recent modified date' were not acually backed up - they are just hard links that were created for the previously backed up files and they will always report the same size as their counterparts but are not actually taking up extra disk space because they are pointing to the same data.

I hope this makes sense!



[ Reply to This | # ]
How should Time Machine be setup?
Authored by: cmcd on Nov 12, '09 07:03:26AM

Thanks, everkleer80, I think I do understand. Based on further analysis, it appears like Time Machine (TM) isn't backing up as much as I originally thought, but I'm still not sure it's behaving as I would hope. I used Grand Perspective (great tool!) to analyze the TM sparsebundle backup file on the Time Capsule, which allowed me to filter out hard linked files. It showed that TM was not actually backing up the full 6gb Identities folders every time, but it was backing up about 500mb every time. The Database file is actually about 1.5gb and the 500mb corresponds exactly to the size of my Exchange server mailbox size, which would be one of the mailboxes included with the Database file.

Is it possible that all the sparsebundle bands associated with the Exchange server mailbox within the Database file change when a single mail message is sent or received? That's the only way I can explain why that much data would be backed up each time. I will continue to monitor the backup performance to try to figure this out. Thanks for your help.



[ Reply to This | # ]
How should Time Machine be setup?
Authored by: everkleer80 on Nov 12, '09 10:16:07AM

I've actually never used Entourage or anything so I'm not really sure what's going on here. You're right it sounds like every band in the sparsebundle associated with the mailbox is changing. If I had to guess, I'd say that maybe it's moving data around in the file to limit fragmentation (similar to defragmenting a hard drive.) That sounds pretty unlikely, but it would result in data moving to different bands in the sparsebundle and therefore all the associated bands would end up changed...

Anyway, GrandPerspective is a great program! I also found it when troubleshooting a different TM issue (my issue was space not being free'd up in my TM disk image when I deleted files from the backup - turned out the 'deleted' data was going into a hidden temp file on the disk.) I should have mentioned before, though, that TimeTracker is a cool little program that will show you how much data and which files were backed up in each backup. There are some better similar programs out there but I don't think the others are quite as free as TimeTracker.



[ Reply to This | # ]
How should Time Machine be setup?
Authored by: everkleer80 on Nov 12, '09 10:39:25AM

Just a clarification, (I thought we were supposed to be able to edit our posts now?) when I said:

If I had to guess, I'd say that maybe it's moving data around in the file to limit fragmentation (similar to defragmenting a hard drive.)
I meant maybe Entourage is moving data around in the mailbox file which would result in all associated bands being changed.

For example, if I have a simple text file ("abcdefghijklmnop") split into bands or 5 characters each (band1="abcde", band2="fghij", etc) and I insert the letter A at the beginning of the file, then every band would be change (now band1="Aabcd", band2="efghi", etc.) Whereas if I just added the letter to the end of the file then only the last band would be changed.



[ Reply to This | # ]
How should Time Machine be setup?
Authored by: cmcd on Nov 13, '09 06:45:35AM

I've done additional analysis, with both TimeTracker and Finder/Path Finder. It does appear that Time Machine is just backing up the bands that have been modified since the last TM backup - as expected. The surprise for me is how much space the bands are taking up - again, I'm seeing about 400-500mb per daily backup. I suspect the issue is the 8mb band size. For example, for the few backups I examined, the number of bands was 50-70 - multiply by 8mb per band and I get my number. And 50-70 modified bands may correspond to about the number of emails I receive/send in a day (though it seems a bit high).

So, maybe the answer is to convert the sparsebundle file to use either smaller - or much larger - bands. Since email messages are usually small, maybe use 1mb bands instead of 8mb. Then, 70 bands will only be 70mb (except when I receive emails with large attachments). I think hdiutil can convert sparsebundle band size. Any thoughts? My problem is I'm dealing with 2 sparsebundles: the relatively small Microsoft Identities file, and the huge Time Machine backup file on the Time Capsule.



[ Reply to This | # ]
How should Time Machine be setup?
Authored by: everkleer80 on Nov 13, '09 09:52:18AM

I think you're right on with changing the band size. But it sounds like the messages may be spread throughout the file rather than being grouped together (not sure about this) which means that multiple messages may or may not go into one band depending on how close together they are in the mailbox file. So if you increase the band size to, say, 16 MB, and have 10 new messages that are all < 1MB and are grouped together, then they may all go into the same band and you will only have to backup 16MB, but then again if the messages are spread apart in the file and each one goes into a different band, then you will be backing up 160MB. So I think smaller is definately the way to go here so no matter where in the data your messages are, you will be backing up closer to the amount of data that actually changed (maybe make the bands 1 or 2MB if you can?)

I'm not sure if you're saying there is any sort of problem with your TM sparsebundle or not or if your just commenting on the fact that we're dealing with two different sparsebundles, but anyway I don't think you should do anything to the TM bundle. I'm not sure why exacly TM backs up directly to the drive if it's local, and to a sparsebundle if it's a network drive, but either way I'd leave that one alone and just let TM manage it.



[ Reply to This | # ]
How should Time Machine be setup?
Authored by: cmcd on Nov 16, '09 01:54:14PM
OK, now I'm using a sparsebundle file with 1MB bands instead of the default 8MB. It's soon after the change and I'm still monitoring, but the conversion appears to have the desired result - my backup is now in the 100MB range instead of 500MB range. Maybe 500KB bands would reduce the backup size further, I don't know. I used this command to convert the sparsebundle file:
hdiutil convert ~/Documents/Office_2008_backup.sparsebundle -format UDSB -tgtimagekey sparse-band-size=2048 -o ~/Documents/Office_2008_IdentitiesNEW.sparsebundle
where Office_2008_IdentitiesNEW.sparsebundle is the converted file with the 1MB bands. After the conversion, I created a new alias pointing to the converted file.
Edited on Nov 16, '09 01:55:08PM by cmcd


[ Reply to This | # ]
How should Time Machine be setup?
Authored by: cmcd on Nov 17, '09 01:26:15PM

According to the man page for hdiutil, 1MB bands are the smallest allowed.



[ Reply to This | # ]
How should Time Machine be setup?
Authored by: kcheon on Apr 13, '11 05:05:08AM

I just tried, you don't need to exclude the sparse bundle file in TM. It seems that TM is clever enough to look at the contents and only backup the changed files in the bundle.



[ Reply to This | # ]
How should Time Machine be setup?
Authored by: everkleer80 on Apr 14, '11 02:33:25PM
Actually it's not so much TM's cleverness as it is the cleverness of sparsebundles. Rather than being one large file like a sparseimage, a sparsebundle is actually a collection of relatively small files (one for each band of data) inside a folder that the OS displays as a file (aka a package.) When TM runs, all it sees is the sparsebundle folder and all the band files inside it; since it is only backing up files that have been modified, only the bands that have changed are backed up.

[ Reply to This | # ]