Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!


Click here to return to the 'How should Time Machine be setup?' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
How should Time Machine be setup?
Authored by: everkleer80 on Nov 11, '09 06:32:04AM

The way TM works is it creates hard links to any files that haven't changed since they were last backed up. If you're not familiar with hard links, they are basically file entries pointing to data on your hard drive. Technically I think every file on your drive is a hard link and once another hard link to a preexisting file has been created, they are identical and there is no way to tell them apart. So if, for example, you have an 8MB file on your drive and you create another hard link to that file, the new file will both A. look exactly like the original, B. contain the exact same data (if the data changes then both files will be changed), C. report a size of 8MB, but the data will only be on your drive once so no matter how many hard links there are to it, the total amount of disk space used by all of them combined will be 8MB. I don't know if this is a good explanation (if you need more info you can look it up - Wikipedia comes to mind) but basically the files you are seeing that do not have a 'recent modified date' were not acually backed up - they are just hard links that were created for the previously backed up files and they will always report the same size as their counterparts but are not actually taking up extra disk space because they are pointing to the same data.

I hope this makes sense!



[ Reply to This | # ]
How should Time Machine be setup?
Authored by: cmcd on Nov 12, '09 07:03:26AM

Thanks, everkleer80, I think I do understand. Based on further analysis, it appears like Time Machine (TM) isn't backing up as much as I originally thought, but I'm still not sure it's behaving as I would hope. I used Grand Perspective (great tool!) to analyze the TM sparsebundle backup file on the Time Capsule, which allowed me to filter out hard linked files. It showed that TM was not actually backing up the full 6gb Identities folders every time, but it was backing up about 500mb every time. The Database file is actually about 1.5gb and the 500mb corresponds exactly to the size of my Exchange server mailbox size, which would be one of the mailboxes included with the Database file.

Is it possible that all the sparsebundle bands associated with the Exchange server mailbox within the Database file change when a single mail message is sent or received? That's the only way I can explain why that much data would be backed up each time. I will continue to monitor the backup performance to try to figure this out. Thanks for your help.



[ Reply to This | # ]
How should Time Machine be setup?
Authored by: everkleer80 on Nov 12, '09 10:16:07AM

I've actually never used Entourage or anything so I'm not really sure what's going on here. You're right it sounds like every band in the sparsebundle associated with the mailbox is changing. If I had to guess, I'd say that maybe it's moving data around in the file to limit fragmentation (similar to defragmenting a hard drive.) That sounds pretty unlikely, but it would result in data moving to different bands in the sparsebundle and therefore all the associated bands would end up changed...

Anyway, GrandPerspective is a great program! I also found it when troubleshooting a different TM issue (my issue was space not being free'd up in my TM disk image when I deleted files from the backup - turned out the 'deleted' data was going into a hidden temp file on the disk.) I should have mentioned before, though, that TimeTracker is a cool little program that will show you how much data and which files were backed up in each backup. There are some better similar programs out there but I don't think the others are quite as free as TimeTracker.



[ Reply to This | # ]
How should Time Machine be setup?
Authored by: everkleer80 on Nov 12, '09 10:39:25AM

Just a clarification, (I thought we were supposed to be able to edit our posts now?) when I said:

If I had to guess, I'd say that maybe it's moving data around in the file to limit fragmentation (similar to defragmenting a hard drive.)
I meant maybe Entourage is moving data around in the mailbox file which would result in all associated bands being changed.

For example, if I have a simple text file ("abcdefghijklmnop") split into bands or 5 characters each (band1="abcde", band2="fghij", etc) and I insert the letter A at the beginning of the file, then every band would be change (now band1="Aabcd", band2="efghi", etc.) Whereas if I just added the letter to the end of the file then only the last band would be changed.



[ Reply to This | # ]
How should Time Machine be setup?
Authored by: cmcd on Nov 13, '09 06:45:35AM

I've done additional analysis, with both TimeTracker and Finder/Path Finder. It does appear that Time Machine is just backing up the bands that have been modified since the last TM backup - as expected. The surprise for me is how much space the bands are taking up - again, I'm seeing about 400-500mb per daily backup. I suspect the issue is the 8mb band size. For example, for the few backups I examined, the number of bands was 50-70 - multiply by 8mb per band and I get my number. And 50-70 modified bands may correspond to about the number of emails I receive/send in a day (though it seems a bit high).

So, maybe the answer is to convert the sparsebundle file to use either smaller - or much larger - bands. Since email messages are usually small, maybe use 1mb bands instead of 8mb. Then, 70 bands will only be 70mb (except when I receive emails with large attachments). I think hdiutil can convert sparsebundle band size. Any thoughts? My problem is I'm dealing with 2 sparsebundles: the relatively small Microsoft Identities file, and the huge Time Machine backup file on the Time Capsule.



[ Reply to This | # ]
How should Time Machine be setup?
Authored by: everkleer80 on Nov 13, '09 09:52:18AM

I think you're right on with changing the band size. But it sounds like the messages may be spread throughout the file rather than being grouped together (not sure about this) which means that multiple messages may or may not go into one band depending on how close together they are in the mailbox file. So if you increase the band size to, say, 16 MB, and have 10 new messages that are all < 1MB and are grouped together, then they may all go into the same band and you will only have to backup 16MB, but then again if the messages are spread apart in the file and each one goes into a different band, then you will be backing up 160MB. So I think smaller is definately the way to go here so no matter where in the data your messages are, you will be backing up closer to the amount of data that actually changed (maybe make the bands 1 or 2MB if you can?)

I'm not sure if you're saying there is any sort of problem with your TM sparsebundle or not or if your just commenting on the fact that we're dealing with two different sparsebundles, but anyway I don't think you should do anything to the TM bundle. I'm not sure why exacly TM backs up directly to the drive if it's local, and to a sparsebundle if it's a network drive, but either way I'd leave that one alone and just let TM manage it.



[ Reply to This | # ]
How should Time Machine be setup?
Authored by: cmcd on Nov 16, '09 01:54:14PM
OK, now I'm using a sparsebundle file with 1MB bands instead of the default 8MB. It's soon after the change and I'm still monitoring, but the conversion appears to have the desired result - my backup is now in the 100MB range instead of 500MB range. Maybe 500KB bands would reduce the backup size further, I don't know. I used this command to convert the sparsebundle file:
hdiutil convert ~/Documents/Office_2008_backup.sparsebundle -format UDSB -tgtimagekey sparse-band-size=2048 -o ~/Documents/Office_2008_IdentitiesNEW.sparsebundle
where Office_2008_IdentitiesNEW.sparsebundle is the converted file with the 1MB bands. After the conversion, I created a new alias pointing to the converted file.
Edited on Nov 16, '09 01:55:08PM by cmcd


[ Reply to This | # ]
How should Time Machine be setup?
Authored by: cmcd on Nov 17, '09 01:26:15PM

According to the man page for hdiutil, 1MB bands are the smallest allowed.



[ Reply to This | # ]