Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

10.5: Use multiple Time Machine disks for redundancy Apps
Time machine, at least as of 10.5.6, will gladly maintain multiple Time Machine disks. This way, you can periodically rotate your Time Machine backup disks to keep one safely off line and off site. The trick is that there is no trick. I say this because there are some dangerous methods posted on the internet involving complex setups like copying the hidden disk ID number, that while appearing to work, actually slowly corrupt your backups.

All you need to do is attach the disk and tell Time Machine to use it. When you are ready to swap drives, attach the other and tell Time Machine to use it instead. The very first time you do this, there is (of course) a big time consuming backup. After that, the swap is pretty fast. A bit longer than a typical backup session (but a lot shorter than say an rsync by comparison).

Time Machine correctly finds the files that were changed since the last time that disk was mounted and backs these up, making a perfectly synced new backup in the older disk's time lineage. And then it proceedes normally. If you are alternating disks weekly, then of course each disk is missing the disk state from the previous week. It still has all the files, it just can't restore to an epoch during which it was not connected -- that time period state is on the other disk.

Other than that, this approach seems to be significantly better than the alternative of keeping an rsynced copy of the Time Machine disk. That's an ugly process, due to all the hard links, that is fraught with many small perils.
    •    
  • Currently 3.47 / 5
  You rated: 1 / 5 (17 votes cast)
 
[51,478 views]  

10.5: Use multiple Time Machine disks for redundancy | 29 comments | Create New Account
Click here to return to the '10.5: Use multiple Time Machine disks for redundancy' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
10.5: Use multiple Time Machine disks for redundancy
Authored by: Ron Crocker on Jun 04, '09 08:22:04AM

I just happen to have a spare 750GB drive (formatted capacity 698.32GB) laying about with which I can give this a spin.

I understand that the "weekly" in the hint is a suggestion, but it seems like perhaps monthly would be a better approach. This matches better Time Machine's structure of "hourly for a day, daily for a month, weekly for ever."

Using a monthly swap gives 2 months of dailies. This could be extended further to N months using N disks, of course, but at some point it becomes ridiculous.

Any thoughts/comments on that approach?

Ron



[ Reply to This | # ]
10.5: Use multiple Time Machine disks for redundancy
Authored by: SOX on Jun 04, '09 08:47:52AM

sounds like an improvement. The trade off is risk aversity to catastrophe and how valuable a months work is. I happen to live in a lighting heavy area so keeping one disk unplugged is appealing.

important tip: when you innaugurate the new disk, don't try to get clever and copy the old disk to speed thing up. let time machine do it itself. it's imprtant that time machine know you have two disks which is will mark with different disk ID numbers so it can tell. if you make carbon copies you will fool time machine and it will not make faithful syncs. that is, just let it do it it's way , don't try to out smart it.



[ Reply to This | # ]
10.5: Use multiple Time Machine disks for redundancy (or in conjunction with Time Capsule)
Authored by: seahermit on Jun 04, '09 08:51:02AM

I do this actually - but for a slightly different reason... I have a Time Capsule at home, but when I'm away (for work or holiday) I use a little USB drive and point Time Machine to that - works really well.

One thing I have noticed, is that if it's been a few weeks between using the other disk (last time this was after 5 weeks) the backup took a while as it had to do a deep node traversal, which meant Time Machine had lost track (for that drive) of which files had changed. This has happened a few times now after swapping, so I'd probably suggest swapping around the week frequency as suggested.

Also, that means that if one of your disks does blow up, you'll only lose a week at most! :)



[ Reply to This | # ]
10.5: Use multiple Time Machine disks for redundancy
Authored by: pecosbill on Jun 04, '09 10:29:05AM

Here's another compelling reason to do weekly swaps: if you do have your active backup drive fail, you lose only a week of data. Otherwise, you'd lose a full month!

---
Pecos Bill



[ Reply to This | # ]
10.5: Use multiple Time Machine disks for redundancy
Authored by: palahala on Jun 04, '09 09:00:42AM
setups like copying the hidden disk ID number, that while appearing to work, actually slowly corrupt your backups

Can you extend a bit on this corruption issue?

(So far I have encountered no issues with using the same Time Machine cookie on two disks, which I started using after TM had already built-up some history. So I use two identical copies as a base, to ensure the old history is available on both disks.)



[ Reply to This | # ]
10.5: Use multiple Time Machine disks for redundancy
Authored by: SOX on Jun 04, '09 09:43:55AM

Okay here's what can happen if you copy the cookies.

I don't know exactly how time machine does it's thing. But nominally it uses hooks in the OS to notify it when things on the main disk change so it has a list of pending backups. It does not do it the way rsync does it by labriously checking concurrency of the siles system and comaring every modification date.

if you have two disks that report the same cookie time machine may beleive they are the same disk so it won't realize that a file that was changed last week then backed up to disk B, is not actually on disk A and needs to be backed up. That file is not on the pending backup list anymore.

so here's a worked example: imagine you have two files, a preferences and a database for an application like say iphoto that have stay synched (entries in one need corresponding entries in the other).

week 0 (disk 1 connected): A and B are changed.
Week 1 (Disk 2 connected): A is changed but not B
Week 2 (Disk 1 connected): B is changed but not A
Week 3 (disk 2 is connected): On tuesday, A is changed.

On Thursday you decide to revert the home directory back to it's state on Wednesday. What happens?

Well you get the A from tuesday week 3 and you get the B from week 0.

it's subtle because if you were too look in the backup directory for wednesday you would see both files, A and B present. You just would not realize they were not mutuall consistent. One was a B written 3 weeks earlier with an A written the previous day.

over time, there would be no day in the backups that contained all the files that were present together on that day!

However when time machine can tell you have switched disks because the cookies changed then it knows it needs to discard the invalid pending list and do an explicit check of what files changed since the last bakcup by comparing them between the disks. hence the sync that occurs makes sure the backup contians mutuall consistent files.

It could be that periodically timemachine does this check anyhow, so some of the time you may get away with the copied cookie method. it's just not assured.

the bottom line is: why bother when it's simpler not to use cookie copies. just let time machine do it for you as it wants to.





[ Reply to This | # ]
10.5: Use multiple Time Machine disks for redundancy
Authored by: palahala on Jun 04, '09 11:37:21AM
I can assure that, when I swap drives, Time Machine always needs to back up a lot more than it would when not swapping the drives. Using for example TimeTracker I can also validate that old files are surely copied...

[ Reply to This | # ]
10.5: Use multiple Time Machine disks for redundancy
Authored by: palahala on Jun 04, '09 11:43:20AM
By the way, Ars Technica has has some nice technical insight about Time Machine and how it relies on fsevents.

[ Reply to This | # ]
10.5: Use multiple Time Machine disks for redundancy
Authored by: mantrid on Jun 04, '09 12:25:51PM

That Ars article is exactly what I was thinking of. I have strong misgivings that the hint submitter is just guessing and stating their guesses as alarmist facts.



[ Reply to This | # ]
10.5: Use multiple Time Machine disks for redundancy
Authored by: osxpounder on Jun 05, '09 11:42:17AM

I look at the hint and our comments as information, not alarmist. If we have facts or experience to offer, or theories to share that others might want to test, that can be helpful.

It's not as helpful merely to vaguely devalue the hint that someonetook time and trouble to share the hint with us, especially when all you've got are 'misgivings', not facts. Seems to me you're doing the same thing you just criticized.



[ Reply to This | # ]
10.5: Use multiple Time Machine disks for redundancy
Authored by: mantrid on Jun 05, '09 01:30:08PM

The difference is that my comment is a comment and explicitly expressed as my personal misgivings. Their's is stated as a fact, in a submitted hint, as justification to use said hint. To me, that is a huge difference. What some idiot says in the comments doesn't carry nearly as much weight as something in a published hint on macosxhints.com.

If you had read the Ars article about TimeMachine and file system events beforehand, and then come across this hint and read that statement, you also might have thought "woah, where did that come from"? Unsubstantiated claims that something is "dangerous" and will "actually" (the word used) "actually slowly corrupt your backups" is alarmist. Consider how the statement would be interpreted if read without prior exposure to the Ars article, or other sources. No corroboration has appeared from the poster in subsequent posts, whereas palahala has included several links rebutting the claim. No instructions to say "do this to demonstrate corruption". No links to documentation. Not even a link to something as anecdotal as a web forum where someone reports corruption. Now after the fact, we are seeing comments like "I don't know exactly how time machine does it's thing" or "I don't think either of us knows" coming from the submitter, and at least one instance of an argument based on a misinterpretion of a linked article. So what exactly were the original claims based on?

I'm still hoping to see the proof, but if an admission is coming that the statement was based on speculation rather than fact, the sooner the better.

By the way, I'm not vaguely devaluing the hint at all. I think it's a fine hint that I'm sure many will find useful. That doesn't excuse the inclusion of (still unsubstantiated) statements of what is looking more and more like speculation as fact. Speaking generally, the fact that hints on macosxhints go through a screening process before publication has helped to keep the reliability of the information that can be found here at a high level, something that would be threatened if unsubstantiated claims and misinformation are complacently allowed to persist.



[ Reply to This | # ]
10.5: Use multiple Time Machine disks for redundancy
Authored by: osxpounder on Jun 08, '09 09:25:04AM

Good points. Fair enough.



[ Reply to This | # ]
10.5: Use multiple Time Machine disks for redundancy
Authored by: palahala on Jun 04, '09 01:22:55PM

Some more notes:

Apparently, Time Machine stores the last known event ID on the backup itself, in an extended attribute com.apple.backupd.SnapshotVolumeLastFSEventID.

So: I doubt the suggested corruption could occur.

And a nice article for developers: Monitoring File Changes with the File System Events API, using some lastEventId to find all files that have changed.



[ Reply to This | # ]
10.5: Use multiple Time Machine disks for redundancy
Authored by: SOX on Jun 04, '09 04:43:42PM

You could answer this in part by doing this:

sudo bzgrep -i backupd /private/var/log/system.log.*.bz2 | grep travers

According to the article you linked to if you don't get a "Node requires traversal" message every time you swap drives then it may not be detecting the out-of-sync condition.

as a further check you could also try

sudo bzgrep -i backupd /private/var/log/system.log.*.bz2 | grep from

and see if there's a huge number of megabytes copy every time you swap drives.



[ Reply to This | # ]
10.5: Use multiple Time Machine disks for redundancy
Authored by: palahala on Jun 05, '09 01:57:28AM
if you don't get a "Node requires traversal" message every time you swap drives then it may not be detecting the out-of-sync condition

My point is: Time Machine gets the com.apple.backupd.SnapshotVolumeLastFSEventID attribute from the backup disk. After swapping disks, this event ID will be lower than the event ID used by the previous backup. This is exactly the same when using two disks like in your hint. Next, TM can simply ask fseventd for the changes since that (lower) last known event ID.

There is no "out-of-sync condition" (other than the fseventsd database having been recreated for unrelated reasons, which requires a deep traversal for both backup disks once plugged in at some later time). One should NOT expect any "Node requires deep traversal" when swapping disks, not when using your hint, nor when using cloned disks.

(And yes, like I wrote: there is a huge number of megabytes copy every time I swap drives, easily noted in the logs with Time Machine Buddy, or by looking a the files that have been copied using TimeTracker.)

[ Reply to This | # ]

10.5: Use multiple Time Machine disks for redundancy
Authored by: SOX on Jun 05, '09 08:51:07AM

Perhaps I'm mistaken but my understanding of this is that in order to recover from a condition where the fsevents log has lost track of what need to be updated-- which will be the case here-- then a deep traversal is required to compare the current state of the main disk to the last backup on the old disk. That's in fact what the page you lined to says.

Thus when you swap in a new disk two things have to happen. First something has to trigger the detection of the incomplete fsevents log, and second a deep traversal has to occur.

When you manually repoint the time machine to a new disk it knows for sure it has to recatalog the disk. But when you simply swap disks that masquererade as each other using the cookie trick then you must reply on some secondary check to trigger the detection that the fsevents lof is out of sync with the disk. You are speculating that it can do this by looking at the lastudate events UUID and seeing if this is still in the fsevents log somewhere. It's possible this is true, I don't think either of us knows.

So what I was asking you to test on your system was: if is true then you should be seeing a node-travesral required message or at least some other message about the detection of this condition.

I'd also be curious to know what you think the purpose of the cookie is and what the negative consequences of removing it are.



[ Reply to This | # ]
10.5: Use multiple Time Machine disks for redundancy
Authored by: palahala on Jun 05, '09 09:56:44AM
a condition where the fsevents log has lost track of what need to be updated-- which will be the case here--

No, that is not the case here... Time Machine knows very well what data is on each backup disk. It then asks fseventd or some related API what has changed since.

When you manually repoint the time machine to a new disk it knows for sure it has to recatalog the disk.

If you're saying that you see the "Node requires deep traversal" message each time you manually assign another disk, then something is wrong on your Mac.

You are speculating that it can do this by looking at the lastudate events UUID and seeing if this is still in the fsevents log somewhere. It's possible this is true, I don't think either of us knows.

That's not speculating, that's exactly what is described in each article I mentioned earlier (though it's not the log's UUID but the FSEventsID counter). Time Machine is not keeping track of any changes. It doesn't have to, as long as it knows the last ID it used when writing to some backup.

So, I don't see anything confirming your "actually slowly corrupt your backups".



[ Reply to This | # ]
Great
Authored by: hamarkus on Jun 04, '09 11:53:18AM

For those who take their laptop to work, this should give two very fine-grained backups one of which is by definition always offsite (and one would have the hourly backups both at home and at work).



[ Reply to This | # ]
It would be nice if it was automatic
Authored by: polarbreeze on Nov 10, '11 07:35:29PM

The strategy, with a laptop, of having two TM backups (one at home, one at work) is pretty much perfect and it's worked very well for me for at least two years. What would be EVEN NICER would be if the MacBook were smart enough to see when it changes TM disks and then switch targets automatically. Currently the only way to do it seems to be to change the target disk location manually in TM Preferences. Unless someone can suggest a better way?!

PS, I noticed that the MacBook already does something similar with printers, switching the default printer automatically depending on location.



[ Reply to This | # ]
10.5: Use multiple Time Machine disks for redundancy
Authored by: Ilo on Jun 04, '09 11:54:51AM

This has always worked since 10.5.0 I have a disk at home and a disk at work, and carry my laptop back and forth as I commute, always plugging it into the local disk when I arrive. And, yes, I have tested the complete system restore function from these disks. This means automatic "off site" backup never older than 12 hours (but a heavier briefcase).



[ Reply to This | # ]
10.5: Use multiple Time Machine disks for redundancy
Authored by: lovemymacs on Jun 04, '09 12:51:46PM

I would like to add that time machine backups are incredibly prone to corruption and when they become corrupted they can become completely unusable extremely quickly. Like in moments. If the computer gets overloaded or crashes in the middle of a wireless backup just one time that can render months of backups worthless. I do use multiple time machine backups now as just a part of my backup system. When my previous backups have become damaged the geniuses have cheerfully told me to reformat and start again. I've had two separate time capsules completely replaced. And on Apple's website the list of people who describe my exact problems is very, very long. Time capsule would be wonderful if it would only work consistently. That said - it has saved me on more than one occasion and those occasions have been well worth all of the trouble. But every night I run a superduper backup and I keep six days of those backups going in rotation. I also use emailbackup pro as well as creating copies of any particularly important data like my schedule and keeping them on multiple volumes and backup disks. I lost about $10,000 of information two years ago when my palm somehow switched to "purge" and I find any amount of trouble in backing up to be a small price. Superduper is a staggeringly useful program. Time machine seriously scares me as any kind of dependable solution. When it is there and ready, though, when it has worked, it is wonderful.



[ Reply to This | # ]
10.5: Use multiple Time Machine disks for redundancy
Authored by: hamarkus on Jun 05, '09 06:25:22AM

I too had one TC replaced by the geniuses. But as long as the TC still mounts, trying a fsck from the command line can fix a lot of errors (but it might easily take more than 24 hours to do this).



[ Reply to This | # ]
10.5: Use multiple Time Machine disks for redundancy
Authored by: JackSloth on Jun 04, '09 01:11:19PM

Ideally, one should do daily rotations. Keep one off site, perhaps at work. This way you only lose a day's worth.



[ Reply to This | # ]
10.5: Use multiple Time Machine disks for redundancy
Authored by: jpl on Jun 04, '09 04:03:30PM

Hi,

I think it would be better/easier to make copies of all your backup disks so that they are all identical.

No need to think where to look for this 3 weeks old file (sigh, it is on the second offsite backup disk, I'll have to wait until Monday to get it).

Now, how do you copy those directory hard links from the finder? Or the command line?



[ Reply to This | # ]
10.5: Use multiple Time Machine disks for redundancy
Authored by: mazzaroth on Jun 04, '09 04:08:22PM

This is what I've been doing for over 6 months now.

I have 2 Disks of 1 Tb. One is offsite, the other one is used as time machine disk. Every week, I take the connected drive, bring it to my off site, swap the drives and bring back the other one, connect it, switch time machine to it and voilą.

Both my laptop and my desktop are backed up on the same drive.

Once boostraped, I never have all my data at the same location and the worst that can happen is that I lose 1 week of data. Data is worth way more than the computers - the computers, I can buy back. Data, I can't.

Works perfectly.



[ Reply to This | # ]
10.5: Use multiple Time Machine disks for redundancy
Authored by: pbeyersdorf on Jun 05, '09 05:49:06PM

I have two time machine disks - one connected at all times and one stored off site. Unlike this hint though, I don't rotate them. Once a month I bring home the off-site disk, connect it to do a backup and then take it back to work. This way the local disk stays up-to-date with hourly backups and I don't have to figure out which disk last weeks file is on if I need to restore. Of course the remote disk only has a monthly archive but I think this is a more useful solution than true rotation.



[ Reply to This | # ]
10.5: Use multiple Time Machine disks for redundancy
Authored by: hamarkus on Jun 13, '09 10:32:43AM

This does not seem to work with a Time Capsule, ie, unplugging a TM drive and switching on Wifi to allow access to the TC (or vice versa, switching off Wifi and plugging in the drive) and initiating a new backup does not automatically switch between the TM target disks, I have to manually select the TC or the drive in question in the TM preferences each time I switch between disks.



[ Reply to This | # ]
10.5: Use multiple Time Machine disks for redundancy
Authored by: mgb99 on Feb 03, '10 02:42:53PM

Has anyone attempted to use Automator or Applescript to change drives in Time Machine Preferences on a daily basis?



[ Reply to This | # ]
10.5: Use multiple Time Machine disks for redundancy
Authored by: GVAN on Apr 17, '11 09:44:28PM

An automator or script approach is what I am looking for. So that each day as I go back and forth to work it will detect the network change, and change the backup drive -- so that it automatically backs up to the disk at work when I am at work -- and automatically backs up to the disk at home when I am at home. without me having to manually switch it in time machine preferences each time.

Best if it does not require a reboot. And just notices the network change on wake from sleep or network plug in.

(I get hundreds of good email a day, write proposals etc and could not recover with a disk that was a week old)

Thanks



[ Reply to This | # ]