Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

Preserve resource forks during backups via tar and ftp UNIX
One of the problems with backups of Mac OS X to a remote machines is the potential for lost resource forks, particularly if the other machine is not running Mac OS X. There are ways around this. For one, the FTP program included with OS X allows for piping from external commands, and there are versions of tar that support resource forks (e.g. Tiger's tar and Xtar). Therefore, a tar archive (that preserves the resource forks) can be created on the fly during an ftp backup process.

Note that the machine you are backing up to must be running an FTP server and have enough free space. OS X and most Linux distributions include an FTP server (it may need to be enabled), and there are a number of free ftp servers available for Windows.

To back up your home directory, you would issue the following commands from a command prompt:
 $ cd ~
 $ cd ..
 $ ftp ftp_server_name
You will then be prompted for the username and password required to connect to the FTP server. Once you have logged in, change to the relevant FTP directory using the cd command, and then at the FTP prompt, issue the following commands:
 ftp> bin
 ftp> put "| tar cvpf - username" username_backup.tar
Replace username with your actual username!

This will place your user directory into a tar file called username_backup.tar, which will be uploaded onto the FTP server. If the machine you are connecting from is running Tiger, then your permissions and resource forks will be preserved within the tar file, hence it does not matter what operating system the FTP server is running.

If you are running an older version of Mac OS X, then you should download and install Xtar and use this in place of the tar program that's included with OS X. This is because the standard tar program included with older versions of OS X did not support resource forks. So you would change the above ftp put command to:
 ftp> put "| xtar cvpf - username" username_backup.tar
To restore your backup, you would issue the following commands from the FTP prompt:
 ftp> bin
 ftp> get username_backup.tar "| tar xvpf -"
Or if you were using older versions of OS X and had Xtar installed:
 ftp> get username_backup.tar "| xtar xvpf -"
A couple of important notes. Firstly, the version of tar you used to backup must also be used for the restore. This is because the way resource forks are stored within the tar file is not fully compatible between Tiger's tar and Xtar. Secondly, you could also use this technique to back up the whole Users directory, and/or the Applications and Library directories as well. If you do this, you will need run the tar command as root to ensure that you have access to all the appropriate subdirectories, and so that permissions are correctly preserved during creation and extraction of the archive. Possibly the simplest way to do this would be to start the ftp command as root via sudo. If you do so, tar will therefore also be running as the root user, since tar is invoked by the ftp program. You will also need to change to a higher directory and alter the tar command appropriately. e.g.:
 $ cd /
 $ sudo ftp ftpserver
You will firstly be prompted for your local password to allow sudo to temporarily switch you to the root user. You will then be prompted by the ftp program for your FTP username and password. Once you have logged in, change to the relevant FTP directory using the cd command, and then at the FTP prompt, issue the following:
 ftp> bin
 ftp> put "| tar cvpf - Users Applications Library" full_backup.tar 
Or if you were using older versions of Mac OS X:
 ftp> bin
 ftp> put "| xtar cvpf - Users Applications Library" full_backup.tar 
The extraction would be as follows:

 ftp> get full_backup.tar "| tar xvpf -"
Or if you were using older versions of Mac OS X:

 ftp> get full_backup.tar "| xtar xvpf -"
Again you would need to make sure the tartt> command ran as root user to ensure a successful restore.

This method could alternatively make use of pax or hfspax in place of tar. However, as I am less familiar with the command line options of pax, I cannot advise of the changes you would need to make to get this to work. You could also use this technique to backup from Linux (or even Windows) onto an OS X machine, assuming you already had appropriate versions of tar and ftp installed on them. Whilst Linux and Windows don't have resource forks, it could be useful to ensure permissions, filenames and other features don't get altered by the FTP process.
    •    
  • Currently 3.17 / 5
  You rated: 4 / 5 (6 votes cast)
 
[18,177 views]  

Preserve resource forks during backups via tar and ftp | 19 comments | Create New Account
Click here to return to the 'Preserve resource forks during backups via tar and ftp' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
Preserve resource forks during backups via tar and ftp
Authored by: jwbales on Aug 10, '05 11:14:36AM

You can (should) also use compression to save time/bandwidth/diskspace...

Old...
ftp> put "| tar cvpf - username" username_backup.tar
New...
ftp> put "| tar zcvpf - username" username_backup.tar


[ Reply to This | # ]
Preserve resource forks during backups via tar and ftp
Authored by: jwbales on Aug 10, '05 11:17:14AM

Sorry forgot the .gz extention....

New...
ftp> put "| tar zcvpf - username" username_backup.tar.gz


[ Reply to This | # ]
well...
Authored by: nick on Aug 10, '05 11:41:33AM

if you really want to save bandwidth, you shouldn't backup tars via ftp but use rsync or unison. rsyncX and unison both support resource forks.



[ Reply to This | # ]
But what about backups to non Mac servers
Authored by: ruaric on Aug 10, '05 06:16:45PM

The point of using a tar file is to save the resource fork within it even if you backup to a non MacOS X machine (i.e. even if you backup to Linux or Windows which don't support resource forks they are still saved within the tar).



[ Reply to This | # ]
But what about backups to non Mac servers
Authored by: peterneillewis on Aug 11, '05 03:54:43AM

I think you will find that rsync under Tiger will backup resources to a non-Mac, it saves the extended data (resources and meta data) to ._ files, probably in exactly the same way that they are stored in the tar file.

It's just a shame that rsync under Tiger is so broken that it cannot backup a decent sized home directory without crashing or locking up (the crash took about five minutes to find in the source code (hint flist_expand), but the lockup I haven't tracked down yet). Perhaps 10.4.3/4/5/6?



[ Reply to This | # ]
Preserve resource forks during backups via tar and ftp
Authored by: rumirocks on Aug 10, '05 12:49:42PM

Where do you find the ftp server?

(Naturally, Spotlight couldn't find it since it cant find anything these days.)



[ Reply to This | # ]
Preserve resource forks during backups via tar and ftp
Authored by: matx on Aug 10, '05 01:37:47PM

The FTP server in OSX client is controlled via System Preferences: Sharing

---
Mat X -- VFX Mac Tech



[ Reply to This | # ]
Preserve resource forks during backups via tar and ftp
Authored by: AquaLight on Aug 10, '05 05:13:05PM

Does this method generate a backup file locally the size of your home directory or does the pipe create the tar file on the fly and send it to the server?



[ Reply to This | # ]
Preserve resource forks during backups via tar and ftp
Authored by: ruaric on Aug 10, '05 06:13:55PM

The pipe creates the tar file on the fly and sends it to the server.



[ Reply to This | # ]
Preserve resource forks during backups via tar and ftp
Authored by: codeman38 on Aug 10, '05 07:33:20PM

Careful on this one if you're running Fink; by default, Fink *prepends* rather than appends itself to the system path, thus making its version of tar take precedence over the one built into OS X. This can be changed in several ways; I've edited /sw/bin/init.sh to change the 'prepend' statements to 'append's.



[ Reply to This | # ]
Which Tar
Authored by: jasont on Aug 10, '05 09:48:09PM

Thanks for that, could you post what you changed to use the default tools. For most people calling tar w/ /usr/bin/tar would be the workaround right?



[ Reply to This | # ]
Preserve resource forks during backups via tar and ftp
Authored by: loren_ryter on Aug 11, '05 10:33:37AM

why does it matter which tar?

i'm also getting /sw/bin/tar since i've got fink installed.



[ Reply to This | # ]
Preserve resource forks during backups via tar and ftp
Authored by: ruaric on Aug 12, '05 03:46:19AM

Because the tar included with fink does not save or restore resource forks.



[ Reply to This | # ]
Preserve resource forks during backups via tar and ftp
Authored by: rygardner on Aug 11, '05 03:21:15AM

It seems like maybe this could be automated with a nice little AppleScript...

If so, has anyone who knows more about this than I done such a thing?



[ Reply to This | # ]
could you make a bash or tcsh alias for the put command?
Authored by: loren_ryter on Aug 11, '05 10:38:52AM

Here's the modified command as given in one of the replies:

put "| tar zcvpf - username" username_backup.tar.gz

Instead of having to cd to the parent of $home couldn't you put the homepath in the command ?

then couldn't you make a bash or tcsh alias out of it?

then you would just ftp to a backup server and type the alias.



[ Reply to This | # ]
Preserve resource forks during backups via tar and ftp
Authored by: John Kiniston on Aug 11, '05 02:22:44PM

OS X Server has a nice feature in its FTP server that allows a user to download disk images or tar files created by the server of directories.

Say I connect to a OS X sever and wanted to download a copy of my Public folder

ftp> get Public.dmg
local: Public.dmg remote: Public.dmg
227 Entering Passive Mode (10,1,1,80,42,99)
150 Opening BINARY mode data connection for /usr/bin/mkdmg - -s Public.
14305 11.04 KB/s
226 Transfer complete.
14305 bytes received in 00:01 (11.02 KB/s)

There's no Public.dmg on the server just the folder Public

You can download Folders as TAR files as well the same way

ftp> get Public.tar
local: Public.tar remote: Public.tar
227 Entering Passive Mode (10,1,132,80,42,136)
150 Opening BINARY mode data connection for /usr/bin/tar -cf - -- Public.
10240 57.06 KB/s
226 Transfer complete.
10240 bytes received in 00:00 (56.41 KB/s)
ftp> get Public.tar.gz
local: Public.tar.gz remote: Public.tar.gz
227 Entering Passive Mode (10,1,132,80,42,137)
150 Opening BINARY mode data connection for /usr/bin/tar -cf - -- Public | /usr/bin/gzip -c.
71680 15.25 KB/s
226 Transfer complete.
71680 bytes received in 00:04 (15.25 KB/s)

If you want to compress just a file that also works

ftp> get config.cache.gz
local: config.cache.gz remote: config.cache.gz
227 Entering Passive Mode (10,1,132,80,42,151)
150 Opening BINARY mode data connection for /usr/bin/gzip -9 -c config.cache.
704 757.15 KB/s
226 Transfer complete.
704 bytes received in 00:00 (254.62 KB/s)



[ Reply to This | # ]
Preserve resource forks during backups via tar and ftp
Authored by: ruaric on Aug 12, '05 03:49:38AM
I haven't tried this yet but if it works, that is very cool! Thanks for the tip!!

[ Reply to This | # ]
Preserve resource forks during backups via tar and ftp
Authored by: macshome on Aug 12, '05 09:19:42AM

Yeah, this is a long time feature that often gets missed. The dmgs are fairly new, but it's always been able to binhex stuff. In fact if you have the conversions turned on and browse the ftp directories it will show forked files with .bin on the names, just so you don't forget.

---
http://www.afp548.com
Breaking my server to save yours.



[ Reply to This | # ]
Preserve resource forks during backups via tar and ftp
Authored by: Spartacus on Oct 12, '05 01:12:14PM

Does anyone know how to *disable* support for resources in Tiger's tar?



[ Reply to This | # ]