Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

Use psync to create incremental backups UNIX
I just found out about psync, which does exactly what I was looking for. I had been wanting to set up a cronjob to automatically backup my hard drive, but I didn't want a compressed backup. I wanted all resource forks kept intact as well as all permissions. To top that off, I wanted only files that had been modified since the last backup to be copied to my FW hard drive.

Read the rest of the article to see how to set up psync...

First off, you need to open up the Terminal. Once it is open, you need to open up the perl CPAN shell:
 % sudo perl -MCPAN -e shell
If this is the first time you've run the CPAN shell, you'll need to configure it (either auto or manual). Once that's done, at the prompt (cpan>) that shows up, type this in:
 install MacOSX::File
After the MacOSX File module has been downloaded and installed, type "quit" to exit the CPAN shell.

Okay, are you ready for this? Type:
 % sudo psync / /Volumes/Your_FW_Hard_Drive
Press return, and type in your admin password. You're done. For more info, see man psync.

P.S. Anyone interested in helping to write a Cocoa wrapper for this cool utility?

[Editor's note: I followed the instructions and then tested psync by backing up my Documents folder to another drive, and it worked as expected -- permissions and resource forks intact. Test it before relying on it, but it looks to be quite useful!]
    •    
  • Currently 1.33 / 5
  You rated: 2 / 5 (3 votes cast)
 
[29,826 views]  

Use psync to create incremental backups | 43 comments | Create New Account
Click here to return to the 'Use psync to create incremental backups' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
Need more on CPAN
Authored by: Yaso on Jul 11, '02 09:43:13AM

I need more info on CPAN. How do I install? Configure? Do I need the developer CD?

Thanks



[ Reply to This | # ]
re: Need more on CPAN
Authored by: dm2243 on Jul 11, '02 01:58:52PM

go to
http://www.cpan.org/misc/cpan-faq.html
http://developer.apple.com/internet/macosx/perl.html



[ Reply to This | # ]
Need more on CPAN
Authored by: David on Jul 11, '02 07:57:44PM

No, you don't need the Developer Tools. You need Perl installed (which it is) and you need to use the Terminal.

Try following my instructions and it should work fine. Post back if you have problems.

David



[ Reply to This | # ]
Need more on CPAN
Authored by: David on Jul 11, '02 07:59:43PM

Whoops, you are right. You do need the Dev Tools to compile the source. Use the binary package below (I haven't tried it) and it should work fine.

David



[ Reply to This | # ]
so long retrospect?
Authored by: dm2243 on Jul 11, '02 02:20:42PM

great hint. some comments:

1. especially useful is the -d option, which will delete files on the target if they're not found on the source. retrospect express' "duplicate" feature lacks this ability.

2. takes less time and considerably less cpu than retrospect

3. if this is the first time you've run the CPAN shell, you'll probably discover (as I did) that you'll need to update CPAN (which can be done within the CPAN shell). CPAN will also suggest that you install a few other things...

4. I haven't yet tried this myself, but i wonder if it can create a bootable FW clone (when conjoined with the bless command).



[ Reply to This | # ]
so long retrospect?
Authored by: jmontana66 on Jul 11, '02 04:22:18PM

I tired using psync a long time ago. It had extreme difficulty writing to network volumes. SMB-mounted volumes in particular. Retrospect does this flawlessly. If all you want to do is backup to another local volume or partition, psync does the job. It won't write to CD-R or network volumes though.



[ Reply to This | # ]
so long retrospect?
Authored by: David on Jul 11, '02 08:05:49PM

You won't need to run bless seperately (although psync may use it, I'm not sure).

Psync will automatically create a bootable drive (at least, it does on my FW drive if I backup the entire HD).

David



[ Reply to This | # ]
psync is excellent!
Authored by: thatch on Jul 11, '02 08:28:28PM

I've been using psync for about a year now and it works flawlessly. I backup to an external firewire hard drive, (oxford 911 chipset with an IBM ATA inside), and it is bootable without any blessing commands. I also use it to clone to spare local partitions for testing purposes. If something goes awry with the testing, it's easy to restore using psync and it's fast too. It also does incremental backups and works in the background. The only caveat is that it won't do backups to networked drives.



[ Reply to This | # ]
-d command ? where does it go in the synapse?
Authored by: rpendell on Jul 15, '02 03:36:38PM

Where do I place th -d command in "sudo psync / /Volumes/Storage/" with Storage being my backup disk? psync duped my startup volume, I've backed up changed files, but I wanna know how to delete files on my backup that have been deleted since my last backup.



[ Reply to This | # ]
-d command ? where does it go in the synapse?
Authored by: David on Jul 15, '02 09:35:41PM

sudo psync -d / /Volumes/backup_here

David



[ Reply to This | # ]
Dev. Tools
Authored by: Gabhlan on Jul 11, '02 04:06:07PM

You need the developer tools installed for this one to compile whatever it is it downloads. Any idea if there's a binary available as I've precious little disk space left for the developer tools.



[ Reply to This | # ]
Dev. Tools
Authored by: natenate on Jul 11, '02 05:22:24PM
The author has a binary package up here.

[ Reply to This | # ]
here is the guy who wrote it !!!
Authored by: rxmac on Jul 11, '02 08:37:17PM

was mouseing around and found this he is the guy who wrote the script very informative site with lots of links.
http://www.dan.co.jp/cases/macosx/backup-volume.html



[ Reply to This | # ]
Just what the doctor ordered
Authored by: Anonymous on Jul 11, '02 10:40:04PM

This is great!

Thanks for the free backup solution! Finally cron & perl goodness makes the process simple with lots of backup options.

My only suggestion would be to edit the write-up with a link to cronnix to show new users how easy this can be (enable root--edit crontab with cronnix etc).

Y



[ Reply to This | # ]
Just what the doctor ordered
Authored by: David on Jul 12, '02 03:58:29PM

So how do you edit the main hint?

Anyways, here is what I have done to enable an automatic cronjob at midnight.
I don't have a link to Cronnix, but it has been mentioned on this site before.

Here are the instructions for setting up a root cronjob.

Type this:

sudo pico /etc/crontab

Type in your admin password and ou will then get a window with your root cronjobs in it.

Here is what mine looks like:

# $NetBSD: crontab,v 1.13 1997/10/26 13:36:31 lukem Exp $
#
# /etc/crontab - root's crontab
#
SHELL=/bin/sh
PATH=/bin:/sbin:/usr/bin:/usr/sbin
HOME=/var/log

#min hour mday month wday user command

# Disabled by default, since it causes disk access every 10 minutes,
# which is useless unless you use at(1). Enable as needed.
#*/10 * * * * root /usr/libexec/atrun

# do daily/weekly/monthly maintenance
15 3 * * * root sh /etc/daily 2>&1 | tee /var ... (etc)
30 4 * * 6 root sh /etc/weekly 2>&1 | tee /var ... (etc)
30 5 1 * * root sh /etc/monthly 2>&1 | tee /var ... (etc)

# do the backup
00 0 * * * root /usr/local/bin/psync / /Volumes/Maxtor\ \(FW\)

The last line is what you are interested in.

The first set of zeros stand for minutes, the second for hours, and I forget what the other three are for
(you can check in man crontab).

After the stars comes who you want to run the job (you need root to so that your permissions will be kept).

Then comes the job. You have to specify where psync is because zsh (which is what root runs the cronjobs with)
doesn't have /usr/local/bin in its path.

Save the file (Control - O) and quit (Control - Q).

Any other questions?

David



[ Reply to This | # ]
use Fink to install MacOSX::File?
Authored by: soob on Jul 12, '02 09:45:47AM

If you\'re using Fink, is installing the macosx-file-pm (lib.perlmod v. 0.61-1) package the same as \'install MacOSX::File\'?

That\'s what I did and it appears that psync is now installed - I went through the steps in this hint, skipping the \'install...\' line. I have not yet tried to back anything up.

Thanks for the great tip, Jim



[ Reply to This | # ]
use Fink to install MacOSX::File?
Authored by: David on Jul 12, '02 03:47:27PM

It is probably the same. Try this:

pico /usr/local/bin/psync

If anything shows up then you have psync installed.

David



[ Reply to This | # ]
use Fink to install MacOSX::File?
Authored by: soob on Jul 13, '02 01:03:23AM

great, thanks... BTW, with the Fink install psysnc is here: /sw/bin/psync



[ Reply to This | # ]
psync makes no backups, it synchronizes!
Authored by: loekjehe on Jul 13, '02 08:47:06AM

I have problems with the use of the term "incremental backup" for psync. An incremental backup is a file containing a copy of all the files that have changed since your last incremental backup. Psync doesn't make an incremental backup. It doesn't do anything else than synchronising the target folder with the original. This process is not even making a "differential backup", since with a differential backup you have a full backup and a set of differential backup files.

I mention this, because synchronising volumes can have disadvantages to creating backup files because you cannot go back to an earlier situation (e.g. with virus problems or corrupt files etc.): with psync you always have a "fresh" full backup.

When you want to make "real" incremental or differential backups using a free CLI utility, I recommend hfspax from Howard Oakley (see www.versiontracker.com).



[ Reply to This | # ]
hfspax - backing up on OS 10.2
Authored by: chanlotus on Oct 07, '02 07:06:42PM

I need a compressed backup. hfspax has been working great for OS 10.1x. However,
it breaks in OS 10.2 returning error as followed and the backup is not complete:

hfspax: Read fault on usr/bin/hfspax <Invalid argument>

Has any experienced the same and is there any solution for it? It'll be great help
if anyone gives me a hint on it. I need that bad!!! Thanks in advance...



[ Reply to This | # ]
hfspax - backing up on OS 10.2
Authored by: bluehz on Oct 08, '02 09:00:01AM

I tried for over a month to get usable backups with hfspax. I never seemed to have any trouble actually creating the archives - but accessing the archive to extract any backup file was like pulling teeth. I finally gave up. Am now trying hfstar.



[ Reply to This | # ]
Can't get psync to work
Authored by: rpendell on Jul 13, '02 12:47:42PM

I followed instructions but I get a prompt in the terminal saying MD5 not installed and "Oops,make had returned bad status". I d/l'd and compiled MD5 but it still doesn't let me get psync. I tried "install Bundle::" to no avail. PLEASE HELP!! This is the tool I've been looking for for quite awhile!!!



[ Reply to This | # ]
Can't get psync to work
Authored by: Thom on Jul 13, '02 04:50:18PM

I had the same problem, but figured that something threw CPAN out of whack.

So I exited it, and started it again using the same command, and this time I was able to install it just fine.

The whole laborious process of upgrading CPAN itself was sort of silly, especially the part where it tried to upgrade perl 5.6.0 to 5.6.1 and then after asking me a bajillion config questions (many of which I had no idea about), the compile died almost immediately. Bah.

But once that was done, I also took its (repeated) advice and told it to

install MD5

so it would quit whining.


Thom



[ Reply to This | # ]
this message board needs stripslashes!
Authored by: Thom on Jul 13, '02 04:53:40PM

I didn't notice that the post I submitted was backslash-escaped in the title, but I sure noticed when I submitted and saw three backslashes.

Doh! I've written messageboards before using PHP, had it happen.



[ Reply to This | # ]
Can't get psync to work
Authored by: David on Jul 15, '02 01:44:23PM

You can try this:

force install Bundle

David



[ Reply to This | # ]
psync obsesses on mp3s
Authored by: dm2243 on Jul 15, '02 11:46:47PM

This is odd. Every time I run psync (which is part of my daily script) it copies all my (800) mp3s - even though these files haven't changed.

Has anyone else observed this type of compulsive behavior?



[ Reply to This | # ]
psync obsesses on mp3s
Authored by: rouet on Jul 23, '02 04:40:45AM

I have the same comportment with my mp3 files.
If you don't wait between 2 psync executions, it works.
But if you wait one day, it copies the mp3 files.
My copy is on a removable disk in an extension bay.
Very strange...



[ Reply to This | # ]
psync obsesses on mp3s
Authored by: Accura on Aug 22, '03 02:44:25AM

maybe because iTunes updates the play-count of the mp3 files?

---
"The time has come," the walrus said. "To talk of many things..."



[ Reply to This | # ]
psync don't work for me in a cron job
Authored by: stift on Jul 23, '02 08:51:08AM

i installed the psync modul without any problems, creating 2 files where i backup my ibook and stationary g3 vice versa, i have no problems using this scripts from the comand line manually, but putting it to my crontab brings the error message psync, command not found, here is one example of my scripts:

echo "ibookole hat folgende Elemente vom g3 gebackupt:"
psync -d -r /Volumes/G3/Dokumente/CD-Finder/ /Users/Shared/G3-backup/CD-Finder/
psync -d -r /Volumes/G3/Dokumente/Frau\ Schmidt/ /Users/Shared/G3-backup/Frau\ S
chmidt/


any ideas ???



[ Reply to This | # ]
psync don't work for me in a cron job
Authored by: rouet on Jul 23, '02 10:35:39AM

psync is installed in /usr/local/bin
try to put the absolute path to psync in your crontab file

JR



[ Reply to This | # ]
thanx
Authored by: stift on Jul 23, '02 05:50:30PM

thats it really don't understand but now it works
fine fine




[ Reply to This | # ]
PsyncX released
Authored by: David on Aug 31, '02 05:17:31PM
You can now download PsyncX (a GUI for psync) from my SourceForge site: http://sourceforge.net/projects/psyncx David

[ Reply to This | # ]
PsyncX, Target Disk Mode, had to
Authored by: sshears on Sep 05, '02 10:33:13PM

Thanks to the author for psync and thanks, David, for PsyncX. Looks very good.

I used PsyncX on a TiBook to create a copy of my hard disk onto another PowerBook (a Pismo) in firewire target disk mode. I wanted to "clone" my whole computer. (... and then use the second machine as a test-bed for installing and checking out Jaguar...)

When PsyncX finished, Pismo wouldn't boot. Putting it back in target disk mode, I ran "bless" and now it boots into OS X just fine.

The command was

sudo bless -folder "/Volumes/Pismo-volume-name/System/Library/CoreServices"

-- Sally



[ Reply to This | # ]
PsyncX, Target Disk Mode, had to
Authored by: sshears on Sep 07, '02 04:50:41PM

Just a followup note... I also had to bless the OS 9 folder in order to start classic.

Without the blessing, the Classic startup window just sat there with a happy mac for a long, long time.



[ Reply to This | # ]
PsyncX, Target Disk Mode, had to
Authored by: sshears on Sep 11, '02 08:38:53AM

It's just a bit more complicated... Repeating the process, I find I have to bless both the OS X and the OS 9 folders in the same bless command:

sudo bless -folder "/Volumes/vname/System/Library/CoreServices" -folder9 "/Volumes/vname/System Folder"

(Use a backslash at line end to continue the command on a second line.)

It was useful to examine the results of "bless" by using the option

bless -info "Volumes/vname"

..and compare to the output of bless -info "/"



[ Reply to This | # ]
Use psync to create incremental backups
Authored by: ssevenup on May 07, '03 08:15:08PM

Has anyone figured out how to get CRON to mount an AFS share and run the psync job unattended? One assumes there would be a password involved, so how does one handle that?

--MM

---
Mark Moorcroft
ELORET Corp. - NASA/Ames RC
Sys. Admin.



[ Reply to This | # ]
Use psync to create incremental backups
Authored by: ssevenup on May 07, '03 09:33:17PM

Never mind... "man mount_afp" and "man mount"

--rtfm

It would be cool if you could just hand an afp:// mount command to psync on the command line though.

---
Mark Moorcroft
ELORET Corp. - NASA/Ames RC
Sys. Admin.



[ Reply to This | # ]
Use psync to create incremental backups
Authored by: ssevenup on May 07, '03 09:37:09PM

Anyone else seeing failures copying application bundles? I'm attempting this over an afp:// share. It seems that parts of the copy succeed and others fail. Is there a likely discussion forum for psync?

--MM


---
Mark Moorcroft
ELORET Corp. - NASA/Ames RC
Sys. Admin.



[ Reply to This | # ]
Use psync to create incremental backups
Authored by: bluehz on May 08, '03 09:10:11AM

I often see failures - specifically with Application bundles while
using Psync, RsyncX, and CarbonCopyCloner. As far as I am
concerned this is a MAJOR FALURE. For one thing - most times
you are not even given an sort of failure notice, and only find
out about the failure when you try to launch the Application. I
admit that in a recent repartition and reload of a 20G disk
packed with applications - using CarbonCopy Cloner - I Have
only run into 2 applications so far that I have had to reinstall.....
BUT the operative word here is "...so far". Since I have yet to
launch EVERY app I copied - I don't know what the future holds.
As far as I am concerned ANY methodology used for bakup/
cloning is NO GOOD unless it can perform 100% accurate
backups OR at least notify you of which files have failed.



[ Reply to This | # ]
WARNING: psync and MS-DOS partitions
Authored by: tcurtin on Jul 11, '03 01:27:52PM

Just a warning to all - psync does not handle syncing to windows partitions at all well. Icon files get dropped, as do files containing many special characters. (':' for one, which is found quite often.)

In other good news, backing up to HFS+ partitions seems to be much quicker than FAT_32.

I tried backing up my hard drive to an external firewire drive with one single large fat_32 partition created on my windows box. Luckily I noticed the error messages, and decided to try reformatting into 1/2 HFS+ and 1/2 MS-DOS. Do a search on FAT32 for a hint describing one way to make two partitions for this purpose. In addition to that hint, another good command-line utility is diskutil - type 'man diskutil' in a terminal window for help, and be very careful - its powerful!

Hope that helps - enjoy!



[ Reply to This | # ]
PsyncX on VersionTracker
Authored by: SteveJ on Jul 13, '03 01:13:13AM

There's a PsyncX on VersionTracker,
http://versiontracker.com/dyn/moreinfo/macosx/16089
Is this the same as in this hint?



[ Reply to This | # ]
PsyncX on VersionTracker
Authored by: bluehz on Jul 13, '03 08:32:55AM

PsyncX is exactly the same thing as in this hint - only it has a gui frontend to make it easier to use (although in my opinion it actually makes it more difficult and slower).



[ Reply to This | # ]
Use psync to create incremental backups
Authored by: srteresi on Oct 21, '03 07:21:34PM

Beware... psync doesn't appear to copy Finder comments (that's text you may have pasted into the comments field of a file's Get Info box). I have extensive information stored in Finder comments and hence can't use psync.

Does anyone know of a synchronization program which will preserve file comments, permissions, modification dates (created/modified/accessed), and all such file metadata? I bet one doesn't yet exist! This page lists some options, but none appear comprehensive:

<http://nrg.cs.usm.my/~tcwan/macosx_essentials.htm>

Scott



[ Reply to This | # ]