Tools like rsync and lftp do a great job of mirroring directories between systems and only copying files that have been updated. On non-LAN networks, however, they still can be slow, since they have to query the remote system over the network. lftp, in particular, is slow
This tip provides a mechanism for testing locally if anything has changed below a directory, and if so, invoking the command of choice to do an update. This is much better if you want to look for changes every minute or every five minutes, since it's nearly instantaneous.
Installation / Configuration:
Download the script and make it executable with the following commands:
$ cd /path/to/desired/save/location
$ curl -O 'http://marty.feebleandfrail.org/macosxhints/modtest/modtest'
$ chmod a+rx modtest
Read the rest of the hint for some usage instructions...
Here's some information on the command and its various options:
modtest -d DIRECTORY [options]
modtest -rebuild [options]
Options
-------
-f|-file FILENAME File to read/store MD5s (default: ~/.modtest-data)
-l|-log FILENAME File to write log to (default: ~/.modtest-log)
-d|-dir DIRECTORY Directory to examine for changes
-c|-cmd COMMAND Command to run if changes found
-rebuild Rebuild MD5 file to reduce size
-noupdate Do not update MD5 data
-q|-quiet Quiet output (no debug messages)
-s|-silent Silent output (no messages at all)
-stdout Output to stdout
-nostdout No output to stdout
Version: Fri Mar 4 13:15:22 EST 2005
The -file and -log options default to two files in your home directory; you can modify the script if you want them somewhere else.
0 no changes detected or data file rebuilt OK
1 changes detected
2 error
3 help message
So a simple script to update a web site using lftp would look like this:
#!/bin/sh
modtest -nostdout -d /Users/msells/public_html/
case $? in
0) ;; # No changes
1)
echo Changes detected.
cd /Users/msells/public_html/
lftp -u user,pass 10.0.0.1/www/public_html/ <<END
mirror -R --parallel=4 --use-cache
END
;;
esac
Notice that we use the -nostdout option so that modtest doesn't output anything, since we're only interested in the exit code. We could have used -silent, which would also prevent modtest from recording things in its log file.
#!/bin/sh
#while [ 1 ] ; do
if ! test -t 0 ; then XOPTS='-q' fi
modtest -d /barn/mira/feeble_www/public_html/ $XOPTS -c
'(time rsync --delete -a -e ssh /barn/mira/feeble_www/
public_html/ marty@www.host.com:feeble/www/public_html/) 2>&1'
modtest -d /barn/mira/feeble_marty/public_html/ $XOPTS -c
'(time rsync --delete -a -e ssh /barn/mira/feeble_marty/
public_html/ marty@www.host.com:feeble/marty/public_html/) 2>&1'
#sleep 10
#done
Getting the -c parameter right might take some fiddling; you can use something like this for testing purposes:
-c 'echo "Changes at " `date` >> /tmp/changelog'
Calling from cron:Mac OS X Hints
http://hints.macworld.com/article.php?story=20050313201539163