Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

A Perl script to watch files for changes UNIX
OS X applications make use of various support files and these files get read and written quite frequently behind the scenes. Especially when troubleshooting some problem, it is often useful to be notified when relevant files get modified. I wrote a Perl script (which I call watchfile [4kb download from macosxhints.com]) that prints a message when one of the specified files gets modified. As usual, you need to make the script file executable (with chmod +x) and install the script in a folder that is in your shell's execution path.

You execute this script in a Terminal window, specifying the file or files you want it to watch in the usual way (a space-separated list of filenames). For example, if you want it to watch the file ~/Library -> Preferences -> com.apple.finder.plist, you would run the command:
 % watchfile ~/Library/Preferences/com.apple.finder.plist
Or if you wanted to watch all of the com.apple preference files, you would run the command:
 % watchfile ~/Library/Preferences/com.apple.*
The script will print a message "info stored" when it starts watching each file and then it will check on the file each 10 seconds after that and print a message if anything about the file has changed. The messages are somewhat cryptic, indicating which things have changed since the last check. Read the rest of the hint for a breakdown on the messages you might see...

[robg adds: I tried this script, and it's really pretty interesting. It's amazing to see how much stuff is changed, and how often it's changed.]

Here's a list of the things that might be reported:
  • ino - The inode number (numerical id) of the file has changed. This means that the file has been removed and recreated (with the same name).
  • mode - The file permissions have changed.
  • nlink - The number of hard links to this file have changed.
  • uid - The owner of the file has changed.
  • gid - The group ownership of the file has changed.
  • size - The size (in bytes) of the file has changed.
  • md5 - The MD5 digest of the file's contents has changed. (See option "-md5" below)
  • atime - The time of last access of the file has changed. (See option "-atime" below)
  • mtime - The time of last data modification has changed. This usually means that the contents of the file have been modified.
  • ctime - The time of last status change has changed. This usually means that the ownership or permissions have changed.
Each time a change is reported, the script also shows the result of doing ls -l on the file.

Options:
  • -interval By default, the script checks on the file(s) every 10 seconds. If you want to specify a different time interval, use the -interval option. E.g.: watchfile -interval 60 ~/Library/Preferences/com.apple.*. The longer the time interval, the less CPU will be used. But normally, CPU usage of this script is not a problem. On my iBook 600, watching all the com.apple preference files with the default 10 second interval takes much less than 1% of the CPU.
  • -atime By default, the script does not report on access-time changes because changes to the files are more significant. And some files are accessed quite frequently so being notified each time would add to the noise. If you want to be notified of access-time changes, use the -atime option. E.g.: watchfile -atime ~/Library/Preferences/com.apple.*
  • -md5 If you want the script to calculate and compare an MD5 digest for each file, use the -md5 option. This is an advanced option and not usually desirable unless you are monitoring a file for security purposes. The use of this option requires the Perl module Digest::MD5 which is not installed by default on OS X. Note that access-time changes will not be reported if the -md5 option is used since the file is accessed in order to compute the MD5 digest. E.g.: watchfile -md5 ~/Library/Preferences/com.apple.*
In my testing, I found it very interesting to watch the com.apple preference files, especially with the -atime option. I was surprised how many preference files are being accessed and how often they get written to (and often recreated).

One final note: Although I haven't yet tried it, I think the script should need only small changes to get it to work on other operating systems (anywhere that Perl is installed). It should work as is on any Linux system. After changing the line that does the ls -l, it should even work on Windows if you install Perl.
    •    
  • Currently 2.29 / 5
  You rated: 5 / 5 (7 votes cast)
 
[42,111 views]  

A Perl script to watch files for changes | 22 comments | Create New Account
Click here to return to the 'A Perl script to watch files for changes' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
How about instead of listing the change, execute a script?
Authored by: pwharff on Oct 01, '03 01:01:28PM

WOW, nice script! I've actually been looking for something like this. Question, can this script actually run in the background and execute another script if the contents of a folder have changed??? Or does anyone else know of an alternative that will execute scripts when a file or contents of a folder have been changed?



[ Reply to This | # ]
Tried it on SuSE Linux
Authored by: pwharff on Oct 01, '03 01:07:14PM

You mentioned that it should work on any linux, well I tested it on SuSE Linux 8.2 and it works as it should! Thanks again for the nice script, I can find many uses for it! Another files that's fun to watch is /var/log/messages in linux ofcourse



[ Reply to This | # ]
Very useful
Authored by: ssevenup on Oct 01, '03 01:18:03PM

Thank's, this is one of those things you never quite get around to writing, but wonder how you got by without once you use it.

I hope someone picks up the baton and builds a Cocoa app with a drag-and-drop file well. Sounds like a very nice project to start learning Cocoa. Maybe I can find time this fall. Of course then it's not so nice and portable anymore.

--Thank You

---
Mark Moorcroft
ELORET Corp. - NASA/Ames RC
Sys. Admin.



[ Reply to This | # ]
another alternative would be Tripwire
Authored by: weefle on Oct 01, '03 01:42:59PM

Install the Tripwire filesystem monitoring tool

Tripwire does pretty much the same thing as this Perl script does with the -md5 option, but with the option of customizing what checks are performed on each category of files. So, for example, you can simply check to make sure that your log files haven't been deleted and haven't had their ownership changed, but you can also check to make sure that system executables haven't had their contents replaced with Trojan code. It takes a lot of work to customize properly, but once you're done, you can be relatively sure that you'll catch any system modifications that are done.

Tripwire has been available for Mac OS X for a few months now.



[ Reply to This | # ]
comparing the alternatives
Authored by: hayne on Oct 01, '03 10:59:15PM
I think Tripwire occupies a different ecological niche. It is explicitly concerned with security and is far more sophisticated, with concomitant complexity of use.

Of similar sophistication but in a slightly different niche is Radmind - it is usually used for maintaining multiple machines in a known state.

A bit lower on the complexity scale but still considerably more sophisticated than the 'watchfile' script is the bubblegum program. It is a compiled C program that is designed to run as a daemon.

The 'watchfile' script was intended mostly for impromptu troubleshooting sessions where the ease of modification of a script (as opposed to a compiled executable) is often a big advantage.

[ Reply to This | # ]

Using bubblegum
Authored by: pwharff on Oct 02, '03 05:07:20PM

hayne,

I have been using your script watchfile and it works great, but it always has to be running and I have to have a Terminal window open always. So I tried bubblegum, but I couldn't get it to work with directories and I emailed the developer and no response so far.



[ Reply to This | # ]
running programs in the background
Authored by: hayne on Oct 02, '03 10:53:02PM
You don't need to run the 'watchfile' script in the foreground of a Terminal window. You can start any program in the background by adding an ampersand (&) at the end of the invocation command. If the program sends results to the terminal window (as 'watchfile' does) then you need to redirect the output into a file.

E.g. you could run the above example in the background as follows:
watchfile ~/Library/Preferences/com.apple.* > ~/myoutput &
where I have redirected the output into the file ~/myoutput
When you start a program in the background like this, it stays running even when the Terminal window is closed. In fact it will stay running until the machine is rebooted. You can examine the output file whenever you want. Doing it this way gives you something very much like what bubblegum does.

[ Reply to This | # ]

running programs in the background
Authored by: pwharff on Oct 03, '03 05:57:06PM

Thanks a bunch, I'm somewhat new to unix/linux. What happens if you dont redirect the output to a file and if there is eventually output, where does that output go or is there an error?



[ Reply to This | # ]
running programs in the background
Authored by: pwharff on Oct 03, '03 06:07:52PM

Also, if I were to run this script watchfile in the background as you described and quit the Terminal, how would I later on kill this process. Usually I use "jobs" to find the currently running jobs, but this only applies to my current tty.

Thanks again for your help, I'm learning.



[ Reply to This | # ]
ps
Authored by: hayne on Oct 03, '03 08:49:49PM

You can kill any process that you started if you know the process id (pid) by using the 'kill' command. You can find out the pid by using the command 'ps'.
Or if you know that there is only one instance of a process with a particular name, you can use 'killall'.

May I suggest that you should read some of the many freely available UNIX tutorials? There are some listed in the links section of this site.



[ Reply to This | # ]
I was able to figure it out!
Authored by: pwharff on Oct 01, '03 02:41:31PM

I hope I did this correctly as I am new to perl. In my previous question I wanted to know if instead of notifying the user of changes, could it just do something else, like run a script? Well I digged around your script and this is what I changed (I hope this is ok under your terms?):

# print "$date: $filename: $msg\n";
system("sh /home/paul/CreateIndex") if $show_info;

This now runs my script when a particular folder is updated. This is much better than a cron job that runs the script every 5 minutes even if the folder wasn't updated.



[ Reply to This | # ]
re: running script
Authored by: hayne on Oct 01, '03 03:53:31PM
Of course there is no problem with you changing the script as you like. The license (Perl's "Artistic License") allows you to do more or less whatever you want as long as you maintain the copyright notice and indicate what things you changed.

Your changes to the watchfile script (to make it invoke another script when a change is detected) seem like they will do what you want. You might want to enhance your shell script to handle command-line arguments and then you could pass it interesting information such as which file got changed.
E.g.:
system("/bin/sh /home/paul/CreateIndex $filename") if $show_info

[ Reply to This | # ]

re: running script
Authored by: pwharff on Oct 01, '03 06:53:15PM

Cool, I had not thought of that. I will make the change now. And thanks again for writing this script. I found another program in Linux that does almost what yours does and is called watch.



[ Reply to This | # ]
I was able to figure it out!
Authored by: damjan on Dec 08, '03 03:35:47AM

hi ,

Is it possibile to change script to send email to let say administrator everytime some files in folder are changed?

how?

br
Damjan



[ Reply to This | # ]
watching folders
Authored by: hayne on Oct 01, '03 06:51:52PM
When I wrote the script, I hadn't considered using it to watch folders but I see that this can be useful too - e.g. pwharff's comment indicates that he is using it to monitor a folder for files added.

To make the script work better when used to watch folders, I would recommend changing the line that does the 'ls -l' to use 'ls -ld' instead. I.e. change it to:
system("/bin/ls -ld \"$filename\"") if $show_info;
This will make it just list the info about the folder itself instead of listing all the files in that folder (which gets rather noisy if the folder contents change much). It will still give you the info for a file as before, so it's a win-win scenario.

In case you wanted to do something different with folders that have changed as opposed to files that have changed, you could test if $filename is a folder by using if (-d $filename)

[ Reply to This | # ]

Why isn't this built in to the FS?
Authored by: martinx on Oct 01, '03 11:33:49PM

What if I want to watch every file in the FS? Not gonna work to poll every file every 10 seconds or probably even every minute. It seems to me like the proper place for this would be the file system. I remember a utility a long time away (in an OS far far away) which would basically trap all system calls so you could see every change to the filesystem.

Why isn't Apple innovating and improving their file system? I know I've complained about Panther not having any DB stuff in the FS, maybe its time to start looking at Reiser 4, it should be out of alpha soon. (Now if only I could completely replace HFS+ with that...)



[ Reply to This | # ]
SGI's File Alteration Monitor
Authored by: ethomas on Oct 02, '03 04:45:57AM
Have a look at the FAM project on SGI's website.
FAM, the File Alteration Monitor, provides an API that applications can use to be notified when specific files or directories are changed.
FAM comes in two parts: fam, the daemon that listens for requests and delivers notification, and libfam, a library that client applications can use to communicate with fam.
FAM was originally written for IRIX in 1989 by Bruce Karsh, and was rewritten in 1995 by Bob Miller. This open-source release of FAM builds and runs on both Linux and IRIX, and is almost identical to the version of FAM that ships with IRIX 6.5.x.


[ Reply to This | # ]
SGI's File Alteration Monitor
Authored by: martinx on Oct 02, '03 10:16:21AM

Interesting! Which method would be used under OS X - IMon, DNotify, kqueue, polling?



[ Reply to This | # ]
A Perl script to watch files for changes
Authored by: Manoj Manchanda on Nov 12, '05 01:05:34PM

Hi,

Any body can provide me a prepared script that I can just copy and give execute permission to script for the purpose as mentioned below:

Checks the folder and send the mail to specified users for updates like
file or folder creation, deletion or change.

Thanks in advance for your help.

Best Regards,
Manoj Manchanda



[ Reply to This | # ]
updated version of script
Authored by: hayne on Mar 25, '06 10:13:21PM
I made a few improvements to the 'watchfile' script:
  • better behaviour when the files being watched don't initially exist, or when they come and go
  • option to watch for changes to the resource fork as well
The revised version is available here

[ Reply to This | # ]
updated version of script
Authored by: osxpounder on Aug 22, '06 03:38:12PM
When I try to run this command, in Terminal, I get an error. Knowing nothing about Perl scripts, I'll just show you what I typed as a command, and then the error, in hopes you can tell me what I'm doing wrong. I typed:
./watchfile.perl.script ~/Library/Preferences/com.apple.* > ~/myoutput.txt &
Then Terminal displayed:
Bareword "O_RDONLY" not allowed while "strict subs" in use at ./watchfile.perl.script line 283.
As far as I can tell, line 283 is as follows:
sysopen(FILE, $filename, O_RDONLY)


[ Reply to This | # ]
updated version of script
Authored by: hayne on Dec 11, '06 08:42:56AM
Sorry, somehow the updated version of that script was missing one line.
I have now fixed it.
(http://hayne.net/MacDev/WatchFile/)

[ Reply to This | # ]