Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

10.4: Monitor file system events in real time System 10.4
OS X 10.4 has a private API that's used by Spotlight to monitor file system events such as file creation, renaming, and permission changes. Several command line and GUI tools are now available that can suscribe to the event notification system and provide a log of the events. These are useful for, e.g., monitoring files created by installers, and so forth. For reasons that will become clear in a moment, let me first mention a few GUI tools:
  • fseventer (donationware): Provides a real-time graphical view of changes occuring in your filesystem.
  • Sonar (commercial): File activity monitor (I haven't tried it).
  • Yank (commercial): An uninstaller (I haven't tried it).
Now for the meat of this hint, the command line tools...

Copy each of these binaries to /usr/local/bin, and run each in Terminal using sudo, e.g., sudo fslogger.
  • fslogger: Provides verbose, plain text output. Source code not available online.

  • fetool: Source and binary included with fseventer (GUI app above). The binary is located in the app bundle in fseventer.app -> (Control-click and Show Package Contents) -> Contents -> Resources. It produces compact, plain text output that uses the following event codes:
    -1: FSE_INVALID
    0: FSE_CREATE_FILE
    1: FSE_DELETE
    2: FSE_STAT_CHANGED
    3: FSE_RENAME
    4: FSE_CONTENT_MODIFIED
    5: FSE_EXCHANGE
    6: FSE_FINDER_INFO_CHANGED
    7: FSE_CREATE_DIR
    8: FSE_CHOWN
  • Sandal: So far as I can tell, you have to dig into the Sonar bundle (GUI app above) to find the binary and source code, Sonar.app -> (Control-click and Show Package Contents) -> Contents -> Resources. It produces XML output.
A typical use of these commands would be to first run one in Terminal, redirecting output to a file, then run an installer, terminating the command when the installer is done. Now you have a record of all file system changes caused by the installer. For instance:
$ sudo fslogger > ~/Desktop/NewAppFiles.log
Related links: [robg adds: I haven't tested any of these apps, GUI or command line. If you are going to test them, you should read the caveat on the fslogger page -- here's a snippet of it:
If you cause heavy enough file system activity (what's "heavy" will vary greatly, depending on your system and its currently available resources), both fslogger and Spotlight may miss events, causing Spotlight to spend some extra time looking at your volume. Note that Spotlight will not reindex the entire volume — it will only look for the changes that it missed.
So if you try these tools and Spotlight seems to be working a bit harder than usual, the above may be the reason...]
    •    
  • Currently 3.00 / 5
  You rated: 3 / 5 (5 votes cast)
 
[50,666 views]  

10.4: Monitor file system events in real time | 10 comments | Create New Account
Click here to return to the '10.4: Monitor file system events in real time' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
10.4: Monitor file system events in real time
Authored by: ktappe on Aug 18, '06 08:26:05AM

I can heartily recommend Sonar--we had been using loggen but it missed several filesystem changes we'd been trying to spot apps making. Sonar picked them all up.



[ Reply to This | # ]
10.4: Monitor file system events in real time
Authored by: garion on Aug 18, '06 08:45:50AM
I've implemented a set Python bindings on this API.. I was frustrated at the lack of something like FUSE on OSX, so I used the API to wrote some python bindings to let me write a automatic SFTP file uploader (serves as a sample of using the API..).. You can find the bindings here.. As they always say, use at your own risk (you have to run it as root.)

[ Reply to This | # ]
10.4: Monitor file system events in real time
Authored by: grasshoppermouse on Aug 18, '06 11:35:12AM

It works on PPC. Sweet.



[ Reply to This | # ]
10.4: Monitor file system events in real time
Authored by: TvE on Aug 18, '06 05:54:42PM
So if you try these tools and Spotlight seems to be working a bit harder than usual, the above may be the reason...


Also interesting since these are the API's the Synk claims to be using - so if Synk does not backup all you changed files, this is why??? Not a good sideeffect for a backuptool!!!

[ Reply to This | # ]
10.4: Monitor file system events in real time
Authored by: osxpounder on Aug 19, '06 10:29:41AM

I see that fseventer is packaged for download with its source code. Has anyone here looked over that source code? Reason I ask: this app wants root access, and I wouldn't know how to review source code to look for any risks or whatever.



[ Reply to This | # ]
10.4: Monitor file system events in real time
Authored by: ManxStef on Aug 19, '06 02:49:04PM

What about fs_usage? (It's included as part of OS X 10.4.) From the manpage:

fs_usage -- report system calls and page faults related to filesystem activity in real-time



[ Reply to This | # ]
10.4: Monitor file system events in real time
Authored by: sd on Aug 28, '06 06:57:54AM

yes 'fs_usage' is great. And it can also track files that are read, not only the ones that are changed. It monitors all used files!



[ Reply to This | # ]
10.4: Monitor file system events in real time
Authored by: sd on Aug 28, '06 12:46:49PM
fs_usage use: fs_usage -w -f filesys | grep AppName

[ Reply to This | # ]
10.4: Monitor file system events in real time
Authored by: grasshoppermouse on Aug 20, '06 12:02:31PM
The command line tools linked in the hint all log events as they happen. The disadvantage of that is that filesystem activity by several processes are all mixed up. Using Garion's Python bindings, mentioned above, I whipped up a little script that sorts the file system events by PID. This makes it easier to examine filesystem changes caused by a particular app. To use it, you'll need to copy the file:
fsevents.so
from the bindings module, linked above, into:
/System/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/site-packages
(You can't do a normal Python install unless you have pyrex installed.) Then simply paste the following code into a text file, and save it in:
/usr/local/bin
and
$ chmod +x scriptname
Here's the script (you can name it whatever you want):

#! /usr/bin/python

# Raw, mostly untested code. Use at your own risk.

import fsevents, os, time

eventdict = {}

def deleteHandler(o):
    addEvent(o.pid, o.vnode, 'deleted')

def createHandler(o):
    addEvent(o.pid, o.vnode, 'created')
    
def modifiedHandler(o):
    addEvent(o.pid, o.vnode, 'modded')

def dirCreatedHandler(o):
    addEvent(o.pid, o.vnode, 'new dir')

def renamedHandler(o):
    addEvent(o.pid, o.string[0], 'mv from')
    addEvent(o.pid, o.string[1], 'mv to')

def addEvent(pid, pth, event):
    if pid not in eventdict: eventdict[pid] = {'name':getprocessname(pid), 'events':[], 'paths':[], 'time':[]}
    eventdict[pid]['events'] += [event]
    eventdict[pid]['paths']  += [str(pth)]
    eventdict[pid]['time']   += [time.strftime('%X %x')]

def getprocessname(pid):
    try:
        # Get the process name
        return os.popen("ps -p " + str(pid)).readlines()[1].split('/')[-1].split(' -')[0].strip()
    except:
        return '??'

def outputLog():
    for k in eventdict.keys():
        print
        print eventdict[k]['name'], k
        for i, e in enumerate(eventdict[k]['events']):
            print eventdict[k]['time'][i], e.ljust(7), eventdict[k]['paths'][i]

x = fsevents.Scanner()
x.setEventHandler( fsevents.FSE_DELETE, deleteHandler )
x.setEventHandler( fsevents.FSE_CREATE_FILE, createHandler )
x.setEventHandler( fsevents.FSE_CONTENT_MODIFIED, modifiedHandler )
x.setEventHandler( fsevents.FSE_CREATE_DIR, dirCreatedHandler )
x.setEventHandler( fsevents.FSE_RENAME, renamedHandler )
x.startScanning()

print "Logging..."
while 1:
    try:
        time.sleep(.5)
    except:
        x.stopScanning()
        outputLog()
        break
To use the script, type
$ sudo scriptname
in the terminal, launch your installer, or whatever. When you're done, terminate with ctrl-c, and all filesystem events recorded while the script was running will be output, sorted by process id.

[ Reply to This | # ]
10.4: Monitor file system events in real time
Authored by: palahala on Jun 04, '09 01:12:51PM
See also Apple's Monitoring File Changes with the File System Events API for a hands-on to create a sample application.

[ Reply to This | # ]