Add timestamps to Unix commands that run at intervals

Jan 22, '10 07:30:00AM

Contributed by: robg

For a little project I was recently working on, I wanted to track some memory usage statistics over time. I figured that the Unix command vm_stat would be a good way to do that, as it includes the four basic memory usage types (free, active, inactive, and wired). My intent was to put this in a shell script that would run the command at a specified interval, dumping the output to a text file each time.

However, the basic output of vm_stat is less than ideal for dumping to a file via a shell script:

$ vm_stat
Mach Virtual Memory Statistics: (page size of 4096 bytes)
Pages free:                         486663.
Pages active:                       140886.
Pages inactive:                      92592.
Pages speculative:                  189887.
Pages wired down:                   138988.
"Translation faults":              1221453.
Pages copy-on-write:                 63521.
Pages zero filled:                  709169.
Pages reactivated:                       1.
Pageins:                             59295.
Pageouts:                                0.
Object cache: 28 hits of 34093 lookups (0% hit rate)
$
Parsing the above in a spreadsheet app would require some serious text manipulation. A brief glimpse at the man page for vm_stat showed another usage option: vm_stat interval, where interval is how often (in seconds) vm_stat should measure memory usage. When used in this way, the output is much nicer for capture:
$ vm_stat 2
Mach Virtual Memory Statistics: (page size of 4096 bytes, cache hits 0%)
  free active   spec inactive   wire   faults     copy    0fill reactive  pageins  pageout
486024 139635 191315    92771 138964  1224506    63581   711642        1    59301        0 
486194 139900 191372    92799 138445      392        0      366        0       10        0 
485056 139830 191381    93499 138966      100        0       36        0       42        0 
...
That was nearly perfect, though I only needed the first 36 characters (the end of the wire column). But I had a problem: I needed to include a timestamp on each row of the output, so I could tie memory usage back to some activities I was starting and stopping at specific times. Read on to see how this is done...

I didn't have the slightest idea how to add text to each row of output for a command that was running in a loop. For help, I turned to a guy who literally wrote the book on Perl, Randal Schwartz and who, coincidentally, also lives in the Portland area. (Odd aside: despite living within miles of each other, the first time I met Randal in person was in Florida for a Geek Cruise!).

After describing my desired output to Randal, he quickly sent back the following piece of Perl magic, which I'm not even going to attempt to decode. Instead, I'll just present it as "it works!," and it can be fairly easily used to attach a timestamp to any Unix program that creates output on a set interval. Here's the final version of Randal's solution:

vm_stat 2 | perl -lne '$| = 1; while (<STDIN>) { $z = localtime; substr($_,36) = ""; substr($_,0,0) = substr($z, 11, 8). ": "; print }'
Note that the above is designed to work with vm_stat output in 10.6; in 10.5, it's slightly different, and the substr($_,36) bit would need to have the 36 character count adjusted to match. The output of the above command is:
05:39:35:   free active   spec inactive   wire
05:39:35: 472838 139574 201659    95823 138999
05:39:37: 472468 139841 201660    95865 138959
05:39:39: 472443 139840 201661    95865 138958
05:39:41: 472213 139689 202127    95915 138940
...
That's exactly what needed. I added a >> /path/to/savefile.txt to dump the output to a text file, and had exactly what I needed. With some tweaking to the first substr command (which is what cuts the columns you want to see), this should work with most any command that generates continual output over time (though I've only tested it with vm_stat so far). Thanks Randal!

Comments (20)


Mac OS X Hints
http://hints.macworld.com/article.php?story=20100122052015773