Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!


Click here to return to the 'Add timestamps to Unix commands that run at intervals' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
Add timestamps to Unix commands that run at intervals
Authored by: Hal Itosis on Jan 23, '10 10:34:22AM

@ bashbash:

>> I'm not sure why you need awk or perl at all.

Because... awking 4 fields will work no matter what their combined width is. Fixating on a 36-char width only works for versions of vm_stat which produce that particular output. As stated in the article, the outputs from 10.5. and 10.6 have slight differences, and who knows if 10.7 will stay with 36-chars? Working with *fields* instead of specific widths obviates all that incompatibility and uncertainty.


>> This simple script also strips the vm_stat headers and formats the output.

Stripping the headers may be nice (or maybe not), but those numbers in the columns are printed out *left*-justified... which is not exactly a desirable "format". ;-)

--

PS: could some mod/admin please cleanup the code tag problem? I think slvrstn's post is missing a closing code tag.



[ Reply to This | # ]
Add timestamps to Unix commands that run at intervals
Authored by: bashbash on Jan 24, '10 05:46:53AM
Many apologies, I should have explained my script.

If you don't want to filter out the headers, don't apply the filter:

vm_stat 2|while read l;do echo $(date +%H:%M:%S) ${l:0:36};done

If you want to retain the original formatting, pop in some quotes:

vm_stat 2|while read l;do [[ ! $l =~ [A-z] ]]&&echo $(date +%H:%M:%S) "${l:0:36}";done

If you want to use positional parameters and printfs use something like this:

vm_stat 2|while read a b c d e;do printf "%s %9s %9s %9s %9s %9s\n" $(date +%T) $a $b $c $d;done

That one mimics the previous awk script.

I thought the OP wanted something that was easily machine readable. In that case maybe something like this would do the trick:

vm_stat 2|while read a b c d e;do [[ ! $a =~ [A-z] ]]&&echo $(date +%T) $a $b $c $d;done

In all cases, bash has plenty of builtin formatting without resorting to language soup :)

I have a great fondness for other languages (especially awk), but let's not perpetuate the myth that bash cannot do these things just as easily. There is also an overhead of invoking extra interpreters.

[ Reply to This | # ]

Add timestamps to Unix commands that run at intervals
Authored by: slvrstn on Jan 24, '10 01:32:13PM
OK, let's look at this extra overhead issue, because it's exactly what I was talking about in my original post when I mentioned the advantage of not shelling out to get the timestamp.

To benchmark, we want to generate a lot of data quickly, so I used

time od -N 1000000 -D /dev/random | wc
   62501  312501 4625008

real	0m0.310s
user	0m0.170s
sys	0m0.153s

to generate around 62k lines of the form

0000000         455208253      2432249641      1505621869      3979473917
0000020        2293418554      1733714563      3308048974       965145382
0000040        2395066143      3983717352      2757720288      3811753968
...

You see that it takes less than half a second to generate the data.

Then, comparing methodologies:

Perl

time od -N 1000000 -D /dev/random | perl -ane 'printf "%.2d:%.2d:%.2d: @F[0..4]\n", (localtime())[2,1,0]'  > /dev/null

real	0m0.869s
user	0m0.813s
sys	0m0.162s

bash

time od -N 1000000 -D /dev/random | while read a b c d e;do printf "%s %9s %9s %9s %9s %9s\n" $(date +%T) $a $b $c $d;done  > /dev/null

real	3m43.331s
user	0m52.922s
sys	2m27.505s

So, it takes less than a second with the pure Perl and over 3 1/2 minutes in pure bash.

Any approach that has to invoke a new command to get the timestamp each time through the loop is going to suffer by the same comparison.

[ Reply to This | # ]

Add timestamps to Unix commands that run at intervals
Authored by: Hal Itosis on Jan 24, '10 03:17:47PM
Holy Moly Perl-man... point well made, and taken. 8-)

How is it that Perl manages to get the current timestamp
for each iteration so much _more_ quickly? Doesn't it need
to request that info (in real time) from the OS as well?
[it also seems to parse the data strings faster too]

Anyway, thanks for that impressive proof.

[ Reply to This | # ]
Add timestamps to Unix commands that run at intervals
Authored by: slvrstn on Jan 24, '10 09:07:11PM

In the end, both perl and the `date` command are making the same system library call, time.
See `man -s 3 time`

The difference is all in the overhead in setting up the execution environment.
In the perl, this is done once and the interpreter can then make rapid-fire system calls.
In the bash (and awk) examples, every invocation of `date` has to exec a full new command / execution environment, which is comparatively extremely resource intensive.

bash is a fantastic and very capable shell, but unless your script is written entirely with built-ins, you are going to pay a performance penalty for every external command invoked. This isn't something exclusive to bash, of course. You have the same issues in Perl, in fact, if your script relies heavily on using the system, exec, or backtick commands.



[ Reply to This | # ]
Add timestamps to Unix commands that run at intervals
Authored by: bashbash on Jan 25, '10 12:28:53AM

Wow!

May I humbly suggest to give C a go. You never know, it might give perl a run for its money? (just joking)

Also, if you really do want to compare the overheads of perl and date, you might need a different benchmark: I was under the impression that "benchmarks" generally try to achieve the same result with the same algorithm. I could be wrong there too though.

In future, I'll just recommend people become perl wizards rather helping them expand their knowledge . That's worked well for everyone so far.



[ Reply to This | # ]