Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!


Click here to return to the 'Add timestamps to Unix commands that run at intervals' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
Add timestamps to Unix commands that run at intervals
Authored by: slvrstn on Jan 24, '10 01:32:13PM
OK, let's look at this extra overhead issue, because it's exactly what I was talking about in my original post when I mentioned the advantage of not shelling out to get the timestamp.

To benchmark, we want to generate a lot of data quickly, so I used

time od -N 1000000 -D /dev/random | wc
   62501  312501 4625008

real	0m0.310s
user	0m0.170s
sys	0m0.153s

to generate around 62k lines of the form

0000000         455208253      2432249641      1505621869      3979473917
0000020        2293418554      1733714563      3308048974       965145382
0000040        2395066143      3983717352      2757720288      3811753968
...

You see that it takes less than half a second to generate the data.

Then, comparing methodologies:

Perl

time od -N 1000000 -D /dev/random | perl -ane 'printf "%.2d:%.2d:%.2d: @F[0..4]\n", (localtime())[2,1,0]'  > /dev/null

real	0m0.869s
user	0m0.813s
sys	0m0.162s

bash

time od -N 1000000 -D /dev/random | while read a b c d e;do printf "%s %9s %9s %9s %9s %9s\n" $(date +%T) $a $b $c $d;done  > /dev/null

real	3m43.331s
user	0m52.922s
sys	2m27.505s

So, it takes less than a second with the pure Perl and over 3 1/2 minutes in pure bash.

Any approach that has to invoke a new command to get the timestamp each time through the loop is going to suffer by the same comparison.

[ Reply to This | # ]

Add timestamps to Unix commands that run at intervals
Authored by: Hal Itosis on Jan 24, '10 03:17:47PM
Holy Moly Perl-man... point well made, and taken. 8-)

How is it that Perl manages to get the current timestamp
for each iteration so much _more_ quickly? Doesn't it need
to request that info (in real time) from the OS as well?
[it also seems to parse the data strings faster too]

Anyway, thanks for that impressive proof.

[ Reply to This | # ]
Add timestamps to Unix commands that run at intervals
Authored by: slvrstn on Jan 24, '10 09:07:11PM

In the end, both perl and the `date` command are making the same system library call, time.
See `man -s 3 time`

The difference is all in the overhead in setting up the execution environment.
In the perl, this is done once and the interpreter can then make rapid-fire system calls.
In the bash (and awk) examples, every invocation of `date` has to exec a full new command / execution environment, which is comparatively extremely resource intensive.

bash is a fantastic and very capable shell, but unless your script is written entirely with built-ins, you are going to pay a performance penalty for every external command invoked. This isn't something exclusive to bash, of course. You have the same issues in Perl, in fact, if your script relies heavily on using the system, exec, or backtick commands.



[ Reply to This | # ]
Add timestamps to Unix commands that run at intervals
Authored by: bashbash on Jan 25, '10 12:28:53AM

Wow!

May I humbly suggest to give C a go. You never know, it might give perl a run for its money? (just joking)

Also, if you really do want to compare the overheads of perl and date, you might need a different benchmark: I was under the impression that "benchmarks" generally try to achieve the same result with the same algorithm. I could be wrong there too though.

In future, I'll just recommend people become perl wizards rather helping them expand their knowledge . That's worked well for everyone so far.



[ Reply to This | # ]