Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

grep faster and more effectively than grep, using grep UNIX
I see many hints here on Mac OS X Hints that use the grep utility to filter text. However, many of these hints use grep to search for a plain string, rather than a regular expression. There's nothing wrong with that goal, but using grep for that is inefficient (you're invoking the full power of the regex engine just for a plain string), and requires that you escape any regex characters in the pattern:

 grep 'Price: $' partslist.txt
 # Incorrect: Only matches "Price: " at the end of a line

 grep 'Price: $' partslist.txt
 # Correct: Matches "Price: " followed by a dollar sign
There's a better way. grep also exists under the name fgrep, which searches for a fixed (hence the 'f') string -- that is, a plain string; no characters are special. You can search for any string this way, with no escaping needed (see note below):
 fgrep 'Price: $' partslist.txt 
 # Correct: Matches "Price: " followed by a dollar sign
You can also do this with grep -F, but fgrep is shorter. fgrep and grep are the same program, just with different behavior, so all the other options (-n, -o, -H, etc.) are supported by both.

Note: Of course, you may need to escape quotes or backslashes according to the rules of your shell, but you would have to do this anyway. My point is that fgrep does not require quoting where grep itself would.
    •    
  • Currently 1.60 / 5
  You rated: 2 / 5 (5 votes cast)
 
[20,117 views]  

grep faster and more effectively than grep, using grep | 12 comments | Create New Account
Click here to return to the 'grep faster and more effectively than grep, using grep' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
grep faster and more effectively than grep, using grep
Authored by: Coumerelli on May 24, '07 08:18:58AM
am I missing something? What's the difference between the 'correct' and 'incorrect' examples?
grep 'Price: $' partslist.txt
vs
grep 'Price: $' partslist.txt

---
"The best way to accelerate a PC is 9.8 m/s2"

[ Reply to This | # ]

grep faster and more effectively than grep, using grep
Authored by: wallybear on May 24, '07 08:30:35AM

the difference was eaten by geeklog, I think. The second command should read like this:


grep 'Price: \$' partslist.txt
# Correct: Matches "Price: " followed by a dollar sign



[ Reply to This | # ]
grep faster and more effectively than grep, using grep
Authored by: rockmanac on May 24, '07 09:05:37AM

Is this supposed to be:

[code]
grep 'Price: $' partslist.txt
# Incorrect: Only matches "Price: " at the end of a line

grep 'Price: \$' partslist.txt
# Correct: Matches "Price: " followed by a dollar sign
[/code]
?

-A

---
"Communication is an ongoing, dynamic process between an infinite
number of variables with no beginning, middle or end."
-- Dr. Ken Ksobiech, Marquette



[ Reply to This | # ]
grep faster and more effectively than grep, using grep
Authored by: boredzo on May 24, '07 09:12:45AM

The backslash was there when I submitted it (I even escaped it with an HTML entity reference); apparently, it got eaten afterward.



[ Reply to This | # ]
grep faster and more effectively than grep, using grep
Authored by: joshewah on May 24, '07 09:13:33AM

fgrep is just grep that doesn't parse any regular expressions. So that is why you don't have to escape any of the characters that would be part of a regular expression. Theres also egrep, which works with extended regular expressions if you want to have even more regular expression power than the regular grep. So I guess if all you ever want to do is search for exact phrases then fgrep is fine for you.



[ Reply to This | # ]
grep faster and more effectively than grep, using grep
Authored by: jefftang on May 24, '07 09:55:06AM

fgrep and grep are actually the exact same program. They are hard links to the same file. Their behavior is determined by the name they're called by.

For fixed string searches their speed is just about the same as far as I can tell. The only good reason to use fgrep is if you have special regexp characters that you don't want to escape.



[ Reply to This | # ]
grep faster and more effectively than grep, using grep
Authored by: wheeles on May 24, '07 11:23:19AM

I always thought that egrep was faster than both grep and fgrep.



[ Reply to This | # ]
grep faster and more effectively than grep, using grep
Authored by: sweth on May 24, '07 07:47:26PM

1) I'm a Unix geek, so I don't have any anti-Unix-hint prejudice, but this is so not a Mac OS X hint that it's kind of scary.

2) If you're going to submit a Unix geek hint, at least make it a legit one. Anyone who is using grep in any of the hints on this site is almost definitely using it in an interactive setting, and any Unix geek worth their salt would know that you shouldn't optimize until performance actually becomes an issue, and in an interactive invocation of grep, the overhead of spawing the new process more than outweighs the overhead of the regex match so the "optimization" doesn't buy you anything of value.

3) If performance really does matter, again, spawning the grep process is going to be the big hit; you can avoid that and usually get much better performance by using shell constructs to do the fixed-string match: echo "foo" | fgrep "oo" will almost always be slower than echo "foo" | while read LINE ; do case $LINE in "oo" ) echo $LINE ; esac ; done. But, again, that's almost never a necessary optimization unless you're looping over that statement hundreds of times, and then you should be using a more robust tool like perl anyways.



[ Reply to This | # ]
grep faster and more effectively than grep, using grep
Authored by: dzurn on May 25, '07 07:17:29AM

Mac OS X has *nix underpinnings, so Unix hints are incredibly valuable for some tasks. I'm using the hints on this site as a Mac (power) user who wants to harness what these tools can provide. Yes, we could obtain and read the entire BASH Reference Manual or scour through the scarcely-readable "sh(1)" man page, but *nix is not something we use for hours on end and have hardwired into our fingertips.

Second, your "faster" example doesn't work for me. I tried it in sh, csh, zsh and it never returns what is expected. The first example works and is much easier to understand, and so might get used in the future.

The whole reason for the hint was to show a way to avoid remembering and escaping all the regex characters when we just want to match a bit of text in some other task.

Please don't denigrate this hint on the basis that it's not fast enough. You can however go ahead and denigrate us because we have only so much time to deal with learning and remembering the huge body of *nix tools.

Mac OS X is a tool to get something else done, it's not an end in itself. Learning all the *nix conventions and optimizations is not first on my list of things to do.

---
Madness takes its toll.
Please have exact change.



[ Reply to This | # ]
grep faster and more effectively than grep, using grep
Authored by: sweth on May 25, '07 06:04:19PM

You need to learn to read better (which might explain why manpages are so baffling to you). I never said that the hint wasn't fast enough; I said that it was pointless, because the speed of an interactive invocation of grep is very rarely going to be functionally helped by the switch to fgrep, and if speed really were an issue, using fgrep would rarely be the best solution. I've seen literally hundreds of people over the years get caught up in trying to optimize things that don't need to be optimized, and in ways that don't really help them; all I was trying to do was prevent people from wasting their time on what is almost always a fruitless endeavor.

(I'll concede that for people not familiar with regular expressions, the ability to use fixed strings instead of regexes could make fgrep useful, but that's not what the hint claims to be showing people to do. And judging from the current rating of the hint, it would seem that I'm not alone in not thinking that this is not a great hint.)



[ Reply to This | # ]
grep faster and more effectively than grep, using grep
Authored by: baba on May 26, '07 05:17:13PM

dzurn is spot on.
And especially don't denigrate the hint using Valley Girl English: "...this is so not a Mac OS X hint..."

Heh. Do you happen to remember that line from The Last Emperor where Peter O'Toole's characater states "A gentleman always means what he says, and says what he means."? Once you get beyond the anachronism (by today's standards) that he pronounces the 'h' in 'what,' you can see he has a point.

Alack, maybe I am the anachronism here ....



[ Reply to This | # ]
check out egrep
Authored by: macubergeek on May 25, '07 04:59:32PM

egrep lets you search for multiple patterns

egrep 'patern1|pattern2' my.txt



[ Reply to This | # ]