Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

10.4: Demystify Spotlight's saved search syntax System 10.4
Tiger only hintSpotlight is terrible. But at the same time, very useful. Let's hope Apple cleans it up a bit in a future release. But until then, here is some more help - to go along with the tons of Spotlight hints we already have here.

Spotlight saved searches, or "Smart Folders," are simply XML files with a .savedSearch extension. Within that XML file, which you can open using any text editor, there is a tag called RawQuery; immediately following that is another called (ambiguously) string. The string tag right after the RawQuery tag is the "code" for your search.

Since Spotlight limits what you can put in the query in various ways, sometimes it is hard to get what you want out of it. Boolean searches are well documented both on this site and others -- but they prove problematic and inconsistent. So this hint is intended to share some info about how to modify the "code" in the saved search to get what you want.

Let me start with a scenerio. I have a bunch of images in a folder named Images, and I want to mark some of them as Oregon. So I use the Batch Spotlight Comment action in Automator to mark them as Oregon in the Spotlight comments. Then they are easy enough to find using Spotlight; just type in Oregon and there they all are. But say I add another batch marked Mountains, and I want to find all images that are marked Mountains but not Oregon. Not so easy to do with Spotlight, at least as the interface allows. If I open a saved search for Oregon-coded pictures, I get this XML (line breaks in the string line are added for readability; there are none in the code):
<key>RawQuery</key>
<string>(* = "Oregon*"wcd || kMDItemTextContent = "Oregon*"cd) &amp;&amp;
(kMDItemContentType != com.apple.mail.emlx) &amp;&amp; 
(kMDItemContentType != public.vcard)</string>
Well, that code isn't too helpful. But Apple's Developer documentation on query syntax and what can be queried are good references. I will highlight the important stuff here for a simple overview, and in case they change the URL of the pages.

First of all, comments in the Finder -- which are called by the UI name "Spotlight Comments" -- are stored in the kMDItemFinderComment key. So to search for Oregon anywhere in the comments, here is the simplest form:
<key>RawQuery</key>
<string>(kMDItemFinderComment == "*oregon*"cd )</string>
The operators are:
  • == equal
  • != not equal
  • < less than
  • > greater than
  • <= less than or equal to
  • >= greater than or equal to
  • * is the wildcard:
    • Placed at the beginning, like *oregon, gets anything that ends with oregon.
    • Placed at the end, like oregon*, will get anything that starts with oregon.
    • Placed on both sides, like *oregon* will return anything with oregon anywhere.
And then there is that strange cd there at the end -- ... == "*oregon*"cd -- which kind of looks like a typo. That is not a typo, but rather specifies how the comparison is to be made:
  • w makes the comparison based on whole words
  • c makes the comparison not case sensitive (ignore case)
  • d makes the comparison diacritical insensitive. I had to look it up, but diacritical means "accents." So this would ignore any accent marks, I am assuming).
So to find anything that was Mountain but not Oregon, the query would look like this:
<key>RawQuery</key>
<string>(kMDItemFinderComment == "*mountain*"cd &amp;&amp; 
kMDItemFinderComment != "*oregon*"cd )</string>
Again, the line break in the string line was added for legibility; remove it when entering the text in the file.

This one example should show you how you can easily edit saved searches to make them do more of what you want. This was just a small example -- I am sure someone will comment with the way to do that search in plain Spotlight. But the intent is to show you how the XML for the savedSearch works, and how to develop your own queries using it...
    •    
  • Currently 3.00 / 5
  You rated: 1 / 5 (5 votes cast)
 
[19,971 views]  

10.4: Demystify Spotlight's saved search syntax | 21 comments | Create New Account
Click here to return to the '10.4: Demystify Spotlight's saved search syntax' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
10.4: Demystify Spotlight's saved search syntax
Authored by: AaronAdams on Oct 12, '05 07:56:28AM

What's so terrible about it?



[ Reply to This | # ]
Re: What's so terrible about it
Authored by: serres on Oct 12, '05 08:31:39AM

Because among other things

– Spotlight so slow, it's unusable. If I need to wait 30 sec min. for simple results like applications I can go there the usual way. Or opening a smart folder containing all applications takes forever here.

– Spotlight very unreliable. You cannot know which ones of your documents get indexed. Spotlight is more like a "Here are some possible related results"-engine.

For the moment you should neither rely on it nor trust it.



[ Reply to This | # ]
Re: What's so terrible about it
Authored by: statefuldotnet on Oct 12, '05 08:47:46AM
– Spotlight so slow, it's unusable. If I need to wait 30 sec min. for simple results like applications I can go there the usual way. Or opening a smart folder containing all applications takes forever here.
Yeah I will admit the search as you type is ridiculous if you fat finger your search. There should be an obtion to turn this capability on or off.
– Spotlight very unreliable. You cannot know which ones of your documents get indexed. Spotlight is more like a "Here are some possible related results"-engine.
Nah this comes from not understanding Spotlight. Not only can you refine where Spotlight indexes, you can FORCE index files, and you can tell exactly what will be indexed with the mdimporter plug-ins. You should investigate a tool called Highlight if you don't grok what you can do with the command line tools. Although I agree they could make configuration of this easier for the masses, the capability is certainly there and is not a mark against Spotlight.
For the moment you should neither rely on it nor trust it.
You should learn more about the technology before making such claims. The comments made show you do not understand how Spotlight works other then the magical upper right hand corner thingie that produces search results from who knows where and isn't seeming to find documents you know you have.

[ Reply to This | # ]
Re: What's so terrible about it
Authored by: osxpounder on Oct 12, '05 03:22:04PM

In my opinion, if you must "understand Spotlight" or "learn more about the technology" just to search for files, then Spotlight fails its mission. It was supposed to make finding things quicker and easier. We shouldn't have to do research to find out how it works just so we can find our files. I find the criticisms of Spotlight valid.

---
--
osxpounder



[ Reply to This | # ]
Re: What's so terrible about it
Authored by: AaronAdams on Oct 12, '05 09:09:00AM
– Spotlight so slow, it's unusable. If I need to wait 30 sec min. for simple results like applications I can go there the usual way. Or opening a smart folder containing all applications takes forever here.
And now I will present my anecdotal evidence to counter your anecdotal evidence: Spotlight is fast and completely usable. I haven't seen a Mac yet where Spotlight wasn't quick and reactive. You obviously have some kind of problem with your system.
– Spotlight very unreliable. You cannot know which ones of your documents get indexed. Spotlight is more like a "Here are some possible related results"-engine.
More anecdotal evidence. I've found Spotlight to be completely reliable and it does a very good job indexing all my documents. Again, it sounds like you have a larger problem.
For the moment you should neither rely on it nor trust it.
I neither rely on or trust your assessment. Before I'll consider Spotlight "terrible", as the article author asserts without any supporting facts whatsoever, I need demonstrable, reproducible, verifiable reasons why. Anecdotes that amount to "It doesn't work right for me in particular" and "It doesn't behave in some arbitrary way that *I* think it should" are not good supporting arguments for Spotlight being "terrible". I really wish those kinds of editorial comments would be edited out of the hints. They're irritating and they detract from the value of the hint.

[ Reply to This | # ]
Re: What's so terrible about it
Authored by: ocdinsomniac on Oct 12, '05 09:25:57AM

I disagree. Spotlight is not a tool that should require knowledge of the command-line, or third-party utilities, in order to be used or useful. There are a number of ways in which Spotlight can fail to work or work improperly for no reason that is obvious to the ordinary user: a Spotlight index, for example, can become corrupt, returning incomplete results, if the indexing process is interrupted, with no warning to the user, and no recourse except a trip to the command-line or a third-party application. This can happen if a firewire drive is ejected or your computer reboots during the indexing process. Spotlight has no error correction in these instances, and must be forced to re-index. But the user is none the wiser when these problems occur. Spotlight just continues to use the incomplete index and return inaccurate results.

Spotlight should "just work" for most people, in most instances, and for many it does not. I've experienced these problems myself in many contexts, and I know all about the command-line tools, and the third-party apps. These issues are documented all over the web. There are many power users who are frustrated with Spotlight. Just because you are not one of them (and there are a number of reasons why that might be) does not make it untrue.

It is, indeed, a "mark against Spotlight" that it can't work properly without constant supervision. So maybe you should learn a bit about the limitations of a technology before you take a condescending attitude towards others who are having real, documented problems.



[ Reply to This | # ]
Re: What's so terrible about it
Authored by: AaronAdams on Oct 12, '05 01:54:55PM
I disagree. Spotlight is not a tool that should require knowledge of the command-line, or third-party utilities, in order to be used or useful.
It doesn't. The problems you've described here don't require any kind of command line intervention. I agree that unfinished or corrupted indexes are a problem, and those errors should be handled better, but they don't make Spotlight "terrible" and they don't require the command line to fix. Indexes can be removed and rebuilt via the GUI.

http://docs.info.apple.com/article.html?artnum=301562

Spotlight should "just work" for most people, in most instances,
It does.
and for many it does not.
And for most it does.
There are many power users who are frustrated with Spotlight. Just because you are not one of them (and there are a number of reasons why that might be) does not make it untrue.
Again, because some user(s) (power users in this case) want something different than what Spotlight provides doesn't make Spotlight "terrible". It makes it possibly not useful to some people, but not "terrible". I've never asserted that anyone's problems were untrue. I've stated that just because some users have problems doesn't mean that Spotlight is "terrible". The author of the original hint states "Spotlight is terrible" as his first line of the story without any supporting evidence and questionable relevance to the hint.
It is, indeed, a "mark against Spotlight" that it can't work properly without constant supervision.
Yes it can, and it does.
So maybe you should learn a bit about the limitations of a technology before you take a condescending attitude towards others who are having real, documented problems.
I know plenty about the limitations of technology, and I know plenty about the limitations of users. I'm asking for some kind of reproducible and objectively verified set of problems with Spotlight that rise to the definition of "terrible". You've explained a problem here - Spotlight's indexing behavior. While I agree that it certainly is an issue, that single issue is far away from making the entire scope of Spotlight technology "terrible".

[ Reply to This | # ]
Spotlight still has a ways to go...
Authored by: Hal Itosis on Oct 12, '05 02:10:26PM

Hey Aaron,

Can you stop cluttering this tip with useless chatter?
(Start a thread in the forums if you want to preach).

Please? Thanks.

-HI-



[ Reply to This | # ]
Spotlight still has a ways to go...
Authored by: dave1212 on Oct 14, '05 02:09:28AM

you must have meant to post a reply to the parent, and got the name wrong too. Opinionated bashing of a technology is pointless in the articles provided here, and on top of wasting reading time, provokes others to respond in kind.

Yes, discussions about relevant issues should be brought to the forum, but when deliberate misinformation is posted in the comments (or in the article) here it either needs to be corrected or edited. I doubt the strongly opinionated persons in this thread (anti-Spotlight, for voodoo reasons, won't listen to others' opinions) are about to comment in the forum, since it would seem to lessen the impact of their statements compared to making them here on the main site.

Perhaps the machines in question have other issues. It's too easy to blame on Spotlight. Ignoring the pointless bias in saying 'Spotlight is terrible' , this hint is quite useful overall, it will be nice to have all that is listed on hand for quick editing.



[ Reply to This | # ]
Spotlight still has a ways to go...
Authored by: raider on Oct 14, '05 10:57:47AM
I am the original author, and here I will defend my statements. First of all, you and Aaron seem to have not got past the "Spotlight is terrible." and never made it to the "But at the same time, very useful. Let's hope Apple cleans it up a bit in a future release."

Spotlight *is* terrible. For crying out loud, there are 700 hints on this site alone trying to make it work better. Search the web and you will find thousands of people talking about the problems they have with spotlight.

An Apple product should be cleaner, simpler, and more consistant. PERIOD.

Some people claim there is no problem with speed. While *I* have not experienced any problems with speed, it would be silly for me to claim that there are no problems, simply because of the massive numbers of people who claim otherwise. This is not like the "iPod battery" issues where there were tiny numbers of people speaking loudly. This is a good 25% of the spotlight discussions revolve around people having issues with it's performance. This area needs improvement.

Spotlight's interface is terrible. It just simply is. It is inconsistant, acting one way in the Finder, a different way in Mail, and yet another way in the Spotlight app itself. And to get it to do what you want is undocumented and confusing. It just is. Maybe YOU don't find it confusing, but when you look around and tons of people are having a hard time getting it to work right or find what they want - that is enough evidence to say Apple can do better. Did you see the special presentation for the new iMac and iPod with video? Steve puts up the slide showing the Microsoft media center remote control with 50 buttons and compares it to the FrontRow remote control with 6 buttons, and he says "This slide more than anything shows what Apple is all about. Well my friends - Spotlight is like the 50 button remote, and half the buttons are not labled and the other half don't do what they say...

And finally - Spotlight results are inconsistant. I don't know why - but it needs to be fixed. I can get different search results on the same search two times in a row... I have a smart folder that I set up. It found all the images I wanted it to find. Then I added two more to the source folder, and tagged them appropriately - and the smart folder will NOT find them. I have forced a re-index of the source folder. I have restarted Finder, and restarted the machine. The smart folder will not "see" the two that I added. They have the exact same comments as the other images that the smart folder finds fine. In fact I even tried copying the comments directly from a working image into the others, and no luck. WHY THE HELL IS THIS? Is this a "smart" folder? No. This is a pain in the ass.

Spotlight has a hell of a lot of potential. It does. And it curently *can* be used well, as many of the hints here help out. But Absolutely Apple needs to improve it..

Something Apple touts as being so great - should be much more polished than Spotlight.

[ Reply to This | # ]
Re: What's so terrible about it
Authored by: MrSandman on Oct 13, '05 01:13:04AM

This is strange. Spotlight is INSTANTANEOUS on all my Macs, for all types of applications.

So far, spotlight has been the raw model for reliability. It *always* finds what I am looking for.



[ Reply to This | # ]
Re: What's so terrible about it
Authored by: serres on Nov 02, '05 10:46:56AM

As of the Mac OS X 10.4.3 spotlight is no longer slow. I'm very pleased. Spotlight is now usable for me and it's great.



[ Reply to This | # ]
10.4: Demystify Spotlight's saved search syntax
Authored by: statefuldotnet on Oct 12, '05 08:38:31AM

What's wrong with just using Raw Query in a Finder saved search? Seems like less effort and you don't stand the chance of accidently corrupting your xml by hacking it with a text editor, not to mention having to do silly things like escaping &&'s.

Open Finder, New Search, Change "Kind" to "Others...", select Raw Query and enter your spotlight query like before. I'm sure this is already covered in a previous hint and short of describing the anatomy of a savedsearch file is considerably more practical then hacking at raw xml.



[ Reply to This | # ]
10.4: Demystify Spotlight's saved search syntax
Authored by: raider on Oct 12, '05 10:33:01AM
WOW!

You are absolutely correct!

I thank you for pointing this out!

Anything that you can put in the RawQuery field in the XML file can be used in the "Raw Query" field in the saved search parameters.

NICE!

This is actually discussed in this hint, but when I first read through that I thought they were talking about command line searches, and missed the whole point I guess. So my bad.....

But these links are still helpful:

Spotlight's query expresion syntax.

Spotlight Metadata Attributes

But hey, I just added to the pile of Spotlight information. :)

Thanks again!

[ Reply to This | # ]
10.4: Demystify Spotlight's saved search syntax
Authored by: aMacUzur on Oct 12, '05 12:28:03PM
I'd like the ability to reliably search all the folders 'n files on my disk volume(s) by name (only) and have folder/file information shown that includes the pathname of the folder/file ... just like in the (good ol') pre-Spotlight days.

Except for the fact that the old CMD-F/Find capability didn't save it's window size/position, it was just fine, for me. In 10.4, I don't see any way to ensure that all folders/files are searched (by name) and to get the pathname information shown (without having to click on each entry).

If someone knows how to configure Spotlight to search all folders/files, "enquiring minds" would like to know how.

In addition, if someone knows how to configure Spotlight to show the pathname of the items (without having to click on each entry), I'd like to know (as would others I've talked with).

Currently, I'm stuck using the (much slower, but priced-right) EasyFind.

[ Reply to This | # ]
10.4: Demystify Spotlight's saved search syntax
Authored by: GEllenburg on Oct 13, '05 09:36:22AM

There is the "locate" command. The index is updated via cron (nightly I think, might be weekly) but it's infinitely more faster than Spotlight for certain queries.

Of course, this requires a trip to the Terminal, but you know... the terminal (rarely) hurt anyone.

---
He who does not challenge one\'s elected decisions today, may find himself challenged by them tomorrow.



[ Reply to This | # ]
There must be different Sportight versions around
Authored by: RobertBen on Oct 12, '05 03:33:02PM

Though I haven't been happy with some of the interface and default choices Apple made with this version of Spotlight, I still prefere it immensely over the older Find File.
It's fast, easy to use and it simply finds much more, much faster.

For more extensive searches (like within the System Folder or on external disks that aren't indexed) I use Easyfind.

Since Rob has been complaining about Spotlight's speed as well, my guess is that Spotlight isn't as good on each Mac.

But it performs flawlessly on my Cube (800MHz) and my 500 MHz Powerbook. (both different Tiger installs, so no clones)



[ Reply to This | # ]
10.4: Demystify Spotlight's saved search syntax
Authored by: gaoshan on Oct 12, '05 06:22:50PM

Given what I have used Spotlight for, I find Quicksilver to be a far superior solution. It finds what I need faster than Spotlight, is more customizable and just plain does more.

A good example... I want to launch Photoshop (which, for whatever reason, is not in the dock).

Quicksilver:

  1. Hit key command to launch it
  2. Start typing "Photoshop"
  3. By the time I hit "o" Quicksilver has Photoshop selected
  4. Hit the enter key to launch
Spotlight:
  1. Hit key command to launch it
  2. Start typing "Photoshop"
  3. Spotlight jumps around the results as I type, finally offering Photoshop after I have typed the entire word
  4. Unfortunately, hitting the enter key will launch an obscure file called "photoshop9-EC-RET.csa". I must arrow down the list to get Photoshop
  5. Now I can launch it
No contest in this sort of situation.

Now, how about trying to find and launch an obscure, little used text file about nmap fingerprinting?

Quicksilver requires typing the word "nmap", hitting the right arrow key (which expands the folder presented by QS), and pressing enter to launch the file.

Spotlight requires the same steps except that I had to hit the arrow down key 8 times to get to the desired file.

Once again, not much of a contest. Spotlight really only gets accessed on my computer by accident, anymore.



[ Reply to This | # ]
10.4: Demystify Spotlight's saved search syntax
Authored by: statefuldotnet on Oct 13, '05 09:55:01AM
You bring up a good point, but I use both Quicksilver and Spotlight as a daily part of my workflow and couldn't do without either of them.
Now, how about trying to find and launch an obscure, little used text file about nmap fingerprinting? Quicksilver requires typing the word "nmap", hitting the right arrow key (which expands the folder presented by QS), and pressing enter to launch the file.
Quicksilver would never find it though if nmap was text inside a pdf and that was really the document you had in mind -- Spotlight would. It is a critical event to consider -- Quicksilver has no concept of metadata, but it is an excellent portal for handing off content to Spotlight.service if you only one a single consistent interface. You are right in terms of the file launcher capability, but that's not a driver for Spotlight, it can do it, but that's not the purpose of it. I find the technologies complementary at best. Spotlight to index data, and Quicksilver to index applications and perform actions. Also if you hate the search as you type in Spotlight just use Quicksilver to invoke your spotlight search from text you type in after hitting the '.' key. If you restrict both technologies to not overlap each other you will find that both Spotlight and Quicksilver are MUCH faster then they are by default. Be sure and use the Privacy settings to your advantage too -- if you know you'll never search for anything in a directory that Spotlight seems to be pulling hits from, immediately block it. Also tune down the types of data you want Spotlight to return... maybe you don't care about it pulling up fonts, movies, music, system preferences, etc -- and you're really only concerned with email, pdf, presentations, etc. Tuning this also significantly speeds up Spotlight. I actually only use 5 of the 14 items, because Quicksilver handles the some of them better (like contacts and bookmarks for example).

[ Reply to This | # ]
10.4: Demystify Spotlight's saved search syntax
Authored by: stonematt on Oct 13, '05 07:49:23PM

I use LaunchBar (http://www.obdev.at/products/launchbar/) to do basic application launching and search by filename. It actually predicts/learns shortcuts, so I can type "pho" to launch iPhoto and "ps" to launch Photoshop, etc.

WAY faster than spotlight, Spotlight isn't indexing some of my folders and I can't figure out how to tell it to, but occasionally it works for me. It's great as the search engine inside Mail. I seach mail all the time and it's very good.

-stone



[ Reply to This | # ]
yes!
Authored by: sgi_oh_too on Oct 13, '05 01:44:46AM

Spotlight is terrible for sure. The worst part is that you can't turn it off! At least the indexing in OS 9 had a "stop indexing" prompt, even though that was annoying every time it popped up. Searching is something that I never really do, and I hate to think of wasting cycles or space maintaining indexes of all my files. If it is a 'dummy' feature, it should be on by default as it is ... however for us 'non-dummies' that can actually maintain an organized disk, we should be able to turn it off! (and i am not talking about some unsupported hack to turn it off either! just a toggle in the system preferences that says "spotlight: off/on")



[ Reply to This | # ]