Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

10.4: Improve Spotlight's search speed System 10.4
Tiger only hintIf you are using Spotlight as a Finder search replacement, you've seen that is fast, but not fast enough. To optimize Spotlight search times, I've tried to renice the process that's behind the searches. Looking at the CPU usage in the Terminal (via top -o cpu) during searches, I've seen that a process named mds is the one that does it all. Using an AppleScript to renice the mds process, we can cut research times even at half.

Here's the script:
tell application "Terminal"
  do shell script 
   "sudo renice -20 -p `ps -axww | grep '[/]mds'| awk '{print $1}'`"
  quit "terminal"
end tell
Save the script as an application, and using the Login Items tab of the Accounts System Preferences panel, list it as a startup file. Now you have to logout and re-login. You'll se a Terminal window appear and disappear soon after. To check if the renice command has been successful, you can open the Terminal and type:
ps -axww -l nice | grep '[/]mds'
and check if the sixth field is -20.

[robg adds: I'm running this mainly to see what others' experiences are with it. I tried it, and I must admit, I don't really see any increase in the speed of Spotlight's results (which is another of my pet peeves with it -- for a supposedly indexed system, it's astonishingly slow at what seems like simple stuff). The rest of this hint details how to allow the renice process to run via sudo without requiring a password. I personally wouldn't do that, as I much prefer the minor inconvenience of typing the password. However, it's here in the interest of completeness...]

The renice command has to be given as "root," and to have it work without giving an admin password every time, you have to modify the /etc/sudoers file. To do that, open the Terminal and type:
sudo visudo
At some place in the file, add the following line:
%admin ALL = NOPASSWD: /usr/bin/renice
That let's any admin users run the renice command without giving their admin passwords. Save and that's done.
    •    
  • Currently 1.33 / 5
  • 1
  • 2
  • 3
  • 4
  • 5
  (3 votes cast)
 
[24,610 views]  

10.4: Improve Spotlight's search speed | 19 comments | Create New Account
Click here to return to the '10.4: Improve Spotlight's search speed' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
10.4: Improve Spotlight's search speed
Authored by: jeffreym on May 10, '05 10:56:28AM

From past experience, the 'feature' that tends to impact speed the most with lists such as this is the displaying of icons. Since Spotlight insists on displaying each custom icon, this will slow down the process in direct proportion to a system's graphics display capabilities. So, for example, it seems to be quite fast on my dual 1.8ghz G5 with 2.2GB of ram and ATI9800pro, but significantly slower on my 1Ghz PowerBook.



[ Reply to This | # ]
10.4: Improve Spotlight's search speed
Authored by: Cameroon on May 10, '05 03:53:59PM

Not just the graphics system, but also the hard drive. It has to load those icons from disk. Even if it caches those icons with the index entry, it still has to load that information from disk.

Turning off the icon would be a nice option, but personally I when I'm in a folder, I want Spotlight to default to searching only that folder. I don't care about how many matches there are across my hard drive when I'm in a particular folder and using the search toolbar item.



[ Reply to This | # ]
10.4: Improve Spotlight's search speed
Authored by: xSmurf on May 11, '05 12:18:35AM
Yup I totally agree, on top of this I when I search from the finder the result is always the same... bogus! I search a folder in which I know there would be tons of results for my search and after a few mins it's still searching. Although, I admit not having searched the net yet for this issue.

---
PM G4 DP 800 / 1.25gb / 120Gb+80Gb / CD/DVD±RW/RAM/DL
- The only APP Smurf ;P

[ Reply to This | # ]

10.4: Improve Spotlight's search speed
Authored by: diamondsw on May 10, '05 11:21:02AM

This really surprises me - I have 4 volumes indexed (about 400GB of data) and on my iBook G3 800Mhz, results come up in under a second, with icons filling in as they show up (they no longer delay the display of the list).

Speed is the *last* problem I have with Spotlight.



[ Reply to This | # ]
10.4: Improve Spotlight's search speed
Authored by: cynikal on May 10, '05 11:22:51AM

I would advise against this.. the mds process appears to be responsible for being handed off to from the kernel hooks when a file is written (or opened for indexing)... originally this process has a positive priority value (meaning it takes a back seat to other processes if there's resource contention)... apple must have done this for a reason, probably so you don't notice it when it is indexing new files (new meaning files not already in the index).

As far as i can tell, when i am creating new files (or when i just installed my system), this process was using the most cpu indicating it's merely responsible for indexing non-indexed files. i don't think it has anything to do with the search results, since that is reading from the index and this is not what that process is responsible for.



[ Reply to This | # ]
10.4: Improve Spotlight's search speed
Authored by: cynikal on May 10, '05 11:25:33AM

or if you insist on trying this, at least don't give it a -20, that's the highest priority available..



[ Reply to This | # ]
10.4: Improve Spotlight's search speed
Authored by: stephendv on May 10, '05 01:31:34PM

The mds process does appear to be responsible for spotlight searches, just watch the CPU usage on the process when running a search - it really does skyrocket.



[ Reply to This | # ]
10.4: Improve Spotlight's search speed
Authored by: bigbold on May 10, '05 11:44:45AM

How amusing! I've done the same thing, but in the OPPOSITE direction! mds is a nightmare. It seems to want to balloon up to using 65MB of memory sometimes, and I have to killall -HUP mds, which knocks it back down to single digit land for a while. Then I give up and renice it up ;-) Still, I now want to find a way to disable this nonsense since I'm sick of mds, mdimport, and friends taking up my CPU time and memory space. Tiger is so memory hungry it's nuts.



[ Reply to This | # ]
10.4: Improve Spotlight's search speed
Authored by: cynikal on May 10, '05 12:02:29PM

Not to mention I/O.. when mds kicks up, it will slow your hard drive reading/writing to a halt as it churns its way through your files..

Shame that a new user's first few hours of using tiger, they will get the impression that tiger is so slow as a result of mds



[ Reply to This | # ]
10.4: Improve Spotlight's search speed
Authored by: bigbold on May 10, '05 04:27:41PM

I've been using Tiger for just over a week now and mds still usually ends up at 64MB or so each day. If I HUP it, it sits at 5MB for a while, then climbs back up.

There's certainly way more paging on in Tiger than ever before (which is probably why it feels so awfully sluggish). I bought 512MB of memory which made Panther totally fly.. now it seems I'll have to buy a 1GB stick to get Tiger to do the same :-(

I'm a developer and am always untarring and compiling bits and pieces. I wonder if mds is getting tied up keeping up with all the files I'm creating. If so, perhaps I can set exceptions on my temporary and development folders to cure the problem..



[ Reply to This | # ]
10.4: Improve Spotlight's search speed
Authored by: sjmills on May 10, '05 07:56:39PM

Since Spotlight doesn't even look at all the data files created/modified during a compile/build, it shouldn't being doing much. It will probably only scan the newly built app if that's what you're building.



[ Reply to This | # ]
10.4: Improve Spotlight's search speed
Authored by: sveinbjornt on May 10, '05 05:51:29PM

Yeah, TIger is pretty memory hungry. The trick is to close all dashboard widgets and never open the bloody thing. Saves you about 60 megs.



[ Reply to This | # ]
10.4: Improve Spotlight's search speed
Authored by: Synchro on May 11, '05 05:57:11AM

This is causing me trouble too. I'm on a dual G4/867 with 1.25Gb RAM, and the mdimport process is consuming 65Mb AND 65% of CPU all the time. It's been running for 7 days solid now and is credited with 124 hours of CPU usage - surely enough to index ~300Gb of files - I'm not introducing large numbers of files to my system. I just HUPd it and it dropped off the radar, but mds has now climbed to 75Mb and 75% CPU.

I'm not wildly impressed with spotlight in mail - it doesn't seem any faster than the old method.



[ Reply to This | # ]
10.4: Improve Spotlight's search speed
Authored by: raider on May 10, '05 12:34:35PM
Before messing with this, you might wat to read this part of the Arstechnica review of Tiger.

It explains how Spotlight actually works deep down, and might explain why users are experiencing what they are... Also, the whole review is GREAT and has a load of in depth technical information about Tiger.

[ Reply to This | # ]
10.4: Improve Spotlight's search speed
Authored by: mistercow on May 10, '05 06:33:49PM

Renice is overrated, although less so than it used to be. Evidence suggests it was entirely broken in pre-Tiger OS X. Even though it works now, renice is not really an effective way to speed things up.

Here's an illustrative example:
Make sure nothing processor intensive is running.
To max out CPU usage you can use the "yes" command in the terminal.

[code]yes x > /dev/null[\code]

That will write the letter x to nowhere as fast as the CPU can. Now look at the yes process in Activity Monitor. What % CPU does it have? Should be around 88-92%.

Now look at its PID. Suppose it's 650. Then in the terminal type
[code]sudo renice -20 650[/code]

Look at Activity Monitor. Has the % CPU changed? Of course not. The CPU time used was already as high as it could be, reserving some time for the system.

The only way you can see any effect from renice is if something else is using a lot of CPU.
If you open up two terminal windows and run yes, you can quickly see that renicing one to -20 gives it a much higher share of CPU time. But it's not going to help when nothing else is taking CPU time.

So if you are having trouble with Spotlight being too slow, for example, renice is rarely going to help. Indeed, the only time it will help you is if the lag experienced with Spotlight is due to something else eating your CPU time.

An apparently common misconception is that most of your CPU isn't being used because of programs' default priority. In OS X, programs already use exactly as much CPU as they need. The only time they use less is when there isn't enough to go around. This is rare.

In short, you aren't going to squeeze any more power out of your CPU than it has to give. If you give Spotlight high priority, you may get it another 10 percent of the CPU. And if Spotlight were relying more on the CPU than say, the hard drive, or any of a dozen other factors, that 10% might even save you a tenth of a second for every hour of use. But practically speaking you're only going to notice a difference because of the placebo effect.

Apple's OS really is using as much of your CPU as it can. Really. Honest.



[ Reply to This | # ]
10.4: Improve Spotlight's search speed
Authored by: mayo2ca on May 10, '05 09:56:58PM

What I've noticed is that diabling "put hard disks to sleep when possible" option in the Energy Saver preferences helps quite a bit too, at least on my powerbook and g5. (I would recommend against it if you want to conserve battery power on your PowerBook, otherwise turn it off)

mayo



[ Reply to This | # ]
10.4: Improve Spotlight's search speed
Authored by: stephendv on May 11, '05 08:28:08AM

How about moving the Spotlight index files to a RAM disk? Would this speed things up?



[ Reply to This | # ]
10.4: Improve Spotlight's search speed
Authored by: johnsawyercjs on Jan 24, '09 04:24:09AM

It's not really possible (in a practical fashion, anyway) to put the Spotlight index files into a RAMdisk. The index files for each volume are stored on that volume, so if you had more than one volume, you'd have to either have multiple RAMdisks (one for each volume), or some clever coding to change this behavior and store them all in a single RAMdisk (not to mention the coding necessary to get OS X to put the indexes onto a RAMdisk in the first place, even for a single volume).

Even if putting the Spotlight index files onto a RAMdisk were possible, it would be a bad idea for most people, since these index files would be rebuilt every time you restarted your Mac (since restarting wipes the contents of the RAMdisk). I know some people manage to keep their Macs running for days, weeks, or longer between restarts, so maybe they'd find this useful.

But I suspect that the speed issue behind the Spotlight index update process isn't so much due to writing the index file's updates, but due to reading from the volume it's indexing--when I use Activity Monitor to watch activity during Spotlight indexing, I see far more reads than writes.



[ Reply to This | # ]
10.4: Improve Spotlight's search speed
Authored by: adams4 on May 11, '05 10:36:16AM

This is a reply, actually, to Rob's comment that, "It's astonishingly slow at what seems like simple stuff." I would think that the searches that Spotlight does are not simple at all, even though the interface we use makes it seem that way.

There's two things going on (that I can tell, given my experience with databases and indexing). The first is that the operating system itself (i.e., the executables responsible for doing the Spotlight searching and displaying those results) has to balance out when it searches and when it displays. For example, if I searched for the word wyoming (hail to thee, Phil Schiller!), if I'm a slow typist, the system might start searching on "wyo" then "wyomi" then "wyoming," paring down the search results as I type. That's a penalty we exact for doing an interactive search. If the system were designed to either have a longer delay before a search started (so you can finish typing) or if you had to always hit Return after a search, then the search itself would appear faster, since the system doesn't have to eliminate results as it displayed them if you keep typing. The rotten thing is most people would probably balk at such a thing as hitting an extra key, when they want to start seeing search results without any other steps.

The other reason searches bog down is because the system is always (and only) doing a keyword search. Using the example (looking for all things "wyoming"), the system has to pull up the most relavent results for that keyword in all the file types it handles, which might mean looking at up to 14 different kinds of files (and therefore, 14 indexes). If there's just a single index, that makes it worse, since the OS must then go back and forth in that single file to obtain the results. In contrast, a much more rapid search is a browse search, where your search "lands" on a term in a specific index (or indexes, if so desired). But, of course, this would detract from the simplicity of doing a keyword search.

Take a look at any library system's online public access catalog (OPAC) to get a better idea of what I mean.

---
Adam Spector.

[ Reply to This | # ]