For some strange reason, my Finder sidebar started to add aliases by itself. Sometimes it would duplicate multiple copies of aliases a few times, but these duplicates were actually just ghost aliases, but it still slowed down the Finder.
Finally, I trashed the file com.apple.sidebarlists.plist, and that solved the problem. Here's the path to the file:
~/Users/Home/Library/Preferences/com.apple.sidebarlists.plist
[robg adds: This is probably a somewhat isolated incident (I've never seen such issues on my Macs), but for those who are having similar sidebar issues, knowing the name of the file to remove will save a bit of time.]
Mac OS X Hints
http://hints.macworld.com/article.php?story=20060722000521435