I was annoyed at Apple's Mail application slow loading of my only five folders with some hundreds of messages in them on application start-up. It took ages until it recognized all the folders.
I do not know whether it depends on the size of the folders or not, but there was always a long delay on simply displaying them. Here's my solution to this problem:
Mac OS X Hints
http://hints.macworld.com/article.php?story=20060311133731281