I keep only one thing (the screensaver module) in the dock when nothing else is running. I was somewhat surprised to find a Eudora icon there today, even though Eudora wasn't running.
Experimenting a bit, I discovered that if you simply move (or even just pick up and drop) the icon of a running application, it turns into a permanent dock icon when you quit that application.
I'm not sure whether this should be called a feature or a bug...
Mac OS X Hints
http://hints.macworld.com/article.php?story=20010104215430821