In Safari 4, the only way to toggle Private Browsing is via the Safari » Private Browsing... menu selection. However, in Safari 5, a button appears in the URL bar similar to the RSS and Reader buttons when Private Browsing is active. Simply click it and Private Browsing will be turned off.
[crarko adds: I tested this, and it works as described. Note this will only turn Private Browsing off, not back on. It really should be a true toggle instead. Maybe there will be (or is) a Safari extension to do this?]
Mac OS X Hints
http://hints.macworld.com/article.php?story=20100614160534520