SSHKeychain used to set the SSH_AUTH_SOCK environment variable to something like /tmp/50x/SSHKeychain.socket, but even getting rid of SSHKeychain didn't get rid of the environment setting.
I used to use a great little program called SSHKeychain to maintain my ssh world on OS X. Even after Apple's keychain started dealing with ssh-agent properly, I still kept SSHKeychain around for the convenience of maintaining my ssh tunnel configurations in one place.
However, there was an occasional mismatch between the SSHKeychain way of doing things and the Apple keychain way, and SSHKeychain seemed to have fallen into disrepair, with no maintenance (on SourceForge at least) since 2007.
A few months back, I finally got rid of SSHKeychain, and started having problems.
There was apparently a phase-of-the-moon dependent race condition (somewhere) that clobbered the Apple keychain setting of SSH_AUTH_SOCK with a setting left over from SSHKeychain. This manifested itself as an occasional (during about 50% of my logins to my OS X account) demand for me to enter my ssh passphrase every time I made an ssh connection somewhere else. This, of course, kind of defeats the whole purpose of using ssh-agent in the first place, and was a major pain as well.
I looked at everything I could think of (e.g. .profile et al.) that was related to the problem to no avail. Spotlight was no help in locating the where the variable was being set from. Finally, yesterday, I decided that enough was enough, and ran a find|grep chain over what I thought was my entire disk. There were a few hits (including one in /private/var/vm/sleepimage that seemed like it could have been causing the problem) but getting rid of those still didn't do the trick.
I finally stumbled upon a useful magic search phrase for the problem in Google, and found this discussion from 2007.
It turns out that the culprit was the file ~/.macosx/environment.plist which appears to only have set that one environment variable. Getting rid of that file got rid of the intermittent clobbering of the SSH_AUTH_SOCK variable, and allowed the Apple keychain to maintain access to ssh-agent in the modern fashion.
Woot.
As an aside, I apparently did not construct my find command (in the find|grep chain that I mentioned above) in a way that allowed it to descend into directories starting with a '.' and my brute force search missed the culprit.
I hope this helps someone else avoid all of the blood, sweat, and tears I expended on the problem.
[kirkmc adds: As obscure as this is, we all know what it's like to search for an uncommon problem and not find it. Every problem of this type that is documented helps others in the future when they search for a response.
It's worth noting that we had a hint about some problems with SSHKeychain back in 2007 as well.]
Mac OS X Hints
http://hints.macworld.com/article.php?story=20120425080226375