I believe this is all due to some sort of Apple Maps bug. Manually touching to get directions versus asking Siri for directions to the same address should give you directions to the same place. I suspected that Siri was using different address mappings than Maps was using, so I tried methodically asking Siri to give me directions to slightly different addresses on the same road to see if I could pinpoint the address that Siri thought my building was located at. This turned out to be an incorrect assumption. I discovered that if I ask Siri these 2 questions, I get two different destinations:
- Give me directions to xxx Road.
- Give me directions to xxx Road, City.
In order for me to be able to ask Siri for directions to work and get the correct result I had to delete only the city from the work address in my Contact card.
This problem is probably location specific; an issue with the map data itself, or possibly with the Maps app, so most people probably won't have this problem. I have submitted this issue to Apple via their feedback form and via the report a problem link in the Apple Maps app.
[crarko adds: Definitely sounds buggy to me, although I have noticed similar behavior in Google Maps and MapQuest before too. Anyway for future reference (if you don't have these) to report a bug in OS X go here and for iOS go here. If you're in the Developer program, you have access to other tools, of course.]

