I've always wondered how to delete music off of an iOS device running 5.0.x, which can be a pain what with iCloud downloading music to your iOS device, and then adding the same song a second time when syncing with iTunes.
When in the Music app on the iOS device just swipe like you would for deleting basically anything on an iOS device and you'll get the option to delete the song. If you have a bunch of songs you want to delete this will take some time to to but at least now you wouldn't have duplicates thanks to the new features of iCloud downloading the song and then iTunes syncing another copy of the song on the same device.
[crarko adds: The standard method for deleting line items in iOS comes to music.]
Mac OS X Hints
http://hints.macworld.com/article.php?story=20120215225957826