This script will allow you to extract artist and album names from your iTunes Library.xml file (File -> Export Library). The list is sorted by artist, then by album name. The output is an unmolested copy of whatever was in your library. This means that non-ASCII characters are still going to be there, for better or worse.
[robg adds: Due to the length of the script, I uploaded it to the macosxhints server instead of listing it in the body of the hint, and I have not tested this one. Remember to make the script executable before you try to run it (chmod +x script_name).]
Mac OS X Hints
http://hints.macworld.com/article.php?story=20050105054852773