For audio files in your iTunes library, it's possible to quickly convert them to your default audio encoding using the command 'Create AAC Version' in the 'Advanced' menu. With video files, that particular command is greyed out. However, when pressing the Option key, the command turns into 'Convert to AAC…' which lets you select any iTunes-compatible file in the browse dialog and turn into AAC audio. While this approach works, it is clunky and cumbersome.
Recently, I've discovered a much more straightforward solution which takes advantage of functionality that seems to be sort of a bug in iTunes.
Select the desired video in iTunes and then choose the aforementioned 'Convert to AAC…' command using the Option key but let go of the Option key as soon as you selected the menu item. Instead of letting you select a file to convert, iTunes will happily convert the selected video to AAC.
Note however that your video will end up in iTunes' 'Music' section. If you want it to appear in 'Podcasts' or 'iTunes U', you will have to select it, get info and change the media type manually.
[crarko adds: It's also seems possible to get 'Convert to MP3' depending on the format of the original video.]