The file alarmclock.mov exists in two different places, and it doesn't work to just delete them (well, it did work in 10.4.2, but not in 10.4.8), so the trick is to make it an empty file. In Terminal, move to the first directory:
admin$ cd /Applications/iCal.app/Contents/Resources/\
iCal Helper.app/Contents/Resources/
Make a backup of the file (-p preserves the permissions) of the file for safety's sake. You will be prompted for your administrator password:
admin$ sudo cp -p alarmclock.mov alarmclock-mov.BAC
Now overwrite the original:
admin$ sudo echo "" >alarmclock.mov
Finally, do the same steps for the file in /Applications -> iCal.app -> Contents -> Resources:
admin$ cd /Applications/iCal.app/Contents/Resources
admin$ sudo cp -p alarmclock.mov alarmclock-mov.BAC
admin$ sudo echo "" >alarmclock.mov
That's it. When the notification dialog is displayed, there will now be nice quiet white space where the alarm clock used to be.
[robg adds: As noted in this previous hint, if you disable QuickTime's Flash support, you'll also disable the alarm clock. This hint, however, provides a more specific solution without requiring you to give up Flash in QuickTime completely.]

