Apple has published a technical document explaining how to set message size limits for the mail server in OS X Server. By default, messages are limited to 10 MB, but you can change this from the command line using the serveradmin command:
sudo serveradmin settings mail:postfix:message_size_limit = numberSee Apple's technical document for more information, and how to specify the number in the command above.
Mac OS X Hints
http://hints.macworld.com/article.php?story=20130118021337177