Submit Hint Search The Forums LinksStatsPollsHeadlinesRSS
14,000 hints and counting!

Run a local IMAPd server for extra Mail storage Apps
For a while, I struggled with a difficult dilemma: my mail provider was trying hard to get me to transition from POP to IMAP, which everyone told me was wonderful, but they only provided 100mb of space -- about one-seventh of what I'd need, just for my current slew of mail. I looked into cutting down on what I've got stored, but in the end that just wasn't an option.

Finally my brother offered a suggestion that had never occurred to me: run a second IMAPd server on my own computer and use it for long-term storage. I did it, and it works beautifully: I now have one IMAP server for inbox, sent mail, etc., set up by my mail provider, and a second on my own computer for archiving. These directions for setting up one's own IMAPd server were written with beautiful clarity by Michael Johnson. I would only add a few things:

  • The line curl -O ftp://ftp.cac.washington.edu/imap/imap-2004.RC.tar.Z should be curl -O ftp://ftp.cac.washington.edu/imap/imap-2004.tar.Z, at least at present. Likewise, tar -zxvf imap-2004.RC.tar.Z should be tar -zxvf imap-2004.tar.Z.
  • In both of the xinetd files (imap and imaps), you might consider the only_from option, as detailed in the man file for xinetd.conf.
  • If you are running Apple's built-in firewall, go to System Preferences, select Sharing, choose the Firewall tab, click on New, and create an IMAP service that opens ports 143 and 993. Ensure that the box next to that service is checked.
  • As far as advanced settings in Mail.app are concerned, I have had the best luck not specifying any account directory at all (certainly do NOT specify the mail subdirectory that you put into env_unix.c before compiling!), automatically synchronizing changed mailboxes, and keeping all messages and attachments; there's one big long synchronization at the start, but then everything's copacetic.
  • If you use SSL, the certificate created in Michael's setup is not recognized by Mail, so on startup one must click away a warning window, every time. I have tried to alter this behavior by adding the certificate to various keychains, with no success. If anyone has successfully circumvented this problem, please let me know....
[robg adds: I haven't tried this one...]
    •    
  • Currently 2.00 / 5
  You rated: 4 / 5 (4 votes cast)
 
[19,914 views]  

Run a local IMAPd server for extra Mail storage | 35 comments | Create New Account
Click here to return to the 'Run a local IMAPd server for extra Mail storage' hint
The following comments are owned by whoever posted them. This site is not responsible for what they say.
Run a local IMAPd server for extra Mail storage
Authored by: Black on Jun 08, '04 11:39:40AM
A local IMAP server has a lot going for it. I recently started using one to get around a pesky communications problem between Mail.app and one of my mail servers. However, this seems like too much work for me. Why not just use PostfixEnabler which bundles UW-IMAP and allows you to setup your mail server in a much more civilized fashion?

[ Reply to This | # ]
Run a local IMAPd server for extra Mail storage
Authored by: jdera on Jun 08, '04 11:40:01AM

Perhaps I'm missing the point of this hint. Why not just use locally stored folders right in Mail.app? It doesn't sound like the IMAP services are being shared out onto the Internet or anything, so why run a whole IMAP daemon? This sounds more like someone who doesn't understand how their mail software works than a tip others should follow for resolving storage issues.



[ Reply to This | # ]
and when you graduate to more than...
Authored by: mzs on Jun 08, '04 01:19:38PM

one computer and want to get at your mail from all your computers... This fellow looks to be an experienced mail user by my estimation.



[ Reply to This | # ]
Run a local IMAPd server for extra Mail storage
Authored by: mccabem on Jun 09, '05 02:59:37AM

The subject is a bit misleading and yes, you are missint the point. ;-P One would not pursue this option for extra mail storage space.

The primary idea is getting this mail Internet-accessible.

IMAP is really the best way to do this. If you read through (and understand :-) ) the article summary it's apparent this is what is happening. Key in on the Firewall modifications.

Good luck!

FWIW, running fetchmail is probably the easiest way to get mail *into* your new IMAP server. Hopefully this will one day become part of PostfixEnabler's installation.



[ Reply to This | # ]
Run a local IMAPd server for extra Mail storage
Authored by: jdera on Jun 08, '04 11:44:11AM

Also, this article describes how to import self-signed certificates into the Mac OS X system hooks, which will bypass the SSL error in Mail.app.



[ Reply to This | # ]
Run a local IMAPd server for extra Mail storage
Authored by: Puzo on Jun 08, '04 12:22:52PM

just wanted to clarify the point that when you make certificate, make sure you enter server name for CN: field (i.e. instead of your own name, as some tutorials may suggest, enter example.com) Otherwise, your Mail.app will still complain after you transfer the certificate to your keychain.



[ Reply to This | # ]
Run a local IMAPd server for extra Mail storage
Authored by: tinker on Jun 08, '04 05:05:30PM
THAT'S what I screwed up! Re-doing my certificate with

cd /System/Library/OpenSSL/certs
sudo openssl req -new -x509 -nodes -out imapd.pem -keyout imapd.pem -days 3650

, using my own domain for CN: and then following jdera's URI's instructions for installation seems to have done the trick. Thanks much!

[ Reply to This | # ]

Run a local IMAPd server for extra Mail storage
Authored by: digitaltvguy on Jun 08, '04 11:51:07AM

I'm also missing the point. Why not just store your excess mail in your local computer ("On Mac").

---

-Chris



[ Reply to This | # ]
Run a local IMAPd server for extra Mail storage
Authored by: rainwadj on Jun 08, '04 01:17:59PM

One advantage is that you can then access your local mailboxes from remote locales, assuming you poke the appropriate holes in your local firewall.



[ Reply to This | # ]
Here's one reason not to depend just on mail.app
Authored by: ubrgeek on Jun 08, '04 02:55:16PM

If you use Norton, watch out for email just vanishing. Completely. Instantly. Happened to me last night. Norton auto-protect determined an incoming piece of mail was infected and promptly deleted my all of my inbox. "Known issue" apparently. Makes me wish I kept my mail on my personal mail server. This link references OS 9, but apple's site confirms that it's still an issue: http://service1.symantec.com/SUPPORT/num.nsf/6164320143cb6f0c88256d01004ee56c/b8737141567374c588256e37008281dd?OpenDocument&prod=Norton%20AntiVirus%20for%20Macintosh&ver=9.0&tpre=eu&src=eu_sg&csm=no



[ Reply to This | # ]
yet another reason ...
Authored by: sjk on Jun 08, '04 06:46:11PM

Sounds like the Subject could have been "Here's yet another reason not to depend on Norton"? ;-)

(sorry, couldn't resist)



[ Reply to This | # ]
yet another reason ...
Authored by: GlowingApple on Jun 09, '05 09:10:07AM

The bonus of Norton I guess is that it also filters out viruses that could affect PCs (AFAIK there are no viruses currently for OS X). So I guess you could say that PCs are plagued by viruses whereas Macs are plagued by Norton, trying to help the sick little PCs...

---
Jayson --When Microsoft asks you, "Where do you want to go today?" tell them "Apple."



[ Reply to This | # ]
Run a local IMAPd server for extra Mail storage
Authored by: tinker on Jun 08, '04 04:50:58PM
Having confused at least two people, let me clarify: mzs and rainwadj are right on. The problem with storing mail "on my Mac" is that there's a lot of it and I want to access it from elsewhere. For some time I had an rsync script set up to synchronize my POP mailboxes across computers, but that's a substantial pain. Worse, I'd go through two days' worth of spam while out of the office, then return to the office and find 100+ messages waiting for me -- the same spam, waiting to be re-deleted!!

The point is, whenever I fire up Mail.app, I'm now looking at the same mail. When I file something at home, it's filed when I get to work. In short, it's all of the benefits of IMAP rather than POP, and I can access the entirety of my massive mail archive wherever I go, despite my mail provider's meager limits.

(And thanks, mzs, for the benefit of the doubt; truth is, this guy's a desperate mail user! And while necessity is the mother of invention, desperation is the doctor who performs the C-section....)

[ Reply to This | # ]

Run a local IMAPd server for extra Mail storage
Authored by: quentinsf on Jun 08, '04 05:49:33PM

Another advantage is that you can switch between different email programs at will without having to import all your old messages.



[ Reply to This | # ]
Run a local IMAPd server for extra Mail storage
Authored by: sjk on Jun 08, '04 06:49:36PM

And POP can only access one INBOX per account. With IMAP any mailbox can be an INBOX.



[ Reply to This | # ]
Run a local IMAPd server for extra Mail storage
Authored by: david-bo on Jun 13, '04 07:11:13PM

This is ridiculous (with one exception, if you receive 100MB+ of mail in every other day or so, this hint makes sense).

Why not just archive all mail stored on your ISP:s mailserver to your local HD once a month or so, i.e., leave mail on server for, for example, 30 days?

---
http://www.google.com/search?as_q=%22Authored+by%3A+david-bo%22&num=10&hl=en&ie=ISO-8859-1&btnG=



[ Reply to This | # ]
Run a local IMAPd server for extra Mail storage
Authored by: tinker on Jun 16, '04 02:17:05PM

Because you only have to file things once -- you can file everything directly to the mail archive instead of having to coordinate folders every 30 days; because you can have access to all of your mail, not just the last 30 days' worth (at most; 0 days at least) from wherever you are; because, once it's up and running, it's easy.

If you want to archive locally as well, you can just go to the Advanced panel and select "Keep copies of messages for offline viewing: all messages and their attachments." Voila.



[ Reply to This | # ]
Run a local IMAPd server for extra Mail storage
Authored by: mscheurer on Jun 09, '04 06:01:22AM

I must be a bit thick today, I've done the install, but what the h... do I enter in Mail as the account details?


michael



[ Reply to This | # ]
Run a local IMAPd server for extra Mail storage
Authored by: tinker on Jun 11, '04 03:55:54PM

Sorry, should have included that info. Account type is IMAP. Description can be whatever you want. E-mail should be whatever e-mail you want to have mail be sent from. Full name is just your full name.

Incoming mail server should be the IP of the machine on which you've just set up your IMAPd service. Before you write in and say, "You rotten son-of-a-toad, I don't have a static IP!!" check out No-IP.com; they'll set you up with an IP that you can use for machines with dynamic IPs. Pretty cool stuff.

The rest is cake. User name is just your account name on the IMAPd host machine, password is the password. Outgoing mail server depends on your setup.

Note that much of this is irrelevant. When I send e-mail, I send it from my OTHER IMAP account, the one that receives mail (remember that this second IMAP account is basically for archiving messages). I don't receive mail on this account, so I'm never prompted to return mail from it. Therefore, the e-mail address, full name, and smtp server are never used.



[ Reply to This | # ]
Run a local IMAPd server for extra Mail storage
Authored by: mscheurer on Jun 11, '04 07:10:37PM

Thanks tinker.



[ Reply to This | # ]
Run a local IMAPd server for extra Mail storage
Authored by: 68lc040 on Aug 10, '04 10:37:47AM

hello,
this does not work for me: the server rejects my system username/password... is it because of the certificate? how do i enter the certificate data in the mail app? is this even needed? what is the necessary information for the creation of the ceritficate?

thanks!



[ Reply to This | # ]
Run a local IMAPd server for extra Mail storage
Authored by: dansroka on Jun 09, '04 08:34:52AM

Interesting hint. Thanks guys for explaining the practical use of a local IMAP server. At first, I too was thinking "why would you need that?". But now I see how it actually could be very useful. I have a several-computer home business, and have also been struggling with the "which computer did I download that mail message on to?"

My question is: how stable are these two solutions (Tinker's original hint, and suggestion of using Postfix) with respect to Apple updates? I'd hate to create a mail solution that had the potential for breaking the next time OS X gets updated.



[ Reply to This | # ]
Postfix configuration
Authored by: sjk on Jun 09, '04 11:28:43PM
I don't know specifically what Postfix Enabler modifies but minimally I'd save a backup of everything in /etc/postfix before and after running it or making any Postfix configuration changes (e.g. using postfix -e ... commands). Same applies for Apple making any unexpected and unwanted updates to its contents that you want to recover from.

Run cd /etc/postfix, then tail -15 main.cf and diff master.cf.defaultserver master.cf to see a couple of Apple's changes.

For the technically inclined, Postfix Basic Configuration is a good starting point.

I hope people setting up e-mail (and other) services have at least minimal interest in understanding what they're doing. :-)

[ Reply to This | # ]
Postfix configuration
Authored by: tinker on Jun 11, '04 03:45:31PM

Yep. Actually Postfix Enabler said it worked for me but didn't, as I discovered when I tried sending a slew of e-mail, none of which arrived. Then, later, when I enabled Postfix by hand, I started getting responses to those e-mails (!!!) So I've become a big fan of rolling up one's sleeves rather than clicking a button.



[ Reply to This | # ]
Run a local IMAPd server for extra Mail storage
Authored by: alani on Jun 14, '04 09:42:03AM

Great hint!

I just finished putting together this combination: Fetchmail-Postfix-Procmail-SpamAssassub-IMAPd.
It grabs email from my POP vendors, filters it through SpamAssassin, and serves it out to Mail.app and other clients on the local machine, LAN, and WAN.

The reasons to do something like this are really obvious to me, but apparently not to everyone, so let me spell it out:
1. Overcome email providers' storage limit.
2. Consolidate email from multiple vendors.
3. Use spam filters trained specifically for your email usage.
4. Multiple email clients on multiple machines stay sync'd.
5. Archive and backup email however you like.
(6. Run SquirrelMail webmail if you want)

I'm a total amateur and it's entirely possible I've opened up security holes, created opportunities for data corruption, and who know what else. But I'm pleased with how things have been working.

I'll get around to writing up the recipe and posting it here in the near future. But, suffice it to say, it's not too hard (now that I've found the installation pitfalls for you), and it's catching better than 9 of 10 spam messages that had been getting through.



[ Reply to This | # ]
Run a local IMAPd server for extra Mail storage
Authored by: fozzy on Jul 03, '04 01:29:20AM
I would add a couple steps/clarifications to Michael's original doc. That is that when creating the SSL certificate, you must specify the client hostname or IP address when prompted for "Common Name" As in the below:

fozzypb12:/System/Library/OpenSSL/certs $ openssl req -new -x509 -nodes -out imapd.pem  -keyout imapd.pem -days 3650
Generating a 1024 bit RSA private key
......................................................++++++
.................................++++++
writing new private key to 'imapd.pem'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:
State or Province Name (full name) [Some-State]:
Locality Name (eg, city) []:
Organization Name (eg, company) [Internet Widgits Pty Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (eg, YOUR name) []:192.168.1.103
It took me a while to figure that out. You'll need a separate imapd-IP.pem file for each host you intend to connect from. Also, this http://www.stepwise.com/Articles/Workbench/eart.index.html seems to be an alternate way that goes into more detail around integrating fetchmail, etc. That's where I found the bit about "Common Name" being IP address.

[ Reply to This | # ]
Original document?
Authored by: dmoren on Aug 02, '04 07:24:21PM

Did anybody snag a copy of the document that was linked to in this post? The server seems to have gone down...



[ Reply to This | # ]
Original document?
Authored by: tinker on Aug 21, '04 10:13:52PM

Not pretty, but here it is:

Secure IMAPd for OS X 10.3

By Michael Johnson
(with the assistance of several people on the OS X Server mailing list)




Compiling the UW IMAP server is a fairly simple task, but figuring it out was almost enough to make you need some Rogaine after pulling out all your hair. This document will hopefully spare you a few of those precious hairs on your head (if you happen to have any left).

The first thing, you should have at least OS X 10.3.1. I imagine it will work with earlier versions, but I haven't tested it or heard that it works.

Now, we need to make a directory for working. I like to use the /Developer directory which was created when I installed the Developer Tools distro from Apple.

cd /Developer
mkdir imapd-build
cd imapd-build

This just made the directory we needed.

Now, on to get the source...

curl -O ftp://ftp.cac.washington.edu/imap/imap.tar.Z


This will get the most recent version of uw-umap. You'll need the RC version for now. Hopefully soon the release version will work

Now, let's get the thing uncompressed.

tar -zxvf imap-2004.RC.tar.Z


That's easy enough. It shouldn't take very long to get that done. On faster machines, blink and you might miss it. On my G4 500, it took only a few seconds.

Now, to get to the fun part. This took a while to figure out, but it wasn't forever. It's really fairly simple and boils down to the build script not seeing where to look for OpenSSL even though it supposedly already knows. We also need to tell it where to look in the home directory to store our mail.

cd imap[version]
sudo pico -w src/osdep/unix/env_unix.c
#change the line containing 'static char *mailsubdir = NIL;' to show 'static char *mailsubdir = "mail";'
sudo make oxp SSLDIR=/System/Library/OpenSSL SSLINCLUDE=/usr/include/openssl SSLLIB=/usr/lib
**You may be able to simply run "make osp" with the latest versions**

This was the hard part of this...figuring out what exactly it needed to be able to build the binary with SSL enabled and how to get PAM working with the new auth scheme in Panther.

Now, it's built and we need to move it to someplace useful.

sudo cp imapd/imapd /usr/local/libexec/imapd


This will actually just copy the binary over so if you blast it out tinkering with something later on, you can recover the binary here.

Now lets move on to making our certificate. We need to tell our imap users we're a fine upstanding person/organization and hope they trust us with our self-signed certificate.

cd /System/Library/OpenSSL/certs
sudo openssl req -new -x509 -nodes -out imapd.pem -keyout imapd.pem -days 3650

This should have created a certificated called imapd.pem which will be good for just shy of 10 years. I did it using sudo. I don't know if it will work as a regular admin user or not. I did it this way to save time since I was not in the mood for being told I couldn't write to the directory at that moment. I think I needed more coffee.

So the system knows what to do when we get a request in on port 143 or 993, we need to tell it using xinetd. This is pretty easy.

cd /etc/xinetd.d
sudo pico imap

Insert the following into the file:

service imap
{
disable = no
socket_type = stream
wait = no
user = root
server = /usr/local/libexec/imapd
groups = yes
log_on_success += DURATION USERID
log_on_failure += USERID
flags = NOLIBWRAP
}


Now, you need to write the file. Since you're in pico, just hit ^x and answer [Y]es to the prompt. Then hit <return> to accept the filename 'imap' (without the quotes) we already gave it.

Let's do the same thing with imaps since we're already in the directory.

sudo pico imaps


Again, we need to insert the stuff it needs to know.

service imaps
{
disable = no
socket_type = stream
wait = no
user = root
server = /usr/local/libexec/imapd
groups = yes
log_on_success += DURATION USERID
log_on_failure += USERID
flags = NOLIBWRAP
}



Save the file just as you did before except of course, use the name 'imaps' (without the quotes).


Now, we need to tell PAM about imap:

cd /etc/pam.d
sudo pico -w imap
Insert the following:
# imap : auth account password session
auth required pam_nologin.so
auth sufficient pam_securityserver.so
auth sufficient pam_unix.so
auth required pam_deny.so
account required pam_permit.so
password required pam_deny.so
session required pam_uwtmp.so



Let's check to see if there's a /usr/include/security directory

cd /usr/include/security
If you get a reply there is no such file or directory
do the following:
cd /usr/include/
sudo ln -s pam security

It's the home stretch now. We just need to restart xinetd.




sudo kill -HUP `cat /var/run/xinetd.pid`



That's it. Really, that's all there is to doing this. If you have any addendum or comment, please let me know by emailing me at my home account, michael (at) emjay (dot) net.

©2003-2004 Michael Johnson




[ Reply to This | # ]
Original document?
Authored by: luhmann on Feb 06, '05 10:45:31PM

tar -zxvf imap-2004.RC.tar.Z

is now:

tar -zxvf imap.tar.Z



[ Reply to This | # ]
Original document?
Authored by: luhmann on Feb 06, '05 10:51:28PM

If you have TextWrangler installed, with the command line tool. (Both are now free!) You can replace:

sudo pico -w src/osdep/unix/env_unix.c

with

edit src/osdep/unix/env_unix.c

And then edit it in textwrangler instead.



[ Reply to This | # ]
Original document?
Authored by: luhmann on Feb 06, '05 11:13:23PM

OK. I've followed all the steps, but now what? How do I make an account, or configure Mail.app to work with this?



[ Reply to This | # ]
Original document?
Authored by: luhmann on Feb 10, '05 07:41:52AM

After struggling with this for days I looked at the comments again. I saw the tip about Postfix Enabler. I downloaded it, clicked "enable IMAP" and everything worked! So much easier!!!



[ Reply to This | # ]
Original document?-Final instructions
Authored by: rtpeters on Jan 07, '06 07:40:52PM

Just found this thread while looking to learn how to host a local IMAP server to keep my mail synced on multiple macs. The idea was explored but not completed on another site. That thread also suggested using Postfix Enabler, along with fetchmail, to retrieve mail from my ISP and serve it to my other macs. I have Postfix Enabler downloaded, but I really don't know what to do with it. I'm running OS10.4.3 and I"m looking for clear (read "for dummies") instructions on how to set this up and get it running. Hopefully,without going to Terminal. These threads are farily old so I'm hoping someone built an interface to handle this by now. If not, I'd still like to give it a try. Or, if someone has since determined this is crazy, or there is a better way, let me know.
Thanks
Ron



[ Reply to This | # ]
Modification for use with Tiger
Authored by: tinker on May 26, '05 11:42:15PM
OK, so I have to say, I felt just lovely after installing Tiger and discovering that it breaks this hint. It seems (??) to install some new files over the ones that are created above.

The solution that I discovered was to go through and re-do the original procedure, verbatim (excluding even the minor modifications that I made to the curl command above), except for the last line:

sudo kill -HUP `cat /var/run/xinetd.pid`

no longer kills the xinetd procedure. I'm sure there's a succinct way to do it, but I just typed in ps -axe, looked for the xinetd procedure, and used sudo kill -9 xxxx (where xxxx is the number of the procedure) to kill it.

The silver lining is that you get a shiny new copy of imapd on your server.

My preemptive apologies to anyone who spent significant amounts of time figuring this out before coming here. I'm afraid Apple never informed me of their plans. :-)

[ Reply to This | # ]
Run a local IMAPd server for extra Mail storage
Authored by: mccabem on Jun 09, '05 03:03:45AM

A somewhat simpler (to administer at least) solution would be to simply tunnel to your Mac from wherever, opening a port for VNC. (Only allow VNC to access localhost connections!) Then just use VNC to read your mail "locally" no matter where you are.

Of course this solution comes with limitations just like any other, so pick and choose!



[ Reply to This | # ]