Polling Remote POP3 Servers for Mail and pushing into Linux IMAP Server
I’ve always struggled with getting my email set up the way I want. Of course I can’t be happy with the “normal” way of getting email through the web, or through a simple local setup.
Here are my “unusual” email requirements:
- Very fast when accessed from local network. I need to be able to click through the messages very quickly.
- Easily accessed from many machines on local network.
- Available online, through IMAP or web interface.
- Easy to backup all messages on the server.
- No space limits so that I have access to all 10,000+ emails that I currently have.
- Expandable so that future users can easily be set up.
Using one of the popular email services like Yahoo Mail or GMail are out of the question just due to speed, and even the amount of email I have.
At one point, I tried using Exchange. It worked good, but I didn’t want to be owned by Microsoft. It’s just way too much money for the simple scenario I’m using.
Now that I’m starting to really use Ubuntu server, I’ve found another way. One of my favorite things about Linux is the fact that they give you a lot of gizmos, and you can organize them in whatever way you want, to do whatever you want. They give you tools for checking mail, and they give you tools for serving mail. It would seem to be a simple matter of hooking the two up.
To tackle the first part of the problem, I started looking into different IMAP servers that would run under Linux. I also wanted to make sure that it would store it’s mail in a standard format. There are two very popular standards for storing mail on a Linux machine. “mbox” and “Maildir“. From what I can tell, “Maildir” is usually the preferred choice.
I found a program called Dovecot, which seems to be very popular, easy to configure, and will serve up your email about anyway you can think of. After looking over the documentation, it took about 30 minutes to set up with their simple configuration examples. I was then up and running with a full IMAP system capable of serving “Maildir” mail. I’m not including my configuration file here, due to the length. I recommend simply looking at one of the configuration examples on their website. The only thing that was a little tricky, was the fact that I had to create a folder called “Maildir” under my home account, which would signal Dovecot that I wanted to use that mailbox format. If you don’t, it will assume mbox. I’m not sure if there is a configuration option for that.
The next piece of the puzzle was getting my email into the system, by having it poll my pop servers. There is a popular piece of software known as “getmail“, which is built exactly for this purpose. It will check your POP3 or IMAP mailbox, and deliver it to your “Maildir” mailbox. It took about 10 minutes getting the configuration file set up for it, and I was able to call a command line to check my email. I had to set up a configuration file for each account. Here is what one of my configuration files look like, except that I masked the username and password for obvious reasons:
[retriever]
type = SimplePOP3Retriever
server = mail.SuperJason.com
username = xxxxxx
password = xxxxxx[destination]
type = Maildir
path = ~/Maildir/[options]
delete = true
Next, to automate the whole thing, I set up a cron job using crontab. The easiest way to set up your cron job is to use “crontab -e”, which allows you to edit the cron jobs file. Once you save it, it will attempt to reload all of the jobs.
My cron job looks like this:
#Call getmail every minute to fetch my pop3 mail and put it into my Maildir
* * * * * /usr/bin/getmail –rcfile=mailacct1rc –rcfile=mailacct2rc –rcfile=mailacct3rc
That’s all there is to it. Now my system automatically polls my pop servers every minute, and pushes the email into my Maildir, which is accessible from anywhere via IMAP. This machine is its own virtual appliance made in VMware, so it runs independently off all my other computers. It just sits there quietly doing its job, and really doesn’t require any maintenance. It’s great having a powerful yet free email system that meets all of my requirements.
Thanks for the tutorial - this works great! Just remember when configuring your E-mail client, you want the username to be your Linux server login name and password, not your E-mail account login (that you put into getmail). Also, in Evolution, be sure to choose IMAP and not IMAP4. Thunderbird works fine, too.
Using Gmail instead of my old convoluted system