Teaching Your Email to Be Even More Fetching

What is Fetchmail?

In my previous article, Teaching Your Email to Fetch, I introduced you to Fetchmail, a wonderful utility written by Eric Raymond. Fetchmail will do just about anything necessary to retrieve your email from any standards-compliant mailbox, and store it elsewhere. Think of it as gravity for your mailbox. If you have email stuck in remote planetary orbits, you can make it all come down to earth with Fetchmail. In the last article, I also talked about how to do standard, straightforward things like fetch your mail from a POP3 or IMAP mailbox, and send it unencrypted to a single local mailbox. That's how most people use Fetchmail.

If you don't already use Fetchmail, or could use a refresher, take a moment and skim through Teaching Your Email to Fetch before you read this one. This article is for those of you who like to push the envelope a bit. For the security conscious, I'll cover ways to use the Secure Socket Layer (SSL) and Secure Shell (SSH) to encrypt the traffic between Fetchmail and your upstream mailbox. I'll also cover how to use SSH to wrap encryption around your downstream Simple Mail Transfer Protocol (SMTP) traffic. Finally, I'll cover how to use alternative methods of delivery to get the mail where you want it.

First, let's take a look at using encryption in your mail-retrieval process. Encryption can thwart any prying eyes that might be watching your correspondence as it travels on the Net from your mail server to your Fetchmail process.

Using SSH in a Plug-in

Let's say that you like to read your mail on remotehost. Because remotehost's network connection is spotty at best, you receive your mail on centralserver at your ISP, and use Fetchmail to pull it down to remotehost as needed. That way, if your network connection goes down for an extended period of time, your mail won't bounce, and you can catch up on it as soon as the network comes back up. Since you have talent and paranoia in equal measure, you shun the idea of retrieving your email in cleartext over the ISP's network. Luckily, you have a shell account and SSH access on both centralserver and remotehost.

One way to use your SSH access and Fetchmail in tandem would be to tell Fetchmail to wrap SSH around a process that logs in using your credentials to centralserver. This process can run the IMAP server on the command line (thus short-circuiting the need for further authentication), and retrieve your mail.

The first thing to do is to enable yourself to log in to centralserver without SSH demanding your password. The really bad (insecure) way of doing it would be to use .rhosts-style authentication, or a .shosts file, so we'll just skip on to the better way of doing this, with RSA authentication keys. If you prefer or need to use Digital Signature Algorithm (DSA) keys instead, the instructions are just about the same, check the man page on ssh-keygen for help. ssh-keygen, the utility you use to generate SSH authentication keys, should come with any full implementation of SSH. Without authentication keys, a simple one-command session, where SSH would let you log in to a remote host, issue one command line, and leave would look like this:

> ssh joeuser@centralserver.isp.net \
  echo This is \`whoami\` at\`hostname\`.
 joeuser@centralserver.isp.net's password: **********
 This is joeuser at  centralserver.isp.net.
>

The local SSH client demands your password before it logs in and issues the command line. This would obviously be problematic if you want to check your mail every five minutes. Use ssh-keygen to generate authentication keys. These can be generated to demand a passphrase/password, or not. In this case, we'll elect not to have a passphrase, since we want to accomplish this process noninteractively. First, we generate a public and private authentication key:

   
   joeuser@remotehost:~> ssh-keygen -b 1024 -f ./mykey
   Generating RSA keys: ...ooooooO...............
   ..............................................
   ..........ooooooO
   Key generation complete.
   Enter passphrase (empty for no passphrase):
   Enter same passphrase again:
   Your identification has been saved in ./mykey.
   Your public key has been saved in ./mykey.pub.
   The key fingerprint is:
   aa:bf:86:db:42:b3:0e:1d:d2:dd:24:03:fe:a9:6c:93
   joeuser@remotehost.example.com
   joeuser@remotehost:~>

In the previous example, we generated a key 1024 bits long with the -b 1024 option, and told it to use the name mykey as the base of the filenames for the key files. At each password prompt, just hit return to give the keys a no passphrase. The ssh-keygen command above actually generates two keys because we're generating keys to perform public-key authentication and that requires both a "private" key, used to encrypt data (sometimes in conjunction with a keyphrase, but not here) and a "public" key, used to decrypt data. Our private key is in the mykey file and the public key is in the mykey.pub file. On a standard ssh or openssh server installation, you make use of the public key by appending it to the file ~/.ssh/authorized_keys. The safest way to do this is to do something like cat mykey.pub>>~/.ssh/authorized.keys.

A word about the security of your keys. Although compromise of your public key represents no substantial threat, your private key should be kept as private as possible. This is especially important because the key we just generated has no passphrase. Anyone who gets this private key can use its credentials to log in to the remote host, where the public key is installed. Since compromise of the private key represents a profound security risk, the key should have no group or other rights, and be in a directory visible only to yourself. If you want to go even further, you could lock down the shell account to which you are connecting via SSH (in this case, "centralserver"), so that you are unable to do anything but run the IMAP server.

Here's a Fetchmail recipe to put the whole thing together. After testing, you can change "skip" to "poll."

  
   # use ssh for a plugin to central imap server
   skip centralserver.myisp.net protocol IMAP:
   preauth ssh
   plugin "ssh -i /home/joeuser/.ssh/mykey
   joeuser@centralserver.myisp.net /usr/sbin/imapd"

Related Reading Managing IMAP By Dianna Mullet, Kevin Mullet Table of Contents Index Sample Chapter Author's Article Read Online--Safari Search this book on Safari:   Only This Book All of Safari Code Fragments only

The recipe starts off just like any other IMAP recipe. It specifies the server and the protocol to use to retrieve your mail. preauth ssh tells Fetchmail to prepare to encounter an IMAP server that doesn't need any authentication. This recipe calls for the IMAP daemon to be invoked on the command line instead of through inetd. When invoked in that way, it isn't necessary to perform additional authentication, and imapd lets you know by sending you (or rather, Fetchmail) the PREAUTH string, once you fire it up.

Everything between the two quotes after the plug-in statement is the plug-in command. Fetchmail isn't picky about whitespace, so feel free to split up long lines onto multiple physical lines. This command provides "virtual plumbing" between Fetchmail and the IMAP mail server. If you issue the command by itself (the entire ssh ... /usr/sbin/imapd business), you'll be face to face with the IMAP server on centralserver, as if you had issued a telnet centralserver.myisp.net imap command. The important difference is that with the ssh command, all traffic between you on remotehost and the IMAP daemon on centralserver is strongly and transparently (at least to you) encrypted.

When we test the recipe by running Fetchmail and specifying the server on the command line (because we've still got that "skip" in there), we get something like this:

  
   joeuser@remotehost:~> fetchmail centralserver.myisp.net
   fetchmail: 5.5.5 querying centralserver.myisp.net 
   	(protocol IMAP) at Mon, 23 Oct 2000 03:55:09 -0500 (CDT)
   fetchmail: 5 messages (5 seen) for kwm at centralserver.myisp.net.
   fetchmail: reading message 1 of 5 (2944 header octets) 
   	..fetchmail: (1619 body octets) ..fetchmail: flushed
   fetchmail: reading message 2 of 5 (1672 header octets) 
   	.fetchmail: (5817 body octets) ......fetchmail: flushed
   fetchmail: reading message 3 of 5 (1491 header octets) 
   	.fetchmail: (1489 body octets) .fetchmail: flushed
   fetchmail: reading message 4 of 5 (1594 header octets) 
   	.fetchmail: (35019 body octets) 
	..................................fetchmail: flushed
   fetchmail: reading message 5 of 5 (1889 header octets) .fetchmail:
   (5091 body octets) .....fetchmail: flushed
   joeuser@remotehost:~>

Since we haven't specifically told Fetchmail what to do with the mail it retrieves, it delivers the mail via SMTP to localhost on port 25. Naturally, you'd replace "skip" in the recipe with "poll" and either run Fetchmail as a daemon or out of cron on a regular basis to automate your mail retrieval. Regardless, you've now got a sure-fire recipe for encrypted mail retrieval from a centralized server. One of the nice things about this particular method is that technically, your mail server ("centralserver," in our example) doesn't even need to have its IMAP daemon servicing connections from the Internet. imapd need only be properly compiled and available for you to run on the command line. The only service "centralserver" need have turned on is SSH.

The SSH/SMTP Plug-out

Taking the idea of the plug-in one step further, we can use the Fetchmail plug-out facility to encrypt the delivery of the mail we retrieve. In fact, it's rather straightforward to just cut and paste the plug-in rule, turn it into a plug-out, and generate a corresponding "empty passphrase" authentication certificate on the SMTP server, just like we did for the plug-in.

Here's the final product:

   
   # use ssh for both IMAP and SMTP encryption.
   skip centralserver.myisp.net protocol IMAP:
   preauth ssh
   plugin "ssh -i /home/joeuser/.ssh/mykey
   joeuser@centralserver.myisp.net /usr/sbin/imapd"
   
   plugout "ssh -i /home/joeuser/.ssh/myotherkey
   joeuser@smtpserver.myisp.net /usr/lib/sendmail -bs"

If your private and public keys are in the right place, this recipe carries out all its work inside encrypted SSH tunnels. Both IMAP and SMTP traffic is now safe from prying eyes, leaving the hosts themselves, and the routes to and from your site as the primary risks.

Using OpenSSL

Although Fetchmail builds by default without SSL support, adding it is as easy as starting the build process with the ./configure--with-ssl command. Presuming you've installed OpenSSL already, you should soon be able to confirm that your Fetchmail build has SSL support with the command fetchmail -V. Your output should look something like this:

   joeuser@remotehost:~> fetchmail -V
   This is fetchmail release 5.5.5+SSL
   Linux remotehost.example.com 2.2.14 #1 
   	SMP Thu Feb 17 11:49:42 MST 2000 i586 unknown
   Taking options from command line 
   	and /home/joeuser/.fetchmailrc
   Logfile is $HOME/fetchmail.log
   Idfile is /home/joeuser/.fetchids
   Fetchmail will forward misaddressed 
   	multidrop messages to joeuser.
   joeuser@remotehost:~>

As you can see, the second line listing the Fetchmail release also says +SSL. Of course, you never really know if SSL support is working until you try it. The following recipe directs Fetchmail to retrieve the mail from account "joeuser" at centralserver, using the standard IMAPS (IMAP over SSL) port:

  
   skip centralserver.myisp.net port 993 protocol IMAP:
   user joeuser password somePassWord
   ssl

All Fetchmail SSL recipes must specify a port over which the secure IMAP takes place. In this recipe, we've told Fetchmail that centralserver provides IMAPS service on port 993. All SSL recipes must also contain the SSL directive.

As you can see, connecting to an IMAP server that already supports SSL is much easier than setting up an SSH tunnel. The availability of both options, however, makes Fetchmail a pretty flexible and secure way to retrieve your mail from less flexible locations. Sometimes, central mail servers don't support IMAP over SSL, and they rarely support SMTP over SSL, so tunneling inside SSH with a plug-in is a good option to have.

Another feature of Fetchmail that lets you "bend it to your wishes" is the ability to use alternate mail delivery agents. Let's say you preferred to run sendmail on the command line to avoid running an SMTP server on your workstation, or you want to pull all the mail from a given mailbox directly into a support ticket application. Fetchmail will let you run a customized delivery agent to do what you need.

Alternate Delivery

As it happens, the utility that was rewritten to give birth to Fetchmail, popclient, used a built-in local delivery facility outside of SMTP. popclient was scrapped early on. Later, Eric Raymond put a rewritten, custom, local-delivery agent facility back into Fetchmail. That facility gives you the ability to pipe your incoming mail into any arbitrary program you desire. Let's take a look at some of the possible uses of this feature:

defaults
   fetchall
   no keep
skip centralserver.myisp.net protocol IMAP:
   user joeuser password passwordOFjoe
   mda "cat >>/dev/null"

The recipe above is just a tool to delete all the mail from a user's inbox. Each message is retrieved from the mail server using IMAP, then disposed of into /dev/null. The mda keyword takes a command line as an argument into which each incoming message will be piped. If you work at an ISP, you may need something like this from time to time, if a user has gotten his or her mailbox into such a state that they just want to delete it and get a fresh start. It isn't terribly frugal with bandwidth, but it gets the job done.

This next recipe is useful for users who want conventional SMTP delivery of their retrieved mail, but don't want to run an SMTP server on their workstation:

skip centralserver.myisp.net protocol IMAP:
   user joeuser password passwordOFjoe
   mda "/usr/lib/sendmail -oem -f %F %T"

As you see here, the variable %F stands for the sender, and the %T is the recipient. This recipe uses sendmail to deliver messages, instead of relying on Fetchmail to route them to the local SMTP server. This is useful if you can't justify the security risk inherent in running an SMTP server, just to delivery your own mail.

The mda facility can be used for all sorts of things. The next recipe, for instance:

skip netmgt.myisp.net protocol IMAP:
   user statusbox password passwordOFstatus
   mda "/usr/local/bin/webstatusboard -mail"

uses a fictitious utility, "webstatusboard," to turn status mail messages into fodder for a Web-based status board. You could use a similar approach to generate support tickets or generate purchase-order requests.

With this article and the last one, we've covered quite a number of Fetchmail capabilities. There are still numerous ways to further organize your mail handling that we haven't touched on yet. For further information on the variety of Fetchmail recipes yet to be written, visit the Fetchmail home page.

Kevin Mullet is the vice president of Atomic Consulting, a Unix and network consulting business for small- and medium-size companies in the Dallas area. Kevin can be reached at kwm@themullets.net.

O'Reilly & Associates recently released (September 2000) Managing IMAP.

• Sample Chapter 9, Cyrus System Administration, is available free online.

• You can also look at the Table of Contents, the Index, and the Full Description of the book.

• For more information, or to order the book, click here.

Sponsored by: