Cryptosystems: Configuring IPSec

In the next two articles, I'll demonstrate how to configure and troubleshoot an IPSec VPN on a FreeBSD system. When I first started configuring VPNs, I quickly discovered two things. First, there is more than one way to configure a VPN correctly . Along with my demonstration configuration files, I'll be including URLs to other IPSec tutorials, and you'll see for yourself that the syntax will vary slightly from tutorial to tutorial. Don't let the discrepancies bother you; instead, choose a configuration style that makes sense to you and results in a working VPN. Second, I found that I was typically left to my own devices when a VPN didn't work. There are few things in life more frustrating than following a set of instructions, only to discover that they don't work for your specific situation. Accordingly, I've included error messages which I have run across and my resolutions in the hopes that they might aid you in troubleshooting.

I'll be demonstrating a tunnel between two FreeBSD machines acting as VPN gateways. While my demonstration is specific to FreeBSD, it's possible to successfully apply the logic underlying the configurations to allow any system to access any IPSec VPN gateway. If this is your first VPN configuration, try it between two FreeBSD systems first. Once you get a handle on how a working tunnel operates and how to resolve the pitfalls you may come across, you'll be in good shape to experiment with other systems. The success of your experiments will depend upon the VPN gateway you are connecting to and how far the vendor has deviated from the IPSec standard by adding extra features.

In order for your FreeBSD system to use IPSec, you must first configure IPSec support into your kernel. If this is your first kernel build, you'll want to read through the kernel config section of the handbook first.

If you're unsure whether your kernel already has IPSec support, use this command:

sysctl -a | grep ipsec

If you don't get anything back, you need to recompile your kernel. If your tunnel is between two FreeBSD machines, both machines need IPSec support. This is how I configured my kernels. First, as the superuser, I copied the generic kernel configuration file to a file I called IPSEC:

cp /usr/src/sys/i386/conf/GENERIC /usr/src/sys/i386/conf/IPSEC

Then, using my favorite editor, I added the following three lines to the options section of /usr/src/sys/i386/conf/IPSEC:

options		IPSEC
options		IPSEC_ESP
options		IPSEC_DEBUG

While you're in the configuration file, ensure that this default line is still there; it should be unless you've removed it in a previous compile:

pseudo-device	gif

Once you've saved your changes:

cd /usr/src
make buildkernel KERNCONF=IPSEC && make installkernel KERNCONF=IPSEC

Normally I would reboot after a kernel is installed, but in this instance I'll wait until I've finished the rest of my configurations.

When you install IPSec support, you are installing the ability to use AH and ESP and to understand SADs and SPDs. However, you still have to create the policy that will be stored in that SPD and the SAs that will be stored in the SAD. It is possible to do all of this manually using a command known as setkey. But you may remember that it's better to use a key negotiation protocol to create those SAs for you on a regular basis. IKE, also known as ISAKMP, is the key negotiation protocol used by IPSec.

Currently, there are two possible ways to install IKE support on your FreeBSD system; both are found in the security section of the ports collection. The first is called racoon and the second is called isakmpd. I've found that the syntax used by racoon is easier for a novice to understand and there are more racoon resources available on the Internet, so I'll demonstrate its usage.

To install racoon:

cd /usr/ports/security/racoon
make install clean

Once this build is complete, I have the necessary ingredients for the VPN and can now concentrate on the VPN policy. That policy will be limited to the parameters supported by racoon, which can be found in man racoon.conf. I've summarized those parameters in the following table:

Remember, it is important to ensure that the policy you configure will match up on both peers. If the other VPN gateway isn't running racoon, you'll have to research that vendor's documentation to see which parameters are supported to ensure you configure the most secure policy that will result in a match on both peers.

I've decided to use the following policy:

Whenever I configure a VPN, I always write the policy parameters on a piece of paper which I can refer to as I configure the policy. Underneath the policy, I sketch out the two gateways I'll be configuring and clearly label their IP addresses:

Figure 1 -- a logical VPN diagram

You'll note that each gateway has 2 interface cards and 2 IP addresses. The external IP is the address used to connect to the Internet. The internal IP will usually be a private range address. Notice that I haven't given you my real external addresses, but have labeled them as A.A.A.A and B.B.B.B for the purposes of this article.

It only takes a few minutes to sketch out your network, but it might save you hours of troubleshooting. It is very easy to inadvertently place the wrong IP in the wrong configuration file if you don't have a sketch to refer to.

Related Reading Virtual Private Networks By Charlie Scott, Paul Wolfe, Mike Erwin Table of Contents Index Read Online--Safari Search this book on Safari:   Only This Book All of Safari Code Fragments only