NISTNet: Emulating Networks on Your Own LAN

The role of the network administrator is broad and varied. While most of our time is spent with day-to-day operational tasks such as assigning IP addresses and configuring hosts and applications to use an existing network, we'll occasionally be called upon to do some more interesting things.

One of the more interesting and rewarding activities of the network administrator involves the planning and design of a brand-new network and the selection and configuration of network applications to run across it. Network design inevitably involves compromise, trading off one characteristic against another. Common trade-offs are link bandwidth against cost, or latency against loss, or bandwidth against latency.

Consider the case of a remotely located site that you wish to network-connect to a central site. A satellite-based connection might offer high bandwidth at a reasonable cost but offer latency times of over a second or more, compared to a terrestrial link which might offer lower bandwidth but with latency of the order of hundreds of milliseconds instead for much the same price. A dedicated point-to-point link might offer a virtually loss-less connection but at a higher cost than a frame relay connection that exhibits some datagram loss.

Choosing the best solution requires a detailed understanding of the end-user requirements and the network applications they will be using. Sometimes the best or only way of knowing what is the best network design is to actually try a few different ones and see which works best. Doing that is difficult and expensive unless you have a way of emulating the behavior of each of the options with your existing network. This is just one use of NISTNet.

NISTNet

NISTNet is a software package developed by Mark Carson of the North American National Institute of Standards and Technology (NIST) that allows network designers, application developers, and network researchers to pretty accurately emulate the performance conditions presented by a variety of TCP/IP networks and network paths.

NISTNet replaces the normal Linux IP forwarding mechanism with one that allows the network administrator to set and control the levels of each of a number of key network behaviors. These behaviors include datagram loss, datagram delay, delay variation, and the maximum available bandwidth in the forward and backward directions. Real networks display each of these characteristics. Consider the following examples:

• Busy or congested IP networks or network links with lots of errors often result in datagram loss.
• Carriage of IP datagrams across low speed or, in the case of satellite, long distance links results in high latency.
• The "bursty" nature of most IP traffic results in varying amounts of buffering of data occurring for a network connection over time, and this results in variation in the latency times.
• Network technologies such as ADSL and satellite provide more bandwidth in one direction than the other. Many high bandwidth LANs are connected via lower bandwidth links.

NISTNet allows each of these conditions to be intentionally introduced into an otherwise healthy network connection to allow administrators to emulate larger, more complex networks and observe the resulting behavior of network protocols and applications.

NISTNet provides both command line and X11-based userspace clients to control the operation of the kernel code.

Finding, compiling and installing NISTNet

The NISTNet software package is public domain software and so is freely available for all to use and abuse as they please. The software provides a number of loadable kernel modules and corresponding userspace applications to control the behavior of the modules. Kernel patch files are supplied that provide replacement timer code that allows finer clock resolution, which is essential for accurate emulation.

The most recent version of the NISTNet software is version 2.0 alpha, revision 3. This version compiles against the most current 2.0 and 2.2 version kernels. The NISTNet source code is available from the NISTNet web site.

The software is fairly easy to compile. No special libraries are required. There are only two small tricks. First, make sure that your kernel source is either in the /usr/src/linux directory or that there is a symlink pointing to where you have your kernel source. There are scripts that automate the application of the fast-timer patches that assume this to be the case. Second, be sure to remember to inspect the monitor/Imakefile file and make sure the OURXAWLIB define is pointing to the version of the Athena Widgets library you wish to use; the default is probably OK.

To compile the software you should run the Patch.Kernel script to patch the kernel. Recompile the kernel with the patches applied using the usual process. Be sure to select both of the new configuration options presented:

Kernel hacking  --->
    [*] Fast timer
    [*] Packet routine hacking

When the kernel is compiled and installed, return to the top level of the NISTNet source and run:

make
make install

This will compile both the text and X11 versions of the userspace programs as well as the new kernel modules, and will install them in fairly sensible locations on your machine. Debian users note: the kernel modules are installed in the /lib/modules/misc/, which is probably not what you want. You can easily move them into the appropriate misc/ subdirectory of the kernel you've compiled. The installation step will also create some special device files in the /dev/ directory that provide the interface that the userspace programs will use to talk to the kernel modules.