Jump to music examples

A computer mouse is a wonderful gadget, no doubt about it. But as a musical instrument, it's pretty much a disaster. If you're serious about playing music with today's wonderful software instruments, you'll soon find yourself casting covetous glances at some of the peripheral hardware that's designed specifically for music. In this tutorial we'll look at a few of these hardware devices, explain their primary capabilities, and explore some of the musical results you can achieve with them.

Hardware controllers are available in price ranges to suit any budget, and with features from ultra-basic to ultra-slick. Whether you're making music with a laptop or in a large studio with multiple high-end computers, if you aren't already working with a hardware controller, it's a good bet you will be before long. Strictly speaking, you could probably achieve the same musical results with a mouse, but using a controller that's designed for music will be faster, easier, and more inspiring.

Many controllers, though by no means all of them, are piano-style keyboards. Even if you're not a whiz when it comes to playing the keys, you'll find a keyboard useful for entering notes on software synthesizers. Most music keyboards designed for computer use also have a few knobs or sliders designed to control other aspects of the sound.

Some controllers don't have keyboards at all, only knobs, sliders, and buttons. Some of these are designed for musicians, while others are meant primarily for recording engineers. (Of course, many of us wear both hats from time to time.) In this article we'll mainly be concerned with hardware intended for musicians. If you're a budding engineer or a producer, you may want to investigate the powerful hardware controllers built by Mackie, TASCAM, Behringer, JL Cooper, and other manufacturers.

Most musician-oriented controllers, whether or not they have keyboards, communicate with the computer using MIDI. Even if you're a hip-hop DJ, headbanging heavy metal guitar player, or folk music specialist who would rather have an appendectomy than use a MIDI synthesizer, you'll find it helpful to know a bit about MIDI in order to use a hardware controller for tasks like mixing and controlling effects onstage. I'll cover the details of MIDI in a future article; for a quick intro, see the "Meet MIDI" sidebar.

Korg Kontrol49 With full-size keys, a small joystick (upper left), and nine context-sensitive LCDs for data display, the Korg Kontrol49 is a class act. The lighted squares are velocity-sensitive drum pads.

The Physical Package

Meet MIDI

MIDI (the Musical Instrument Digital Interface) is a hardware and software system designed to control electronic musical instruments (including software-based instruments). For instance, the MIDI note-on message includes information about which note should be played (the note number) and how hard the key on the keyboard was struck (the key velocity). When a synthesizer or other MIDI instrument receives a note-on message, it starts playing a note. When it receives a note-off message with the same note number, it stops playing the note.

In addition to note-ons and note-offs, MIDI includes pitch bend messages, key pressure messages (also known as aftertouch), and more than 100 control change messages. MIDI defines 16 different channels, and messages on all 16 channels can travel down the same cable simultaneously, so a computer with a single MIDI output could control as many as 16 hardware synthesizers at the same time, each playing its own musical part.

Recent developments, including USB and Yamaha's mLAN protocol, are changing the way MIDI devices are cabled together and removing some of MIDI's speed and bandwidth limitations. But the structure of MIDI messages hasn't changed for more than 20 years, and isn't likely to. Some excellent MIDI tutorials are available online; try Borg.com, the MIDI Manufacturers Association, or Harmony Central.

A MIDI hardware controller typically has at least four knobs or sliders, and allows you to choose which type(s) of MIDI data they'll transmit. Some controllers offer 8, 16, or even more knobs or sliders. Some also have buttons, joysticks, or wheels that can be set up to transmit a variety of MIDI messages.

A controller box can be handy if you already have a MIDI keyboard and don't have much space to add hardware to your rig. Two of the boxes that lead the way are the Peavey PC 1600 and the Keyfax Phat Boy, which is no longer in production. Happily, several devices have stepped into the gap, including the Kenton Control Freak, which looks eerily like the 1600 thanks to its 16 sliders; the Kenton Spin Doctor, which uses knobs instead of sliders; and the Evolution UC33e, which sports a generous complement of 8 sliders and 24 knobs. There's a long list of available controllers at the online store AudioMIDI.com.

Keyboard controllers, sometimes called master keyboards, look like synthesizer keyboards but lack an internal sound generator. (To play sounds, you connect the keyboard to an external MIDI sound module or a computer running a software synthesizer.) Many recent synthesizer keyboards have enough sliders, knobs, and other input widgets on the front panel that they can double as software controllers. For example, the Yamaha Motif ES synthesizer includes four sliders as well as Play, Stop, Record, and Fast Forward/Rewind buttons, which makes it easy to remote-control your computer like a tape recorder. Dedicated keyboard controllers usually offer many more knobs and faders, though.

These days, keyboards built strictly as controllers almost always have USB connectors, allowing them to interface directly with computer-based music systems. If you're using a hardware synthesizer as a MIDI controller keyboard, you may be able to use USB, or you may need to connect the keyboard to your computer using a standard 5-pin MIDI cable. That will require a separate hardware box called a MIDI interface that hooks to your computer using USB or FireWire. (Yes, that's a Musical Instrument Digital Interface interface.)

Hardware controllers are a fertile field for designers with innovative ideas. The Keyfax Flat.Boy, for instance, has a touchpad that senses the vertical and horizontal movement of a fingertip, plus pressure. (See the demonstration videos.) The M-Audio Trigger Finger provides four sliders, eight knobs, and sixteen velocity-sensitive pads suitable for triggering percussion sounds. (See the demonstration video.) Don Buchla's Thunder, Lightning, and Marimba Lumina, none of which is still in production, offered both exotic hardware interfaces and unusual dimensions of software control.

Keyfax Flat.Boy The Keyfax Flat.Boy (designed by Mercurial Innovations) makes touch control as easy as X-Y-Z. See it in action here.

Venerable products, like the Yamaha WX5 wind controller, which uses a fingering system similar to a saxophone, are still available, and have a passionate following. There are numerous other "alternate controllers," shaped like guitars, hand drums, marimbas, trumpets, and even more exotic objects.

Yamaha WX5 A familiar saxophone fingering system and sensors for breath and lip pressure make the Yamaha WX5 MIDI controller inviting for reed players.

For that matter, guitarists can use a hexaphonic pickup and a converter box to play MIDI synthesizers directly from the guitar, while drummers can take advantage of inexpensive drum-mounted pickups or pricey digital drum kits. These devices, however, are designed mainly to trigger notes using a performance interface that's familiar to musicians, and usually don't support the real-time sound-shaping you can get with continuous controllers.

Usually, a MIDI controller lets you choose which type of MIDI message will be transmitted from each knob, slider, or button. After configuring the hardware, you may be able to save your knob/slider/button definitions as a preset. This feature can be quite handy if you work with several different synths (or other software) and want to create a controller template optimized for each of them. However, many music programs can be configured to "listen to" whatever MIDI data is available, so a hardware controller with only one set of definitions for its knobs and sliders can still be used in a variety of musical contexts without reprogramming it.

M-Audio Trigger Finger Its velocity- and aftertouch-sensing pads give the M-Audio Trigger Finger a more than passing resemblance to the classic Akai/Linn MPC drum machines, but this implementation adds knobs, sliders, and USB output. See it in action here.

The Feel Factor

Playing a musical instrument, even one that lives inside a computer, is a physical activity, so the physical feel of a controller can make a real difference. Here are some factors to think about when choosing a controller:

Wobbly versus solid. Knobs and sliders that wobble just don't feel as good as those that are solidly anchored in the panel. Also, I like sliders that have a bit of resistance, because that makes it easier to move them smoothly by small amounts.

Slider length. With a short slider, it's too easy to make sudden jumps in the sound, even when you don't mean to. The sliders on my Peavey PC 1600 have almost 2-1/2 inches of throw, which gives them plenty of room to move. In contrast, the pitch and mod wheels in the Korg Kontrol49 are unusually small and short-throw, making them a blemish on this otherwise capable keyboard.

Evolution X-Session The Evolution X-Session's compact size and crossfader make it handy for DJ moves.

Data jump. With short sliders and tiny joysticks, you may not be able to access all of the possible MIDI control change values between 0 and 127. While modern software synths can usually smooth out sudden jumps in controller data, "data jump" sucks. Any MIDI controller should be able to transmit a smooth contour of values. Many of them can't.

Smooth versus stepped knobs. Stepped (detented) knobs have the advantage that you can "tick" them precisely from one value to the next, and in any case MIDI data is always stepped, not continuous. But stepped knobs just don't feel as good for expressive sweeps.

Infinite versus single-turn knobs. A rotary encoder or infinite knob (turn it as many times as you like) sends "+1" messages when you turn it clockwise and "-1" messages when you turn it counterclockwise. In other words, it transmits relative values rather than absolute ones. With compatible software, that avoids the problem of having the parameter you're controlling jump to a radically different setting. Unless the controller has a display, though, such as a ring of LEDs around the knob, you won't be able to determine the parameter's value by looking at the knob. Some mixing control surfaces have infinite knobs, but on MIDI controller hardware single-turn knobs are far more common. This type of knob "pegs" at 7 o'clock on the left and at 5 o'clock on the right.

Dead zones. Joysticks and pitch bend wheels sometimes have a "dead zone" in the middle where no data is transmitted. The problem with a dead zone is that you can't play a smooth contour from low to high.

Behringer DJ The Behringer B-Control DeeJay BCD2000 provides audio I/O and comes with its own DJ software. The infinite-turn knobs at the top are surrounded by LEDs so you can gauge their value.

What They're Good For

Once you have a bank of configurable knobs, sliders, and/or buttons at your fingertips, you'll quickly find uses for them. Here are some of the most common applications:

Controlling synthesis parameters. You can assign a knob or slider to move the filter cutoff frequency (brightness), change the oscillator waveshape, increase or decrease the attack time of an envelope generator, add a sub-octave to the mix, add vibrato from an LFO, pan the sound from left to right...the list of possibilities is practically endless. (The MP3s in the "Saw Control" sidebar illustrate a few of them.) MIDI buttons can be used for sending program change messages, calling up new sounds, for example.

Controlling effect parameters. Synth effects, track effects, and even real-time effects designed for live use are also great candidates for MIDI control. Some effect parameters (such as the delay time in a delay effect) tend to "glitch" if they're changed while a signal is passing through the effect, causing pops and clicks, so some forethought may be required when assigning the knobs and sliders. Other effect parameters, such as wet/dry mix, feedback amount, and chorus depth, can usually be controlled in a smooth, sweet-sounding manner.

Assigning Controllers In AAS Ultra Analog, which I used to create the demos in the "Saw Control" sidebar, MIDI controller assignments can be edited in this dialog box.

Controlling a mix. If you have a long-throw slider available for each audio channel in a project, or even a bank of sliders that can quickly be switched from one group of tracks to another, you can use a MIDI controller very effectively for mixing your tracks. In the studio, this turns a MIDI controller with sliders into the equivalent of an old-school mixing console. Controlling multitrack mixes in live performance is also a possibility.

Controlling interactive software. Musicians who are exploring the outer limits of composition and performance use hardware controllers, often onstage, to interact with such software as Cycling '74 Max/MSP. This type of software can turn a flick of a knob into a flurry of notes, respond to a button-press by modulating to a new key, or add more randomly placed rests to a looping sequence as a slider is pulled down. Such musicians often find musical uses for unusual controllers, such as infrared sensors, graphics tablets, and video cameras.

Transport control. You can generally assign hardware buttons to send basic MIDI commands such as Start, Stop, and Continue. If your sequencer can respond to these messages, you'll be able to control playback remotely. Many hardware controllers can also be set up to send MIDI clock messages, and to assign the clock's tempo to a knob or slider. After setting the sequencer's transport to external MIDI clock sync, you can control the tempo remotely. Naturally, most of the engineer-oriented hardware controllers have dedicated transport buttons.

Graphic MIDI Controller Editing This window from Steinberg Cubase SX3 shows how multiple types of controller data can be displayed together in the piano-roll edit window. Note the pencil tool in the pitch-bend lane. (Click to enlarge.)

System Integration

Most hardware controllers are strictly transmitters: They neither know nor care what's at the receiving end of the message. But as software gets more sophisticated and musicians' needs mushroom, hardware manufacturers are responding by adding two-way communications, which foster a new level of system integration. Two devices I've worked with recently (and written about) illustrate this type of integration.

The Korg Kontrol49 keyboard has 8 knobs, 8 sliders, 16 buttons that can be used as percussion pads, a small joystick, and other features. It can be used with any music software, but mates well with Propellerhead Reason. As your Reason "rack" grows to include a number of software modules, each time you select a different module Reason instructs the Kontrol49 to reconfigure itself. Not only is this process automatic and transparent for the user, but because the Kontrol49 includes a bank of LCDs that display the current knob and slider assignments, you'll see the names of parameters displayed for Reason's Mälstrom, Subtractor, NN-XT, or other modules.

The Frontier Design TranzPort is not a MIDI device, but it's worth glancing at. For one thing, its small size may interest home studio owners who are playing and engineering at the same time. The TranzPort is a wireless controller that links with DAW (digital audio workstation) software from various manufacturers. In addition to Stop, Start, Record, and Rewind functions, it allows you to select a track and then modify the track's level and panning. Here again, communication is two-way: You'll actually see the track's current settings in the TranzPort's LCD, something that's seldom seen with MIDI slider boxes. You can define location markers from the TranzPort panel, and other functions are available depending on which DAW you use.

Frontier TranzPort No, the cables haven't been Photoshopped out. The Frontier Design TranzPort is wireless.

The Yamaha Motif ES synthesizer also provides system integration. When the Motif is put into Remote Control mode, its front-panel sliders and buttons transmit data to a DAW (digital audio workstation) as if the Motif were a more limited version of a Mackie Control Universal. Since Yamaha now owns Steinberg, the big music software company, a little crystal ball-gazing might suggest that tighter integration between the Motif and Steinberg Cubase could become a reality in the future.

Fancy Features

When trying to decide which hardware controller to buy, it's worth keeping several factors in mind, in addition to such things as portability and on-board preset memory, which we've touched on already:

Saw Control

This demonstration shows how a hardware controller can be used to shape a synthesizer line. I used an elderly Peavey PC 1600 controller for most of the shaping, but I performed pitch bending with the joystick on a Korg 01W ProX keyboard. (I then smoothed out the pitch data by hand in Steinberg Cubase SX3 using a mouse, owing to the Korg's poor sensing resolution.) I generated the sound itself with the Applied Acoustics Systems Ultra Analog software synth.

The rather boring synth line is played ten times, with no changes in the note data. The first time only the notes are heard, with one oscillator playing a sawtooth wave. Ugly, isn't it? With each repetition, a new controller contour is added. All of the parameters except pitch bend are under the control of MIDI Control Change (CC) data.

  1. Plain Sawtooth (192KB MP3)
  2. Pitch Bend (208KB MP3)
  3. Vibrato (LFO amount) (208KB MP3)
  4. Filter Cutoff (208KB MP3)
  5. Detuned Oscillator Added (216KB MP3)
  6. Reverb Added (wet/dry mix) (236KB MP3)
  7. Sub-Octave Added to Last Note (224KB MP3)
  8. Portamento (glide) Time Increased (220KB MP3)
  9. Attack Time Lengthened (220KB MP3)
  10. Modulated Highpass Filter Added (236MB MP3)

USB power. Some MIDI controllers can be powered directly over the USB bus. If you're using a laptop, however, that will drain your battery quickly, so you may want to shop for a model that also has its own power adapter or can run on batteries.

Keyboard size. The smallest, most portable keyboard controllers have short mini-keys, and may have only a two-octave keyboard. Such a keyboard is adequate for data entry, such as recording bass lines and MIDI drum parts, but is not ideal for live work, especially if you're a real keyboard player.

Multi-channel output. MIDI defines 16 channels for music data. The simplest controllers always transmit on a single channel. More powerful controllers let you assign knobs and sliders, individually or in groups, to various channels. If there's a keyboard, you may be able to split it into several zones, each of which transmits on a different channel. Depending on the features of your software, you may be able to control several devices at once even with a controller that transmits on only one channel, but multi-channel operation can come in handy.

Message choices. The MIDI messages available for controlling synthesis parameters include Control Change, Channel Aftertouch, Polyphonic Aftertouch, and Pitch Bend. A controller that limits you to Control Change data is less desirable than one that gives you more choices. When defining buttons, you'd like to be able to send not only program changes and notes but also Control Change messages such as Sustain Pedal. A few controllers let you define message strings to transmit with a single button-press. A message string could be a single system-exclusive message or a series of program changes on different channels, for example.

Buttons may also be able to operate in toggle mode, which is a cool feature: Alternate button presses send maximum and minimum values for the same controller, thus allowing you to turn a parameter on with one tap and turn it off again with a second tap.

If you're using a consumer-type General MIDI (GM), GS, or XG synth, look for a controller that will send GM-standard RPN (registered parameter number) messages, which let you access parameters such as filter cutoff and attack time. Not all hardware supports this more complex message type.

Control message scaling. Many controllers always give you a full-range response: Pulling the slider all the way down or turning the knob all the way to the left will send a control change message with a value of 0, and pushing the slider all the way up or turning the knob all the way to the right will send a value of 127. A more flexible system allows you to define the maximum and minimum values that will be sent. (You may also be able to define maximum and minimum values for the controller response in your software.)

If you can assign two or more message types to the same physical knob or slider, you can use message scaling and inversion to create complex sonic changes with a single gesture. For instance, by assigning volume (CC 7) data on two different MIDI channels to the same slider and inverting the data on one channel, you can turn the slider into a crossfader.

Software support. Some of the newer and more complex hardware controllers come bundled with editor software, which makes creating presets easier. Also, keep your eyes open for bundled software, such as an entry-level DAW, which may be included in the package.

Motorized faders. Some high-end controllers designed for mixing have motorized faders. With motorized faders, when you switch to a new set of functions (for example, assigning the faders to the levels of tracks 9-16 rather than 1-8) the faders will scoot up or down to display the correct settings.

Other functions. We're starting to see high-end MIDI controller hardware (such as the M-Audio Ozonic and Novation X-Station) that also includes audio I/O for the computer. Doubtless manufacturers have more clever ideas up their sleeves as well.

Novation X-Station The Novation X-Station has sliders, buttons, knobs, keys, touchpad, joystick--and it's an audio interface too.

New Dimensions

When MIDI was created in 1982, it was designed first and foremost to work with keyboard instruments. But if you're using your controller strictly to play keyboard parts, you're missing out on about 95 percent of the musical possibilities. With a little help from MIDI control change, aftertouch, and pitch bend data, you can shape individual notes or entire phrases. With a little help from hardware knobs and sliders, you can turn a static, boring keyboard part into a living, breathing musical experience.

Morrison Digital TrumpetDesigned in Australia, the Morrison Digital Trumpet makes no sound itself; it's a MIDI controller with numerous input sensors.

Expressive Controller Examples

Thanks to Matt Traum of Patchman Music, here are a few examples of how controllers can make a synthesizer sound come alive. Patchman specializes in wind-controller and breath-controller systems, and Traum himself is an accomplished performer, as you'll hear below. For more background on wind controllers, see Traum's extensive FAQ.

Related Reading

Audio Editing with Adobe Audition
By Richard Riley

Example 1. The Crumar Electronic Valve Instrument uses just a single sawtooth wave going through a filter that is controlled via the player's breath. Pitch is controlled with the player's thumb for what Traum calls "human-quality" vibrato. Compare this with the bland sawtooth wave in the "Saw Control" sidebar.

Example 2. Synth snobs often slam sample-playback instruments ("ROMplers") for their lack of character. This example shows the expression Traum can squeeze out of a such a synth, a Roland XV-5080, through a combination of custom settings, a wind controller, and a polished performance technique.

Example 3. A relatively new type of synthesis called physical modeling promises unprecedented expressivity by mating software models of the physics of acoustic instruments with extensive controller input. In this QuickTime video, Traum plays a modified Yamaha VL70-m physical modeling synth with a new MIDI trumpet controller called the Morrison Digital Trumpet (MDT). For more information and hundreds of MP3 examples, see Patchman's Turbo VL page.

Turbo VL and MDT Demo

Jim Aikin writes about music technology for a variety of publications and websites. His most recent books are Power Tools for Synthesizer Programming and Chords & Harmony.


Return to digitalmedia.oreilly.com