O'Reilly    
 Published on O'Reilly (http://oreilly.com/)
 See this if you're having trouble printing code examples


Behavior Driven Development Using Ruby (Part 3)

by Gregory Brown
09/20/2007

We kicked off this three part exploration of Behavior Driven Development Using Ruby by diving into RSpec basics. With the knowledge of this comprehensive framework in hand, we walked through a practical example of using BDD practices to develop a simple application. By writing the specs first, we were able to use them to drive the design and ended up with nice, clean code as a result.

Of course, the more complex our needs get, the more we'll want to take advantage of advanced techniques that can help make our lives a little easier. In this final example, I'll cover a grab bag of RSpec features as well as some essential third party tools that will help make your life easier when writing your specs.

Wherever possible, I've used code based on the source package from the second part of this series. Here, I'll mainly be focusing on the specs, so please take a look back at the second article if you're curious about implementation details.

Bringing Specs Directly from the Whiteboard to the Text Editor

When writing test unit code, I've often put in tests that flunk, just to remind me to implement them later. Usually, this kind of code looks something like this:

class UserTest < Test::Unit::TestCase    

  def test_user_should_have_valid_email_address  
     flunk "write test verifying user's email address" 
  end

end

This gives an output something like this:

  1) Failure:
test_user_should_have_valid_email_address:5
write test verifying user's email address.

This works absolutely fine for its purpose, but definitely doesn't look like it is made for this sort of thing. RSpec handles this issue in a clever way, automatically detecting empty examples as not yet being implemented. Therefore, the same functionality could be mirrored like this:

describe "user" do    
  it "should have a valid email address" 
end

The output for something like this is quite nice, by comparison:

Finished in 0.03517 seconds

1 example, 0 failures, 1 pending

Pending:
user should have a valid email address (Not Yet Implemented)

With this in mind, we can actually flesh out a simple outline of what tests are needed for the user interface code that I snuck into the source package untested a couple weeks ago:

spec/interface_spec.rb
   
require File.join(File.expand_path(File.dirname(__FILE__)),"helper")
require "#{LIB_DIR}/interface" 

describe "An interface" do

  it "should prompt for players" 

  it "should prompt for grid size" 

  it "should be able to update board display" 

  it "should display a score board" 

  it "should prompt for a players move" 

end

This code, when run, yields a nice report of the work to be done:

PPPPP

Finished in 0.010666 seconds

5 examples, 0 failures, 5 pending

Pending:
An interface should prompt for players (Not Yet Implemented)
An interface should prompt for grid size (Not Yet Implemented)
An interface should be able to update board display (Not Yet Implemented)
An interface should display a score board (Not Yet Implemented)
An interface should prompt for a players move (Not Yet Implemented)

It goes without saying that any initial set of specifications is going to change drastically once you start fleshing it out, but it's really nice to be able to annotate your intentions like this, and encourages you to start writing specs right away.

Bringing Bug Reports Directly from the Tracker to Your Specs

Very few developers like "breaking the build" by checking in failing tests. The policy varies from project to project, but typically failing tests are not committed to the main line of development, or are at least commented out upon commit.

This is risky business, because it means that things can easily be forgotten. However, RSpec offers a way to mark bits of code as pending, which allows you to hide their failure messages but still have a note about them show up in your reports.

Here's a simple demonstration of how that works:

describe "the answer" do 

  before :each do
    @answer = 0
  end 

  it "should be 42" do
    pending("We need to wait 7.5 million years") do
      @answer.should == 42
    end
  end

end

When this code is run, our report looks like this:

P

Finished in 0.037488 seconds

1 example, 0 failures, 1 pending

Pending:
the answer should be 42 (We need to wait 7.5 million years)

The interesting thing is really that when someone comes along and fixes the problem, it will show up as a failure in your report, e.g., changing the setup so that @answer = 42 results in this output:

F

1)
'the answer should be 42' FIXED
Expected pending 'We need to wait 7.5 million years' to fail. No Error was raised.
/Users/sandal/Desktop/foo.rb:11:
/Users/sandal/Desktop/foo.rb:4:

Finished in 0.034342 seconds

1 example, 1 failure

In more practical usage, this construct might be a good way to mark code that is broken but perhaps has a workaround for it elsewhere in your system or code that has a ticket that should be closed when the bug is fixed.

When the examples pass, a failure will show up, and this will serve as a reminder that the pending() call can be removed and that some action might be necessary based on what the change was.

Though it's probably wise not to use this feature gratuitously, it is a much safer bet than leaving some commented out code laying around to eventually be forgotten.

Sharing Behaviors Between Contexts

It is common to find that a certain base configuration is used across several contexts, with minor tweaks in each to cover specific cases. It's also reasonably common to find that there are certain behaviors that are common to all contexts of an object.

If we take a look at our code that describes the Dots::Box class from our little game example, you can see a lot of patterns emerge between the contexts:

describe "A dots box" do   

  before :each do
    @box = Dots::Box.new 
  end

  it "should have 4 edges" do
    @box.edges.size.should == 4
  end

  directions do |dir|
    it "should have an #{dir} edge" do
      @box.edges[dir].should_not be_nil
    end 

    it "#{dir} edge should be :not_drawn by default" do
      @box.edges[dir].should == :not_drawn  
    end   

    it "#{dir} edge should be :drawn when draw_edge(#{dir.inspect}) is called" do
      @box.draw_edge(dir)
      @box.edges[dir].should == :drawn
    end
  end       

  it "should return nil for owner() by default" do
    @box.owner.should be_nil
  end

end  

describe "An incomplete dots box" do

  before :each do
    @box = Dots::Box.new   
  end   

  it "should return false for completed?" do
    @box.should_not be_completed
  end

  it "should not allow an owner to be set" do
    lambda { @box.owner = "Gregory" }.should raise_error(Dots::BoxIncompleteError)
  end

end  

describe "A completed dots box" do

  before :each do
    @box = Dots::Box.new
    directions { |dir| @box.draw_edge(dir) }
  end    

  it "should return true for completed?" do
    @box.should be_completed
  end 

  it "should allow an owner to be set" do
    @box.owner = "Gregory" 
    @box.owner.should == "Gregory" 
  end     

  it "should not allow an owner to be set more than once" do
    @box.owner = "Gregory" 
    lambda { @box.owner = "Joe" }.should raise_error(Dots::BoxOwnerAlreadySetError)
  end

end                               

describe "A positioned dots box at (0,0)" do

  require "set"                  

  before :each do
    @box = Dots::Box[0,0]    
  end  

  it "should have generated line coordinate tuples by compass direction" do
    @box.lines.should == lines   
  end  

  lines.invert.merge(Set[[10,1],[10,2]] => false).each do |edge, dir|

    it "should give #{dir.inspect} for edge?(#{edge.inspect})" do
       @box.edge?(edge).should == dir
    end

  end   

end

We are using a similar setup for each context, and really, most of the behaviors we check for in our "A dots box" context are common to all the boxes we will create, and should be verified. Luckily, RSpec has the notion of shared behaviors. These constructs allow us to share our setups and examples between several contexts. With some reworking, it's easy to refactor the code above to make use of this technique:

describe "A dots box", :shared => true do   

  before :each do
    @box = Dots::Box.new 
  end

  it "should have 4 edges" do
    @box.edges.size.should == 4
  end

  directions do |dir|
    it "should have an #{dir} edge" do
      @box.edges[dir].should_not be_nil
    end    
  end       

  it "should return nil for owner() by default" do
    @box.owner.should be_nil
  end

end  

describe "An incomplete dots box" do

  it_should_behave_like "A dots box" 

  directions do |dir|

    it "#{dir} edge should be :not_drawn by default" do
      @box.edges[dir].should == :not_drawn  
    end

    it "#{dir} edge should be :drawn when draw_edge(#{dir.inspect}) is called" do
      @box.draw_edge(dir)
      @box.edges[dir].should == :drawn
    end        

  end

  it "should return false for completed?" do
    @box.should_not be_completed
  end

  it "should not allow an owner to be set" do
    lambda { @box.owner = "Gregory" }.should raise_error(Dots::BoxIncompleteError)
  end

end  

describe "A completed dots box" do

  it_should_behave_like "A dots box" 

  before :each do
    directions { |dir| @box.draw_edge(dir) }
  end    

  # ... examples unchanged

end                                

describe "A positioned dots box at (0,0)" do

  require "set" 

  it_should_behave_like "A dots box"                 

  before :each do
    @box = Dots::Box[0,0]    
  end  

  # ... examples unchanged 

end

Notice that we're taking advantage of the fact that we're free to make use of the nested before :each calls. In our "A completed dots box" context, we simply make some changes to the constructed @box object from our shared context ("A dots box"), while in "A positioned dots box at (0,0)" we actually replace the object entirely. This is fairly powerful, because although we're dealing with a different object, our shared examples use the same variable name, so we can be sure that a box created with Dots::Box[0,0] works the same as one created with Dots::Box.new.

A quick run to generate the spec docs shows that each context is now also verifying its shared behaviors:

$ spec spec/box_spec.rb -f s

An incomplete dots box
- should have 4 edges
- should have an north edge
- should have an south edge
- should have an east edge
- should have an west edge
- should return nil for owner() by default
- should return false for completed?
- should not allow an owner to be set
- north edge should be :not_drawn by default
- north edge should be :drawn when draw_edge(:north) is called
- south edge should be :not_drawn by default
- south edge should be :drawn when draw_edge(:south) is called
- east edge should be :not_drawn by default
- east edge should be :drawn when draw_edge(:east) is called
- west edge should be :not_drawn by default
- west edge should be :drawn when draw_edge(:west) is called

A completed dots box
- should have 4 edges
- should have an north edge
- should have an south edge
- should have an east edge
- should have an west edge
- should return nil for owner() by default
- should return true for completed?
- should allow an owner to be set
- should not allow an owner to be set more than once

A positioned dots box at (0,0)
- should have 4 edges
- should have an north edge
- should have an south edge
- should have an east edge
- should have an west edge
- should return nil for owner() by default
- should have generated line coordinate tuples by compass direction
- should give :north for edge?(#<Set: {[1, 1], [0, 1]}>)
- should give false for edge?(#<Set: {[10, 1], [10, 2]}>)
- should give :south for edge?(#<Set: {[0, 0], [1, 0]}>)
- should give :east for edge?(#<Set: {[0, 0], [0, 1]}>)
- should give :west for edge?(#<Set: {[1, 1], [1, 0]}>)

This technique can really come in handy for making your code a bit more DRY, and making it easy to ensure that common behaviors that span different contexts are actually checked.

You can actually include several shared behaviors in a given context, and also include shared behaviors inside other shared behaviors, so this technique will scale to arbitrary complexity.

Clarifying Examples with Custom Matchers

Those who've written a fair bit of nontrivial Test::Unit code have probably created some custom assertions to clean up their code. It is possible to do the same thing in RSpec, via custom matchers.

Let's take a quick look at some of the code from our Dots::Box spec, where we're writing some reasonably ugly specs:

directions do |dir|

  it "#{dir} edge should be :not_drawn by default" do
    @box.edges[dir].should == :not_drawn  
  end

  it "#{dir} edge should be :drawn when draw_edge(#{dir.inspect}) is called" do
    @box.draw_edge(dir)
    @box.edges[dir].should == :drawn
  end        

end

Though it's not used in a ton of places, it'd be nice to say something like:

  @box.edges[dir].should be_drawn

We'd want this code to expand out to mean the same thing essentially as our original spec, but have more meaningful error messages and descriptions. By creating a simple object with a few hooks and then creating a Kernel method that initializes a matcher for us, we can do exactly that:

class BeDrawn
  def matches?(edge)
    @edge = edge
    @edge == :drawn
  end

  def description
    "be drawn" 
  end

  def failure_message
     "expected edge to be drawn but wasn't" 
  end                                          

  def negative_failure_message
    "edge was drawn but wasn't expected to be" 
  end
end

def be_drawn
  BeDrawn.new
end

With this code loaded, we can now write our specs like this:

directions do |dir|

  it "#{dir} edge should not be drawn by default" do
    @box.edges[dir].should_not be_drawn  
  end

  it "#{dir} edge should be drawn when draw_edge(#{dir.inspect}) is called" do
    @box.draw_edge(dir)
    @box.edges[dir].should be_drawn
  end        

end

Though this does reduce granularity of the tests a little bit (I'm no longer checking for :not_drawn), it actually captures the expected behavior a little better than our original code. It also lets me re-use this code as needed and only need to change the underlying comparison in one place if the underlying implementation changes.

Of course, we can actually go a little bit farther if we create a matcher with an argument:

class HaveDrawnEdge
  def initialize(direction)
    @direction = direction
  end

  def matches?(box)
    box.edges[@direction] == :drawn
  end

  def description
    "have #{@direction} edge drawn" 
  end

  def failure_message
    "expected #{@direction} edge drawn but wasn't" 
  end

  def negative_failure_message
    "#{@direcion} edge should not have been drawn" 
  end
end 

def have_drawn_edge(dir)
  HaveDrawnEdge.new(dir)  
end

We can now write specs that are really expressive, abstracting our core matching code even more:

directions do |dir|

  it "#{dir} edge should not be drawn by default" do
    @box.should_not have_drawn_edge(dir)
  end

  it "#{dir} edge should be :drawn when draw_edge(#{dir.inspect}) is called" do
    @box.draw_edge(dir)
    @box.should have_drawn_edge(dir)
  end        

end

As systems get bigger, custom matchers become a very handy way to avoid large and ugly looking examples that are prone to maintenance headaches. Since they're literally basic Ruby objects that just have a certain interface they need to implement, they're very simple to work with and can easily be stashed away in a helper file to be used across your project's specs.

Though there are lots more tricks to learn in RSpec, it's worth taking the time now to talk about a few well integrated third party tools that help you make sure your specs are doing what you think they are.

Checking for Missing Specs with RCov

An easy way to identify potential problem areas in your code is to find areas that aren't even being executed by your specs. RCov is a simple and powerful code coverage tool that does this kind of analysis, and though it can also be used with Test::Unit, it is trivial to get it working with RSpec as well.

Here's the basic Rakefile I'm using for the Dots game code:

require 'rake'
require 'spec/rake/spectask'

desc "Run all examples" 
Spec::Rake::SpecTask.new('examples') do |t|
  t.spec_files = FileList['spec/*_spec.rb']
end          

desc "Run all examples with RCov" 
Spec::Rake::SpecTask.new('examples_with_rcov') do |t|
  t.spec_files = FileList['spec/*_spec.rb']
  t.rcov = true
  t.rcov_opts = ['--exclude', 'spec']
end

Now, when I run rake examples_with_rcov, I get nice HTML reports that look like this:

figure
Figure 1.

It's pretty easy to see which classes were and were not developed using BDD. The neat thing is that RCov does a line-by-line output of your code, highlighting lines that were not run in red:

figure
Figure 2.

Of the interface implementation, this was the easiest method I could pick to layer some specs on top of. Going back to the outline we started earlier, something like this is enough to get us coverage:

describe "An interface" do

  it "should prompt for players" 

  it "should prompt for grid size" 

  it "should be able to update board display" 

  it "should display a score board" do
    game = mock("game")
    players = %w[Greg Pete Paul] 
    game.should_receive(:players).and_return(players)
    players.each do |p|
      game.should_receive(:score).with(p).and_return(0)
    end 

    @interface.score_board(game).should == " Greg: 0 Pete: 0 Paul: 0"    
  end

  it "should prompt for a players move" 

end

Even though we're using mocks to avoid passing in a real Dots::Game object, you can see that this simple example is enough to at least make sure this code is running, and working as expected:

figure
Figure 3.

With each small change, our overall coverage starts to look better:

figure
Figure 4.

Though RCov is very handy for letting you know how much you suck, it can't do much for telling you whether your specs are of any quality. We'll now take a look at the next step you can take to beat up your specs once you know they're at least running your code.

Checking for Dumb Specs with Heckle

Heckle works by trying to break your code assuming that it will result in failing specs. Although this isn't a new test verification technique, it's definitely an interesting one. The assumption is that if you can mutate parts of your implementation code without creating failure, that code is either doing nothing, or your specs are incomplete.

The spec command integrates with Heckle when it is installed. It allows you to type in a module name or method name and see if Heckle can break your specs. With this sample run, you can see that I've left some details out of the Dots::Game specs:

$ spec spec/game_spec.rb -H Dots::Game#start
.........

Finished in 0.027817 seconds

9 examples, 0 failures

**********************************************************************
***  Dots::Game#start loaded with 2 possible mutations
**********************************************************************

2 mutations remaining...
1 mutations remaining...

The following mutations didn't cause test failures:

def start
  @players = interface.get_players
  @turn = -8
  @score = Hash.new(0)
  rows, cols = interface.get_grid_size
  @grid = Dots::Grid.new(rows, cols)
  interface.update_display(self)
end

Here's the original code:

def start
  @players = interface.get_players
  @turn = 0    
  @score = Hash.new(0)
  rows,cols = interface.get_grid_size 
  @grid = Dots::Grid.new(rows,cols)
  interface.update_display(self)  
end

What Heckle is telling us is that we never specify that turn must start at 0. Though for the moment this isn't really essential to the way the game operates, the fact that it works with a non-zero initial state was mildly surprising to me. A simple spec is sufficient to get rid of this ambiguity once and for all:

describe "A newly started game" do

  # ...

  it "should start at turn 0" do
    @game.turn.should == 0
  end   

  # ...
end

Now, whenever we run heckle on Game#start, it reports back 'no mutants'

$ spec spec/game_spec.rb -H Dots::Game#start
..........

Finished in 0.029037 seconds

10 examples, 0 failures

**********************************************************************
***  Dots::Game#start loaded with 2 possible mutations
**********************************************************************

2 mutations remaining...
1 mutations remaining...
No mutants survived. Cool!

I tend to look at tools like this as a good way to investigate areas that you think might already have some flaws. I wouldn't count on them as a substitute for careful consideration of your spec quality, but they definitely come in handy for shaking things down when it's necessary.

It's worth mentioning that like RCov, Heckle is by no means specific to RSpec. It works just fine with Test::Unit as well. It definitely does come in handy when you're focusing more on interactions however, since it will pick up potential issues with state corruption that might not be exposed until later on down the line when you're using BDD to drive your project.

Bringing BDD to Test::Unit via test/spec

So far, this entire discussion has been RSpec-centric, and for good reason: If you want to practice BDD in Ruby and are able to start with a fresh canvas, it is very likely that RSpec is the right tool for the job. You've seen in this article that it integrates just as well with tools like RCov and Heckle as Ruby's Test::Unit does, provides a comprehensive mocking framework, and has tons of room for extensibility.

However, there is a very real issue worth mentioning: Ruby's Test::Unit is part of the standard distribution, and RSpec is not. This means that there is a ton of test code out there, along with helpers and extensions, that is incompatible with RSpec and would be quite time consuming to rewrite in one fell swoop.

If you're working on a project that needs to maintain Test::Unit compatibility, but you'd like to introduce the BDD friendly syntax and constructs that you find in RSpec to your test suites, Christian Neukirchen's test/spec is what you're looking for.

In fact, if you take a look at our RSpec code for Dots::Grid, you'll see that only a couple minor changes need to be made to get them running under test/spec.

grid_spec.rb (converted to test/spec)
describe "An empty dots grid" do                                  

  # all else the same

  it "should throw an error when connecting non-adjacent dots" do        
    # === RSpec Style ===
    #lambda { @grid.connect([0,0],[0,5]) }.should raise_error(Dots::InvalidEdgeError) 

    # === test/spec style ===
    should.raise(Dots::InvalidEdgeError) {
      @grid.connect([0,0],[0,5])
    }
  end  

end

describe "A drawn on dots grid" do  

  # all else the same

  it "should return a set with a box when connect() completes one box" do  
    result = @grid.connect [1,1], [1,0]
    result.size.should == 1
    result.each do |b|         

      # === RSpec Style ===
      #b.should be_an_instance_of(Dots::Box)   

      # === test/spec Style ===
      b.should.be.an.instance_of?(Dots::Box) 

    end   
  end 

  it "should return a set of two boxes when connect() completes two boxes" do
    @grid.connect [1,0], [2,0]
    @grid.connect [2,0], [2,1]
    @grid.connect [2,1], [1,1]      

    result = @grid.connect [1,1], [1,0]

    result.size.should == 2

    result.each do |b|         

      # === RSpec Style ===
      #b.should be_an_instance_of(Dots::Box)   

      # === test/spec Style ===
      b.should.be.an.instance_of?(Dots::Box) 

    end 

  end    

end

Despite the change in framework, almost all the semantics are preserved. The fundamental difference is that test/spec is implemented on top of Test::Unit, allowing you to mix it in with ordinary TDD code. If you've already created helpers that are based on the assert_* family of friends, there is nothing to stop you from using them. You can even define contexts side by side with test cases, and use your normal test runners as desired.

Although test/spec does include many of the RSpec niceties, even things like spec docs generation, it is definitely a less all-encompassing tool. You'd have to glue together some of the missing bits, such as a mocking framework, by downloading other packages such as FlexMock or Mocha. Even still, if migrating to RSpec isn't an option, test/spec is about the best compromise that's available in Ruby these days.

Putting It All into Practice

I hope that this longish exploration of BDD using Ruby has been helpful to you. Though I think that we've covered enough of the essentials to get you started writing specs and using some helpful tools to make your job easier, you may wish to learn more about the principles behind BDD and the philosophies that go with them. A lot of this background information can be found on behaviour-driven.org, as well as strewn across the blogosphere.

Of course, like any other software development practice, the best way to figure out to what extent these tools and techniques will be useful for you is to hack on some projects and see how they work out. Hopefully you'll find some fun and better code at the other side of the tunnel.

Gregory Brown is a New Haven, CT based Rubyist who spends most of his time on free software projects in Ruby. He is the original author of Ruby Reports.


Return to the Ruby Blog.

Copyright © 2009 O'Reilly Media, Inc.