advertisement

Print

Behavior Driven Development Using Ruby (Part 3)
Pages: 1, 2, 3, 4, 5

Checking for Dumb Specs with Heckle

Heckle works by trying to break your code assuming that it will result in failing specs. Although this isn't a new test verification technique, it's definitely an interesting one. The assumption is that if you can mutate parts of your implementation code without creating failure, that code is either doing nothing, or your specs are incomplete.

The spec command integrates with Heckle when it is installed. It allows you to type in a module name or method name and see if Heckle can break your specs. With this sample run, you can see that I've left some details out of the Dots::Game specs:

$ spec spec/game_spec.rb -H Dots::Game#start
.........

Finished in 0.027817 seconds

9 examples, 0 failures

**********************************************************************
***  Dots::Game#start loaded with 2 possible mutations
**********************************************************************

2 mutations remaining...
1 mutations remaining...

The following mutations didn't cause test failures:

def start
  @players = interface.get_players
  @turn = -8
  @score = Hash.new(0)
  rows, cols = interface.get_grid_size
  @grid = Dots::Grid.new(rows, cols)
  interface.update_display(self)
end

Here's the original code:

def start
  @players = interface.get_players
  @turn = 0    
  @score = Hash.new(0)
  rows,cols = interface.get_grid_size 
  @grid = Dots::Grid.new(rows,cols)
  interface.update_display(self)  
end

What Heckle is telling us is that we never specify that turn must start at 0. Though for the moment this isn't really essential to the way the game operates, the fact that it works with a non-zero initial state was mildly surprising to me. A simple spec is sufficient to get rid of this ambiguity once and for all:

describe "A newly started game" do

  # ...

  it "should start at turn 0" do
    @game.turn.should == 0
  end   

  # ...
end

Now, whenever we run heckle on Game#start, it reports back 'no mutants'

$ spec spec/game_spec.rb -H Dots::Game#start
..........

Finished in 0.029037 seconds

10 examples, 0 failures

**********************************************************************
***  Dots::Game#start loaded with 2 possible mutations
**********************************************************************

2 mutations remaining...
1 mutations remaining...
No mutants survived. Cool!

I tend to look at tools like this as a good way to investigate areas that you think might already have some flaws. I wouldn't count on them as a substitute for careful consideration of your spec quality, but they definitely come in handy for shaking things down when it's necessary.

It's worth mentioning that like RCov, Heckle is by no means specific to RSpec. It works just fine with Test::Unit as well. It definitely does come in handy when you're focusing more on interactions however, since it will pick up potential issues with state corruption that might not be exposed until later on down the line when you're using BDD to drive your project.

Bringing BDD to Test::Unit via test/spec

So far, this entire discussion has been RSpec-centric, and for good reason: If you want to practice BDD in Ruby and are able to start with a fresh canvas, it is very likely that RSpec is the right tool for the job. You've seen in this article that it integrates just as well with tools like RCov and Heckle as Ruby's Test::Unit does, provides a comprehensive mocking framework, and has tons of room for extensibility.

However, there is a very real issue worth mentioning: Ruby's Test::Unit is part of the standard distribution, and RSpec is not. This means that there is a ton of test code out there, along with helpers and extensions, that is incompatible with RSpec and would be quite time consuming to rewrite in one fell swoop.

If you're working on a project that needs to maintain Test::Unit compatibility, but you'd like to introduce the BDD friendly syntax and constructs that you find in RSpec to your test suites, Christian Neukirchen's test/spec is what you're looking for.

In fact, if you take a look at our RSpec code for Dots::Grid, you'll see that only a couple minor changes need to be made to get them running under test/spec.

grid_spec.rb (converted to test/spec)
describe "An empty dots grid" do                                  

  # all else the same

  it "should throw an error when connecting non-adjacent dots" do        
    # === RSpec Style ===
    #lambda { @grid.connect([0,0],[0,5]) }.should raise_error(Dots::InvalidEdgeError) 

    # === test/spec style ===
    should.raise(Dots::InvalidEdgeError) {
      @grid.connect([0,0],[0,5])
    }
  end  

end

describe "A drawn on dots grid" do  

  # all else the same

  it "should return a set with a box when connect() completes one box" do  
    result = @grid.connect [1,1], [1,0]
    result.size.should == 1
    result.each do |b|         

      # === RSpec Style ===
      #b.should be_an_instance_of(Dots::Box)   

      # === test/spec Style ===
      b.should.be.an.instance_of?(Dots::Box) 

    end   
  end 

  it "should return a set of two boxes when connect() completes two boxes" do
    @grid.connect [1,0], [2,0]
    @grid.connect [2,0], [2,1]
    @grid.connect [2,1], [1,1]      

    result = @grid.connect [1,1], [1,0]

    result.size.should == 2

    result.each do |b|         

      # === RSpec Style ===
      #b.should be_an_instance_of(Dots::Box)   

      # === test/spec Style ===
      b.should.be.an.instance_of?(Dots::Box) 

    end 

  end    

end

Despite the change in framework, almost all the semantics are preserved. The fundamental difference is that test/spec is implemented on top of Test::Unit, allowing you to mix it in with ordinary TDD code. If you've already created helpers that are based on the assert_* family of friends, there is nothing to stop you from using them. You can even define contexts side by side with test cases, and use your normal test runners as desired.

Although test/spec does include many of the RSpec niceties, even things like spec docs generation, it is definitely a less all-encompassing tool. You'd have to glue together some of the missing bits, such as a mocking framework, by downloading other packages such as FlexMock or Mocha. Even still, if migrating to RSpec isn't an option, test/spec is about the best compromise that's available in Ruby these days.

Putting It All into Practice

I hope that this longish exploration of BDD using Ruby has been helpful to you. Though I think that we've covered enough of the essentials to get you started writing specs and using some helpful tools to make your job easier, you may wish to learn more about the principles behind BDD and the philosophies that go with them. A lot of this background information can be found on behaviour-driven.org, as well as strewn across the blogosphere.

Of course, like any other software development practice, the best way to figure out to what extent these tools and techniques will be useful for you is to hack on some projects and see how they work out. Hopefully you'll find some fun and better code at the other side of the tunnel.

Gregory Brown is a New Haven, CT based Rubyist who spends most of his time on free software projects in Ruby. He is the original author of Ruby Reports.


Return to the Ruby Blog.