O'Reilly    
 Published on O'Reilly (http://oreilly.com/)
 See this if you're having trouble printing code examples


Rails Testing: Not Just for the Paranoid

by Gregory Brown
06/07/2007

It's true that writing tests for your applications means writing more code. However, unless you are excellent at writing bug-free software that never needs to change or be worked on by anyone else, it's safe to say that testing is not optional, but essential while working with Rails.

Ruby is a very adaptive and malleable language. Rails pushes this to its functional limit and introduces a ton of new behaviors: many helpful, some surprising. Without tests to ensure that your application is behaving as you intended it to, it's a near promise that you will get bitten.

The real issue most people have with testing is not that they think it's a bad idea, but that it often means a whole lot more configuration, a whole lot more to learn, and lots of things that smell like extra work. Folks who have been in that crowd will be pleasantly surprised when working with Rails.

I'll start by giving an overview of the testing facilities built into the framework, and then we'll work with them hands-on by layering some new tests and functionality into Tasty, the mini app we've been building across the Understanding ActiveRecord two-part article.

If you're not already partially familiar with Rails testing, you'll want to read the section on test/unit from Understanding ActiveRecord: A Gentle Introduction to the Heart of Rails (Part 2). You'll also want to grab the source for Tasty so you can follow along with the rest of this article.

The Rails Testing Toolbox

Rails offers three kinds of testing: unit tests, functional tests, and integration tests. All three have important roles to play in your application development, and when used properly can weave together a very solid safety net for development.

Unit tests

If you've written some Ruby before, you may already be familiar with test/unit, the built-in unit testing framework. This is a general purpose tool that is quite similar to some of the other xUnit derivatives found in many other languages including Java, C++, Perl, Haskell and probably countless others.

In the context of Rails, unit tests are meant primarily to cover the domain logic in your models, which include things like validations, calculations, search methods, and any other interesting functionality that your models implement. Since Rails sets up a database just for tests, you can test complex interactions in the same type of environment your application actually will run in, without worrying about damaging live data.

Functional tests

Some controllers are bound to be bland, but most of the time, you're going to have some sort of interesting logic that you'd want to test. Functional tests provide a way to verify that the actions for a single controller are working as expected, and allow you to do things such as post data to a specific action and verify the correct response is returned.

If your app is written cleanly enough and there is little or no logic in your views, functional tests can provide pretty solid coverage of your controllers and their interactions with any associated models.

Integration tests

Between units and functionals, the components of your application will be pretty well tested in isolation. Still, in practice any given session with a Rails application will span across several models and controllers. Integration tests provide a way to test those kinds of interactions. Essentially, an integration test is written at the story level, allowing you to verify the correct behavior of your application for a given use case.

For example, an integration test might cover something like "Joe logs in and creates a new Entry, and then Sue checks to see if the Entry shows up in the listing". Keep an eye out for something similar when we begin layering integration testing into the Tasty app in a bit.

Solidifying your models with unit tests

Rails automatically lays out the boiler plate for your tests when you generate models.

For example, when you type script generate model foo, the following unit test related files are generated:

test/unit/foo_test.rb
require File.dirname(__FILE__) + '/../test_helper'

class FooTest < Test::Unit::TestCase
  fixtures :foos

  # Replace this with your real tests.
  def test_truth
    assert true
  end
end
test/fixtures/foos.yml
# Read about fixtures at http://ar.rubyonrails.org/classes/Fixtures.html
one:
  id: 1
two:
  id: 2

Though the boilerplate is neat because you get to say "Cool, I didn't have to write that code", it's not really worth anything without some good tests.

What parts of my models need testing?

According to the folks who take test driven development seriously, the only features that need testing are the features that need to work. Though this is obviously a bit facetious, it's fairly close to the truth.

To break it down into a few categories, you will definitely want to test the following components of your models:

The Entry model from Tasty has several components that fall into these categories:

app/models/entry.rb
class Entry < ActiveRecord::Base

  validates_uniqueness_of :url

  belongs_to :user
  has_many :taggings
  has_many :tags, :through => :taggings

  def created_date
    created_at.strftime("%Y.%m.%d")
  end

  def updated_date
    updated_at.strftime("%Y.%m.%d")
  end

 # Adds a tag with the given name, if it's not already present
  def tag_as(tagname)
    unless tagged_as?(tagname)
      tags << Tag.find_or_create_by_name(tagname)
    end 
  end

  # True if tags include a Tag with the given name, False otherwise
  def tagged_as?(tagname)
    tag_names.include?(tagname)
  end

  # returns a list of tag names
  def tag_names
    tags.map(&:name)
  end

  protected

  def validate
    if short_description =~ /rube goldberg/i
      errors.add("short_description", "can't include references to Rube")
    end
  end

end

We'll start with the validation tests, and then move on to some of the other functions in this model, showing several different tests and how they work.

Testing validations

Since the purpose of a validation is to maintain data integrity, it's very important to test that they work properly. If you think about it, one intuitive way of testing a validation is to attempt saving invalid data and then to ensure it is handled correctly. That's exactly what we'll do, so let's start by ensuring our unit tests know how to fail (proving that they're hooked up). We also won't be using fixtures for this set of tests, so you can remove that line too.

test/unit/entry_test.rb
require File.dirname(__FILE__) + '/../test_helper'

class EntryTest < Test::Unit::TestCase
  def test_validates_unique_url
    flunk "Test failed as expected" 
  end
end

If all goes well, running rake test should give you something like this:

  1) Failure:
test_validates_unique_url(EntryTest) [./test/unit/entry_test.rb:5]:
Test failed as expected.

This proves that test_validates_unique_url is being called, which means we can replace it with a real test.

test/unit/entry_test.rb
require File.dirname(__FILE__) + '/../test_helper'

class EntryTest < Test::Unit::TestCase
  def test_validates_unique_url

    # Add an Entry to the DB so we have something to compare against
    base = Entry.create(:url => "http://rubyreports.org")

    assert_valid base

    e = Entry.new(:url => "http://rubyreports.org")

    # entry has an identical url, so we expect it to not be valid
    assert(!e.valid?, "Should not save entry unless url is unique")
    assert(e.errors.invalid?(:url), "Expected an error for duplicate url")
  end
end

Running the tests again, you should see that they pass. If you're paranoid, go ahead and remove the validates_uniqueness_of call from your model, and watch the tests fail.

Why no fixtures?

In the User tests for Tasty, I showed how to use fixtures just because its inevitable that you'll encounter them while working with Rails. However, it turns out that they've got a few sticky spots, and a lot of times, you simply don't need them.

By just explicitly calling Entry.new and Entry.create, it's very clear what our tests are checking for, and it's also reflecting how the code will actually be used, rather than relying on the implementation details of fixtures.

Making ugly assertions pretty via test_helper.rb

At the very top of your test, you see that rails requires the test_helper file. This is used for providing functions to simplify your tests and make them cleaner. One of the most common ways of doing this is to create custom assertions.

For example, it would be nice to have an assert_not_valid to match the assert_valid call in our tests. This is very easy to write:

test/test_helper.rb
# other code omitted
# add more helper methods to be used by all tests here...

  def assert_not_valid(model,msg="record was valid but shouldn't be")
    assert_block(msg) { !model.valid? }
  end

# ...

This is using test/unit's assert_block method to display the given message when the block returns false, and otherwise, pass the test if the block returns true.

This means that our old assertion:

assert !e.valid?, "Should not save entry unless url is unique"

now would look like this:

assert_not_valid e, "Should not save entry unless url is unique"

Custom assertions can become arbitrarily complex, and help keep your tests easily readable and focused on the actual things you are trying to verify. Since they are shared between all your tests, you can begin to establish a very high level set of assertions if needed.

If you want to play with this a little more, you might want to create something like an assert_invalid_field method that would work as shown below, but I'll leave the implementation details to you.

  assert_invalid_field :url, :for => e

What About That Stuff in validate()?

You'd also need to test Entry's custom validation, as clearly no one should be creating entries mentioning Rube Goldberg. The reason I didn't give this special treatment in the discussion of testing validations is that you handle it exactly the same way:

  def test_validates_rube_goldberg_check
    e = Entry.new(:url => "google.com", :short_description => "Nothing")
    assert_valid e

    f = Entry.new(:url => "apple.com",
                  :short_description => "Has Rube Goldberg")

    assert_not_valid f, "Should not save entry" 
    assert f.errors.invalid?(:short_description),
           "Expected an error for Rube reference" 
  end

Testing domain logic

We've also got a few other things that need testing in our Entry model, namely the tagging convenience methods and the date formatting methods. You essentially want to use your tests to define how you expect these methods to be used, and how you expect them to behave. The result is a fairly clear form of organic, self-verifying documentation:

test/unit/entry_test.rb
  # ...

  def test_tagging
    # needs to be put in DB because we're dealing with associations
    e = Entry.create(:url => "http://smurfs.com")
    e.tag_as(:foo)

    assert e.tagged_as?(:foo)
    assert !e.tagged_as?(:bar)

    assert_equal [:foo], e.tag_names

    e.tag_as(:bar)
    assert_equal [:foo,:bar], e.tag_names
  end


  def test_date_format
    e = Entry.create(:url => "http://snakesonaplane.com")
    assert_match /\A\d{4}\.\d{2}\.\d{2}\Z/, e.created_date
    assert_match /\A\d{4}\.\d{2}\.\d{2}\Z/, e.updated_date
  end

  # ...

I'm hoping that the above is easy enough to read, even for beginners. It's worth noting that I use a pattern to match my date format rather than use a specific date. I trust Rails to populate the underlying fields correctly, I just want to make sure the format is as expected.

By now it should be fairly clear that you can rather easily cover the use cases for your model. Once you write the tests once, they will continue to protect you down the line. Units will save you tremendous effort during refactoring, as they can quickly lead you to the source of various problems, and to a concise set of use cases that are easy to read.

Take control with functional tests

While it's true that your data model is often going to be the heart of your application, they're not the only thing that needs testing. Since controllers are the vital link between your models and views, they're just as test-worthy as a model. Luckily, it's pretty easy to handle the most important cases.

Tasty already has two controller actions we can test, the index pages for both entries and tags. When we generated those controllers, Rails already generated functional tests.

These default tests are already successfully doing nothing, but let's change them to be a little more useful.

The following set of tests checks to make sure that we can simply hit the tags index, whether or not any tags exist in the database:

test/functional/tags_controller_test.rb
require File.dirname(__FILE__) + '/../test_helper'
require 'tags_controller'

# Re-raise errors caught by the controller.
class TagsController; def rescue_iction(e) raise e end; end

class TagsControllerTest < Test::Unit::TestCase
  def setup
    @controller = TagsController.new
    @request    = ActionController::TestRequest.new
    @response   = ActionController::TestResponse.new

    Tag.destroy_all
  end

  def test_index_non_empty_tags
    Tag.create(:name => "foo")
    Tag.create(:name => "bar")

    get :index
    assert_response :success

    # assigns() lets you access instance variables assigned by the controller.
    tagnames = assigns(:urls_for_tags).keys
    %w[foo bar].each { |n| assert tagnames.include?(n) }
  end

  def test_index_empty_tags
    get :index
    assert_response :success
  end
end

You'll notice that I call Tag.destroy_all in my setup. This is a workaround for the fact that when you run your unit and functional tests together, it seems as if the fixture data from the unit tests wants to linger. Another option would be to run rake test:functional instead of just rake, but I'm extremely lazy.

Dealing with forms

Though it's helpful to ensure you can at least successfully call an action, usually you'll be more interested in testing out form handling. We're going to design a simple Entry search feature test-first. Writing tests before code can be a very powerful technique, as it requires you to think about design right away rather than as an afterthought.

Let's start by setting up a test that makes sure that you can access a search action:

test/functional/entries_controller_test.rb
def test_search
  get :search
  assert_response :success
end

If you run the tests now, you'll notice this generated an error:

ActionController::UnknownAction: No action responded to search

In order to make this pass, we need to add an action to our entries controller, for now it doesn't need to do anything:

app/controllers/entries_controller.rb
class EntriesController < ApplicationController
  def index
    @entries = Entry.find(:all)
  end

  def search

  end
end

If you've already ran your tests after adding this chunk of code, you'll see that we're missing a template. For now that also doesn't need to do anything, it just needs to have an empty file at app/views/entries/search.rhtml.

Once you have that, the tests should be all green, and that means we can make them more interesting. This test adds the expectation that if we pass a parameter for a tag that doesn't exist, it will stick a notice in the flash object.

 def test_search
    get :search
    assert_response :success

    get :search, :tag => "not_here" 
    assert_equal "No Entries With Tag: not_here", flash[:notice]
  end

Here's the code that makes that pass:

def search
  if params[:tag]
    @entries = Tag.entries_by_name(params[:tag])
    unless @entries
      flash[:notice] = "No Entries With Tag: #{params[:tag]}" 
    end
  end
end

Now we only need to deal with one more case, successful search. Here's our full test:

def test_search
  get :search
  assert_response :success

  get :search, :tag => "not_here" 
  assert_equal "No Entries With Tag: not_here", flash[:notice]

  e = Entry.create(:url => "apple.com")
  f = Entry.create(:url => "alsoapple.com")
  e.tag_as("apple")
  f.tag_as("apple")

  get :search, :tag => "apple" 
  assert_equal "Showing 2 Matches For Tag: apple", flash[:notice]
end

This code will make it pass:

def search
  if params[:tag]
    @entries = Tag.entries_by_name(params[:tag])
    if @entries
      flash[:notice] = "Showing #{@entries.length} Matches For Tag: #{params[:tag]}" 
    else
      flash[:notice] = "No Entries With Tag: #{params[:tag]}" 
    end
  end
end

Time to move on

You're most of the way to having a functional little tag search together. It's not particularly elegant, but will suffice for showing some functional testing in action. I'll leave it as an exercise to the reader to hook up a proper view for this, but include my minimalistic one in the source package for Tasty.

The key thing to take away from functional tests is that they allow you to verify that your application is responding as expected to various requests, and that they map to your controller actions, providing a safety net for them and making it easier to refactor things down the line.

Integration testing, for the high-level stuff

Getting an incredibly realistic use case for integration testing into Tasty at this point would be tricky. The reason for this is that integration tests often take the form of "big picture" user stories. An ideal case might be something like "James logs in, James purchases the very last super cool robot. Sam tries to buy a super cool robot, but can't because James already scooped it up". We don't quite have room in this article to expand Tasty to that level.

That having been said, integration testing can be quite useful for when you're dealing with session data, even for simple things.

Anyone who's messed with testing sessions by hand in a browser without some extra supporting apps probably has inevitably felt some considerable pain.

We're going to add a simple feature to Tasty that lets users indicate which entry in the system is their "favorite." What this will demonstrate is that we can create two separate sessions in our tests which do not interfere with each other.

Let's start by generating some code:

 $ script/generate integration_test stories

The following integration test shows how to build a custom user object with some helper methods, borrowing a trick from Agile Web Development With Rails.

test/integration/stories_test.rb
require "#{File.dirname(__FILE__)}/../test_helper" 

class StoriesTest < ActionController::IntegrationTest
  def test_users_have_separate_favorite_entries

    populate_entries

    greg  = regular_user
    andra = regular_user

    greg.gets_a_random_favorite_entry

    greg_first = greg.favorite_entry

    andra.gets_a_random_favorite_entry

    # make sure session does not change after other user gets an entry.
    assert_equal greg_first, greg.favorite_entry
  end

  def regular_user
    open_session do |user|
      def user.favorite_entry
        session[:favorite]
      end

      def user.gets_a_random_favorite_entry
        get "/entries/random_favorite" 
        assert_response :success
        assert Entry.find(session[:favorite])
      end
    end
  end

  def populate_entries
    %w[abc.com def.org hij.net].each do |u|
      Entry.create(:url => u)
    end
  end

end

Our first failure happens as expected; it can't find the random_favorite action.

Expected response to be a <:success>, but was <404>

The following chunk of code should get you passing the tests, but be sure to add a view, even if it's empty.

app/controllers/entries_controller.rb
  def random_favorite
    session[:favorite] = rand(Entry.count) + 1
  end

From here, you'd probably want to consider moving this kind of feature down into the User model, after adding a way to login and identify yourself to Tasty. For now, it suffices as a way to show how you can build tests that run with independent sessions for your applications.

Another thing that integration tests are important for, as you get into more advanced situations, is for testing any interesting logic you may have in your routes. Functional tests bypass a lot of the routing stuff, so integration tests are about the only way to test them effectively.

I didn't bother building a view for this new feature, but you could probably just add something like a favorite? method to Entry which inspects the session data and does some interesting formatting in your view as needed.

The End of the Road, But Only the Tip of the Iceberg

Rails Testing is a very big topic. I hope that this article has at least shown by example the bare minimum necessary for you to know what the various kinds of tests are used for, and how to get everything hooked up. There are plenty of folks out there writing Rails apps that don't fire up a browser until they're ready to work on design. I'm not quite that macho, but I do enjoy spending more time in my preferred programming environment than in the browser.

As you get more into testing, you'll find no shortage of interesting topics that I omitted from this article. I recommend doing a deep dive into the resources scattered across the blogosphere, because in the end, this stuff really does make you more productive.

If you'd like to see tasty in its present state, please do download the latest source , and if you'd rather start with the same base I used to create this article, you can grab that instead.

Thanks for taking another deep dive with me, and have fun writing those tests!

Gregory Brown is a New Haven, CT based Rubyist who spends most of his time on free software projects in Ruby. He is the original author of Ruby Reports.


Return to O'Reilly Ruby.

Copyright © 2009 O'Reilly Media, Inc.