O'Reilly    
 Published on O'Reilly (http://oreilly.com/)
 See this if you're having trouble printing code examples


Cookin' with Ruby on Rails - Designing for Testability

by Bill Walton
06/28/2007

Editor's Note: This series continues with Cookin' With Ruby on Rails: More Designing for Testability.

NOTE TO READER: This tutorial is intended for use with Rails 1.2.6. It has not yet been updated to support Rails 2.x.

CB has just finished lunch and is thinking "As much as I like the pizza Boss brings by, it'll be even nicer to get to work with Rails 'on the clock'." And just as expected, Paul knocks on the cube 'door,' ready to pick up where they left off at their last session.

Paul: Hi, CB. Ready to get started?

CB: Hi, Paul. You bet. You ready to put on your tester hat?

Paul: Sure. Like you said, it could be a good way for me to get familiar with Rails. I'm not sure I understand what you meant when you said the tester role would be a "natural fit" for me because of the regular contact I have with Boss, though. You wanna say a little more about that?

CB: To tell you the truth, Paul, I'm hoping you'll help me convince Boss to try a different approach to testing. A few years back we had a lot of focus in our product development, manufacturing, and warranty repair groups on testing. I think we called the program Design for Testability. The premise was that the more testable a design was, the cheaper it would be to build and support and the faster we'd get it to market. And it proved out. I think the same general approach will work for software. And I'm hoping you'll help me get Boss onboard. The approach I'd like to see us using advocates writing the Customer Acceptance tests before the developers write application code. Your "day job" puts you in contact with Boss on a regular basis. I'm hoping you'll be able to use some of those encounters to get Boss writing Acceptance tests as part of his giving us Requirements.

Paul: Well, CB, I'm not sure I'm following you entirely. But to tell you the truth, as a project manager, I'm pretty frustrated with the current state of things around here. Seems like most of my big projects go pretty much the same way. Everything's going along fine and then we get to testing. As soon as the customer gets their hands on the software, the schedule goes to hell. Customer says there's a bug. Developer says it's working the way the requirements say it's supposed to work. And then the "he said, she said" starts. And I get to sit in meetings with Boss and his bosses and explain why I can't bring the project in on schedule and budget. So, if you think that by me taking on the tester role on this little project, I'll learn something that can help me make that better, I say, let's get to it!

CB: Cool! I've already got Instant Rails started. How about you take the keyboard, start the web server and then make sure our app's still working?

Paul: No problem. Let's see... start Mongrel,

mongrel_rails start

starting our web server
Figure 1

then browse to our app.

http://localhost:3000/recipe

our cookbook app
Figure 2

Paul: Looks OK to me. You want me to bounce around a bit and make sure it's all still working like it was before?

CB: You sure you can remember exactly how it worked before?

Paul: Well, this is a pretty simple app right now, and it hasn't been that long since our last session, but I think I get your drift. In fact, that's another one of the ways we get into trouble testing our big projects. Customer comes in, hasn't touched the app in a while, and next thing I hear is, "That's not the way it used to work. That's a bug!" Developer says, "That's the way it's always worked." Then I get to go digging through test plans to see if maybe there's something in there that'll help me get closure.

CB: Yep. So, how's the test plan thing workin' for ya?

Paul: They take a long time to produce. It's hard to get customers to write them, especially with any real detail, so I'm having to hire folks to help them. I'm using contractors right now, but it's starting to look like that's just opening up a real can of worms. The customer groups are starting to make noises about Boss needing a dedicated test group to write the test plans and do most of the testing for them. Any discussion about adding headcount for testing doesn't sit real well with Boss and his bosses, of course. It's starting to feel like a real tar ball.

CB: Yep. Documenting test plans and "monkey on the keyboard" manual test execution doesn't fit most folks' idea of a good time. Speaking of time, how long's it going to take you to make sure our little app still works right?

Paul: I don't imagine it'll take more than a few minutes for this little thing. Not like my "day job" projects. Heck, right now we're in the middle of an end-to-end test on our ERP system. It takes half a dozen customers testing full time for almost a week just to make one pass through it. This'll be a breeze. Three or four minutes tops.

CB: Cool. I'm going to go grab us soda while you do that. Be right back.

CB walks off grinning, thinking, "By the end of this session, Paul's going to be well on his way to becoming a convert."

CB: You done?

Paul: Yeah, everything looks like it's working like it was. How about giving me an idea of where we're headed? I mean, I know we're going to be doing automated testing. But that's about all I know at this point. How's that work in Rails? One of the testers I've brought in is talking up GUI test tools. This anything like that?

CB: Arghhh... no!!! Now don't get me wrong. I'm not saying those tools don't have their place. I've got a buddy who's working on a "green screen" app across town. His boss wants to put test automation in place and they really don't have any option but to test through the UI. There's just no other way to exercise the app in his situation. But Rails gives us much better options. For starters, Rails has a test framework built in. It's not something in addition to Rails. It's an integral part of Rails. And it fully supports automated testing of the MVC pattern that Rails uses. It may take a little getting used to the terminology, though. Rails uses terms that may be familiar, but that mean something different than you might be used to. We'll test our Models using what Rails calls Unit tests. In Rails, Unit test means code that tests the functioning of a Model. Functional test means code that tests the functioning of a controller. And Integration tests means code that tests the functioning of the application across controllers. Some folks take exception to these words being used differently than they're used to. My take on it is that the terms make a lot of sense in the context of the MVC pattern that's being tested. It'll get easier to understand once we get into it, so what do you say we just get started?

Paul: Sounds good. So what's our first step?

CB: First thing we need to do is set up our test database.

Paul: Oh, great ;-(. I figured you'd already have that done. Now I'm thinking I should have blocked out a bigger chunk of time this afternoon.

CB: Settle down, big guy ;-). You've seen how easy development with Rails is, right? And I just told you that test automation is integral to Rails, right? Watch how easy Rails makes it.

We're going to have a dedicated database for our testing. I'll get more into this later, but for now it's important just to understand that the contents of our test database are going to get deleted and rebuilt every time we run our tests. So, we need to make real sure we're running our tests against the test database and not our development or production databases. Remember the database configuration we talked about the first time we worked on this? Let me pull up the file.

database.yml
Figure 3

CB: So the first thing we need to do is to create the database. We'll do it using the MySQL command line just like we did it for the development database. First, we'll open a new command window, then login to MySQL as root with no password. I know that we really should be using a password, but since I'm the only one right now who even has access to the system, I don't think the security police will be too upset ;-). We'll add a password as soon as the situation changes to warrant it.

mysql -u root -p

Hit enter at the password prompt...

Then create the database.

create database cookbook2_test

Oops, forgot the semicolon

;

Do the grant to 'ODBC' just so we don't run into that avoidable problem. (I really wish I understood why some Windows systems have a problem if we don't do this and others don't.)

grant all on cookbook2_test.* to 'ODBC'@'localhost';

And we're done with that.

exit

creating the test database
Figure 4

CB: So now we've got a test database, but it's empty. We need tables to put our data in. Rails wants to make testing easy; something we really have no excuse to not do. So it provides us with a set of rake tasks that do the "heavy lifting." All we have to do to recreate the tables that exist in our development database is...

rake db:test:clone_structure

creating tables in our test database
Figure 5

Paul: It doesn't look like it did anything.

CB: Yeah. Depending on how the rake task is coded, sometimes it only tells you if it has a problem. Easy way to check is just to open up your database front end of choice and take a look.

So here's our development database...

development database schema
Figure 6

and here's our test database.

test database schema
Figure 7

Paul: Wow. That was easy. What about data? Did Rails copy that over too?

CB: No. The manual testing we're doing in our "normal" development pretty much needs a close approximation of our production data. Customers seem to find it easiest to test when they can exercise the system using the data they're used to seeing. But successfully using automated testing requires a different approach to test data; one some folks refer to as "engineered test data." The data we'll put into our test system is exactly what we need to test the system; not more and not less. If we do it well, that is ;-).

Paul: Oh boy ;-(. I knew there'd be a catch. The contract tester that's been talking up the GUI test tools started talking about the need for "engineered data" the other day. Boss was standing there and said he'd never heard the phrase before, so I asked her about it. She started talking about a whole new project to go back into all the test plans and figure out what data was being used. Of course, that was after they'd gone back and re-executed everything to make sure the plans were still valid, added detail to the existing test cases, added new test cases, and on, and on... She was all "wouldn't this be great?" I could tell from the veins in his neck that it was all Boss could do not to say something less than complimentary. In the end, he just said, "Hmm. That's interesting. We should think about that." And then he excused himself. I don't think I want to revisit that topic with Boss, if you know what I mean.

CB: I do know what you mean. The difference between that and this comes down to one word: Legacy. Working with Legacy systems is expensive; no question about it. And if there's one generally accepted characteristic that defines a Legacy system it's the absence of automated tests. Remember the first day you and Boss came by on this? Remember me saying to him, "One of the things that Rails really enables, Boss, is an iterative and incremental approach to development." And remember me saying just a few minutes ago that testing is an integral part of Rails? There's one guaranteed way to avoid the situation you just described: stop building Legacy systems. I realize your "day job" makes it a little difficult to see how this could possibly work, but... well, maybe it'll be easier to just show you.

We're going to create our test data in what're called "fixtures." Remember that there are three types of tests in Rails. Working from the top down we've got Integration, Functional, and Unit tests. The basic top-down relationships for each type of test looks like this. Test results get produced by executing test cases which consist of test methods which use test data, much of which is specified in test fixtures. So, we'll start at the bottom-bottom and work our way up.

You probably don't remember seeing it, because I didn't draw your attention to it at the time, but when we generated our scaffolding for the recipe and category models and controllers, Rails actually produced the stubs we need to do our Unit and Functional testing of them. Let's take a look. The test fixture files are in the test\fixtures directory...

categories fixture
Figure 8

 recipes fixture
Figure 9

And the test case files are in the test\unit directory. Here are the Unit-level test cases...

category fixture
Figure 10

recipe fixture
Figure 11

Paul: OK. They definitely look like "stubs." What do we need to do to them to make them useful?

CB: Well, like many things in Rails, they may not do a lot right now, but they'll already do something useful. Let's take a look at what happens when we run the test cases. Let's start with the category test. From the application root directory, run it with

ruby test\unit\category_test.rb

category_test stub run results
Figure 12

CB: And now let's run the recipe unit test.

ruby test\unit\recipe_test.rb

recipe_test.rb first run
Figure 13

Paul: Hmm... looks like something went wrong...

CB: Actually, something went exactly right. Let me walk you through what happened with both of these "bare bones" test cases. Let's start with a quick look at our test database. Here's our categories table.

category table
Figure 14

And here's our recipes table.

recipe table
Figure 15

CB: You can see that the categories table has two records in it. The records are empty except for having an id, but that's because the fixtures that we told Rails to use don't specify anything but the id field. When it executed the test case Rails found the line

fixtures :categories

That tells Rails to load the categories fixture (categories.yml) before executing each test method. In order to make sure that test methods don't "step on" each other, Rails resets the database to the state specified by our fixtures before running each test method. It deletes all the records in the table, and then reloads the table based on the fixtures we've told it to use. When Rails finished running the category test case it reported:

1 tests, 1 assertion, 0 failures, 0 errors

That's telling us that when it ran the test case (category_test.rb), Rails found one test method (test_truth) which contained one assertion (assert true) which produced the result we told Rails to expect (0 failures) and it didn't encounter any problems along the way (0 errors).

Paul: So assertions are how we compare the results we actually got with what we expected?

CB: Exactly. And Rails provides us with a whole set of custom assertions to make the comparisons easy to do. The standard set Rails provides includes things like like assert_equal, assert_nil, assert_not_nil, and so on. Plus, we can create our own. I'm not going to go into them all right now, but before you leave I'll give you a reading list so you can dig into that and more. In general though, assertions take two general forms. The first, most general, form is:

assert  expression

This lets us write assertions like:

assert Recipe.count <= 3

We can also use custom assertions. Rails provides some primitive ones, like assert_equal, and we can also define our own in the test_helper.rb file. Custom assertions have the form:

assert_some_condition(expected_value, thing_to_test_for_that_value)

Paul: So, fixtures are how we load data into the database, and the database gets reset before every test method execution. Is that right?

CB: Yep. There's also another way to prep the environment prior to the execution of test methods. If there's a setup method defined in the test case, category_test.rb for example, Rails will execute it prior to every test method. It's Ruby and Rails code, as opposed to the data in fixtures. Setup can be really helpful in cases where we need to set up data for specific test cases. And there's a related teardown method that gets executed after every test method, if it exists. We'll get into both of those later on.

Paul: And assertions are how we test to see if specific objects have the values we expect. OK. Let's move on to the recipe test. What happened with that? It looked like something went wrong to me, but you said it meant things went right. Want to explain that?

CB: You bet. Let's take another look at the recipes table.

object view of recipes table
Figure 16

And now look at the recipes fixture again.

recipes fixture without foreign key
Figure 17

Now look at the error message again. See the fourth line of the error message? The line that starts with "a foreign key constraint fails." Notice anything missing from the fixture?

Paul: Yeah. There isn't any value defined for the foreign key, category_id.

CB: Good eye. So let's add a category_id for each fixture. I like to keep the numbers lined up for readability, but you need to be a little careful doing it. This is a YAML file, and YAML doesn't allow tabs. So you either have to use the spacebar to do this, or make sure your text editor is set up so that hitting the tab key inserts spaces. Otherwise you'll get a parsing error.

update recipes fixtures
Figure 18

And now let's save our changes and rerun the test.

recipe_test results with new fixtures
Figure 19

Paul: So, if I understand this right, MySQL enforced the assignment of the foreign key that we have defined in our schema. Is that right?

CB: Exactly right. And that's why running the test before we'd added the foreign keys to our fixtures resulted in Rails telling us that had encountered an error, not a failure. Rails reports a failure when an assertion fails; that is, when we don't get the value we expected. Errors are things like this, where the system, rather than the application, is misbehaving, or when we have syntax errors in our code. That sort of thing. What happened a minute ago is basically that Rails told us, "MySQL says you're doing something wrong."

Paul: OK. But now that I think about it, our schema also says that categories have to have names and recipes have to have titles and that both names and titles can only be so long. MySQL is obviously treating those differently than the foreign key constraints. How do we test that those rules are getting enforced?

CB: I'm glad you asked ;-). Because the question underlines why I want to see us move to a test-first model of development. Think about it. The way it sits today, if we put this app in the hands of customers, they'd already be writing bug reports. Fire up Mongrel and our app again and you'll see what I mean. Create a new category but don't give it a name. Just click the Create button. See what happens?

create category with no name
Figure 20

Paul: I'll be darned. And I'll bet it would let the same thing happen with recipes too. Right? Oh yeah. I can see the bug list growing right in front of my eyes. So if telling the database not to allow it isn't enough, how do we make sure those rules get enforced?

CB: You know, the first thing we ought to do is figure out why the database is allowing it. Let's take a look at the schema. It's over in our db directory.

schema.rb
Figure 21

There's the problem. See the :default => '' parameter on the name and title columns? We told the database the field couldn't be null, but an empty string is not null to the database. I don't remember why I did that. Probably just to hurry things along during that first session. I probably thought something like, "I'll just do this for now and come back and fix it later." Yet another great example of why we ought to be writing our tests first. To answer your question, though, we'll use Rails validation methods.

But before we start talking about validations, think about what you've seen so far. We generated a test database in two steps. Using the test stubs that Rails had already generated for us, we found out that Rails and the database were working properly. And right out of the box, our tests told us we'd missed some important things when we wrote the code. And now they're telling us what code we need to write to fill those holes. If we'd written the tests before we wrote the code, we wouldn't have those holes at all.

Paul: Well, I'm definitely impressed with how easy it was to get this far. And it's definitely obvious now that we've got some holes that would turn into problems if we didn't get them taken care of before we gave the app to customers to test. But, to tell you the truth, I haven't gotten my arms around what you mean by writing tests before we write code or how that would work.

CB: That's fair. Since we know now that we've got some code we need to write, what do you say we use that as an opportunity to explore the notion? Go ahead and delete that record we just created, and then let's go ahead and flush out our fixtures so we'll have a couple of good records in the database for every test method to use. I think I'll just copy the info off the show page from the records we've already created.

So categories.yml will have:

one:
  id: 1
  name: main course
two:
  id: 2
  name: beverages

And recipes.yml will contain

one:
  id: 1
  category_id: 1
  title: pizza
  description: CB's favorite lunch
  date: 2007-05-09
  instructions: Phone pizza joint. Place order. Pay delivery guy. Chow down!

two:
  id: 2
  category_id: 2
  title: iced tea
  date: 2007-05-09

CB: So now we've got a couple of records to work with. Let's write some tests! Let's start with our category model. Open up category_test.rb and we'll write some methods to test our app's basic CRUD functionality. Let's get rid of the test_truth method and replace it with one that tests the Category model's basic read and update functions.

def test_read_and_update
  rec_retrieved = Category.find_by_name("beverages")
  assert_not_nil rec_retrieved
  rec_retrieved.name = "beverage"
  assert rec_retrieved.save
end

Notice how we named our new method. It's important to remember that all Rails test methods have to begin with test_. So now let's rerun our category test case.

ruby test\unit\category_test.rb

category read and update test fails
Figure 22

Paul: Again? I thought we fixed this!

CB: Actually, this is a different error, Paul. See the third line of the error message? It says, "cannot delete or update ..." The error we had earlier said "cannot add or update." What's going on here is that when Rails encounters the test method, the first thing it does is delete all the rows in the table and then load the table from the fixtures. And since there's a recipe in the table for each category, MySQL isn't going to let us delete the category records.  

We've seen this problem before too. Remember towards the end of our first session when we tried to delete a category that had recipes? We got this same error (Figure 33). At that point, we just put a quick patch in place to check to see if there were any recipes before we attempted to destroy the category, and, if there were, then we just returned without doing anything. At that point we said we'd figure out how we really wanted to handle it later. Looks like later has arrived ;-)

Let's tackle the problem in two pieces. The first thing we need to do is decide how we want to avoid this problem during our testing. The way it stands right now we're not going to be able to load any category fixtures if there are records in the recipes table. The simplest way to solve this is to make sure that there aren't any records in the recipes table. When we run the recipes test, we need to clean out all the records in the table before we finish. Remember me mentioning the setup and teardown methods? They get run before and after each test method if they exist in the test case. Let's use the teardown method to fix this problem for our testing purposes. Open up recipe_test.rb and add:

def teardown
  recipes = Recipe.find(:all)
  recipes.each do |this_recipe|
    this_recipe.destroy
  end
end

Now we'll rerun recipe_test.rb

ruby test\unit\recipe_test.rb

rerunning recipe_test.rb
Figure 23

And now let's make sure that our new teardown method has cleared out the database.

recipes table after teardown
Figure 24

CB: Oops ;-) My bad. The good news is, we know the teardown code worked since the test case finished without errors. What's going on here is something we haven't talked about yet. Remember I said that Rails reloads the table from the fixtures before every test method? Well, there are two ways we can make that happen. One way is to let Rails actually delete all the records and then reload them. But that can burn a lot of cycles if we have a lot of methods and a lot of fixtures, so Rails gives us a higher-performance option: a transaction-based approach. By default, Rails will delete all the records and load from fixtures for the first test method, then use transactions to roll back the table to that state at the end of that method so the table's ready for the next one. After the next method completes, Rails rolls back the table again. And so on. The transaction-based approach is the default setting, and I forgot to change it. Let's open up test\test_helper.rb and change the line that says:

self.use_transactional_fixtures = true

to

self.use_transactional_fixtures = false

And now let's run the recipe test case again, and see if that fixed the problem.

rerunning recipe test case
Figure 25

And then take another look...

recipe table after changing use_transaction_fixture
Figure 26

CB: There we go. Now that the recipes table is empty, let's run our category test again.

ruby test\unit\category_test.rb

rerunning category test
Figure 27

Paul: That's more like it. But, and don't take this wrong CB, but that's really not what I'd call a complete test. It seems to me like we need to check a couple more things. First, Rails is telling us it changed "beverages" to "beverage" but we haven't checked to see if there's actually a record in the table named "beverage." And we haven't checked to make sure that there's not a record named "beverages" any more.

CB: I knew you'd be good in the tester role, Paul ;-) You're right. Let's add those tests. So now our test method looks like:

def test_read_and_update
  rec_retrieved = Category.find_by_name("beverages")
  assert_not_nil rec_retrieved
  rec_retrieved.name = "beverage"
  assert rec_retrieved.save
  changed_rec = Category.find_by_name("beverage")
  assert_not_nil changed_rec
  unwanted_rec = Category.find_by_name("beverages")
  assert_nil unwanted_rec
end

So now when we run category_test.rb, we get

test results after adding new assertions
Figure 28

CB: So, we're sure that we can read and update records that are already in the table. Let's make sure we can create and delete records.

Paul: But haven't we already proved that with the fixtures setting up the table and the teardown method clearing it out?

CB: That's a good question, Paul. I think you can argue it both ways.  

On one hand, we've definitely shown with the read_and_update method that the database got loaded, otherwise there'd be nothing to read and update. And we took a peek at the table itself after running the recipe test and saw that the table had been emptied.  

On the other hand, there are a couple of things that make me, personally, choose to go ahead and write a couple more methods to test the create and delete capabilities explicitly. First, I look to my test cases to tell me what an app does. If I don't put in methods to test the create and delete functionality, then I have to remember that the app does that but that it's being tested some other way. I guess I'm getting to an age where I'm less apt to trust my memory ;-) Second, my unit tests are testing the Model and I want to see that happen. If you look at the fixture files, you don't see any explicit use of Models per se. So if I've got a lot of trust in my memory, and I trust that Rails itself is working properly, I could argue the additional tests aren't really necessary. But when I've got on my Tester hat, my SOP is "show me," not "trust me."

It's not much work and it's not going to add significantly to our test execution time. So, I'm going to go ahead and add the methods to category_test.rb. Actually, I think I'll combine them like we did for read and update.

def test_create_and_destroy
  initial_rec_count = Category.count
  new_rec = Category.new
  new_rec.save
  assert_equal(initial_rec_count + 1, Category.count)
  new_rec.destroy
  assert_equal(initial_rec_count, Category.count)
end

And then we rerun our test case, and...

category_test after adding create and destroy tests def 
Figure 29

CB: So now we're sure that, as it stands, our tests make us confident that our Category model is working as designed. Now let's do the same for our Recipe model. Open recipe_test.rb and replace the test_truth method with:

def test_read_and_update
  rec_retrieved = Recipe.find_by_title("pizza")
  assert_not_nil rec_retrieved
  rec_retrieved.title = "pie"
  assert rec_retrieved.save
  changed_rec = Recipe.find_by_title("pie")
  assert_not_nil changed_rec
  unwanted_rec = Recipe.find_by_title("pizza")
  assert_nil unwanted_rec
end

def test_create_and_destroy
  initial_rec_count = Recipe.count
  new_rec = Recipe.new
  new_rec.category_id = 1
  new_rec.save
  assert_equal(initial_rec_count + 1,Recipe.count)
  new_rec.destroy
  assert_equal(initial_rec_count, Recipe.count)
end

And now we run the recipe test case...

new recipe test case
Figure 30

And it looks like we've got the basics working there too. What do you think, Paul?

Paul: Pretty good so far, CB. Are we ready to start filling those holes we spotted?

CB: You bet. Let's use our tests to put a spotlight on them. The approach I'm hoping you'll help me introduce to Boss says it's best to put off writing application code until you have a failing test that demands it. Let's start with the Category model. In Rails, we use our Unit tests to make sure the Model is working properly. "Properly" means, at a minimum, the CRUD functionality that's at the core of pretty much all Rails apps. That's what we just wrote tests for. The next piece of "properly" means that the validations we need our application to do on the data being written to the database are working. And finally, we need to make sure that any methods we include in our Model are working.

We've decided that all our Category records have to have a name and that the length of the name can't be longer than 100 characters.  And we've already seen that we're not currently enforcing that rule. Even if I hadn't given the name field a default value of an empty string, the way it sits right now, a visitor could hit the space bar and effectively do the same thing. So I'd say we're probably going to need to use both validations and a method to check for visitors entering blanks for the name. But let's let the tests tell us what we need.

So let's add another method to our category test case to make sure we only save records that have a valid name. First we'll try to save a record with no name and expect that save to fail, then we'll try to save a record that does have a name and expect it to get saved.

def test_must_have_a_valid_name
  rec_with_no_name = Category.new
  assert_equal(false, rec_with_no_name.save)
  rec_with_name = Category.new(:name => "something new")
  assert rec_with_name.save
end

And now lets run it.

ruby test\unit\category_test.rb

testing for non-blank name
Figure 31

Paul: So Rails is reporting that the test failed, but we expected it to fail didn't we? I'm not sure I'm following this.

CB: No problem. We didn't expect the test to fail. We expected the save to fail. Note that I'm using the word expect a little loosely here. In fact, we knew the test would fail because the test makes sure the system is working correctly and we already know that's not true. If the system were working correctly, Category.save would have failed which would have returned nil and the test would have passed. Our test proved that the system's not working correctly by reporting the failure. But because the database is assigning an empty string as the default value, the save is working even though it's not supposed to. We could change the schema, and later we will, so that the database doesn't assign that default value, but we'd still have the same basic problem with visitors creating blank entries by hitting the space bar a time or two for that field. Our tests are telling us we need to write some code. Let's start by adding a validation that checks to make sure something's been entered for the name. We need to add one line to our Category model. So we'll open app\models\category.rb and, right below our relationship definition, add one line:

validates_presence_of :name

And now let's see what our tests are telling us.

ruby test\unit\category_test.rb

a new failure
Figure 32

Paul: Well, I guess that didn't fix it. It's still failing.

CB: Look closer, Paul. That's not the same failure. But don't feel bad that you didn't pick that up right away. It's my bad that I haven't told you how to read the output yet. See the line right below "Started"? The "F" in the first position is telling us that the Failure was in the first test method that Rails executed, but that two more methods executed successfully after that. Each success is shown as a ".". The last time, our failure was in the second test method and the first and third methods executed successfully. Those indicators get posted while the test case is executed. After it's finished, the details of where the failure (or error) occurred and what happened get posted. So, in this case, we can see that, by adding the validation, we've introduced a failure in the test_create_and_destroy test method and the specific assertion that failed is at line 21. Let's take a look.

a new failure
Figure 33

CB: See where the failure message is telling us "<3> was expected but was <2>"? It's saying that the first argument in the assert_equal evaluated to "3" but the second argument evaluated to "2" and so the assertion failed. The reason it failed is that our new_rec record didn't get saved, and the reason it didn't get saved is that our new validation prevented it, because we tried to save the record without assigning a value to the name field. Doh ;-p So, all we need to do is give it a name before we try to save it. I'll add...

new_rec.name = "for validation"

just before the save and then run our category test case again.

ruby test\unit\category_test.rb

now working!
Figure 34

CB: Ta Da! ;-)

Paul: Not so fast, CB. I'll admit it's pretty cool that we caught a problem with our tests by running our tests. But we still need to make sure that we don't allow records to get saved if the visitors just hit the space bar a couple of times. And we've also got to make sure the system doesn't allow names that are too long either.

CB: I know, I know. I just love to celebrate the small wins too ;-) To test the space bar case, we'll just add a couple of lines to our test_must_have_a_valid_name:

rec_with_blank_name = Category.new(:name => '   ')
assert_equal(false, rec_with_blank_name.save)

And now we rerun out category test...

validation's not fooled by blanks
Figure 35

Paul: Well that's cool. It looks like validates_presence_of isn't fooled by a couple of blanks.  

CB: So now we just have to test for a name that's too long. I think I'll add a new test method for this because I'll need to do a little set up. And I think we'll want to test to make sure that we can save one that's exactly on the limit we've set, and not save one that's just over the limit. I'll add this to category_test.rb.

def test_long_names
  partial_name = ''
  'a'.upto('y') {|letter| partial_name << letter}
  rec_with_borderline_name = Category.new(:name => (partial_name * 4))
  assert_equal(100, rec_with_borderline_name.name.size)
  assert rec_with_borderline_name.save
  rec_with_too_long_name = Category.new(:name => ((partial_name * 4) << 'z'))
  assert_equal(101, rec_with_too_long_name.name.size)
  assert_equal(false, rec_with_too_long_name.save)
end

And now let's run it.

new run with length check
Figure 36

CB: And we've got a failing test, so it's time to write some code!

Paul: Before you do, CB, I've got a couple of questions. For starters, I notice that the line just under "Started" shows the "F" in the second position. But I just saw you add that new method at the end of the file. That output says the failure was close to the beginning. What's up with that?

CB: Good eye, Paul. The test methods don't necessarily run in the same order they appear in the test case file. That's why we need to look at the numbered explanations to see what line the assertion that failed is on. In this case, it's line 44, which is the last line of the new method, which is, in fact, the last method in our test case.

assert_equal(false, rec_with_too_long_name.save)

And what the failure is telling us is that the record got saved, even though the name is too long. What happens with strings that are too long in MySQL is that they just get truncated. That's not a very user-friendly way to handle things, but we'll fix that by adding another validation to our model. Open up app\models\category.rb and, right below validates_presence_of, add a new line:

validates_length_of :name, :maximum => 100

Paul: OK, but before I do, what about a minimum length? Do we want to allow names like 'X'?  

CB: That's a really good question, Paul. One I wish Boss were here to answer. Tell you what, let's use a different option on the validation that'll let us specify a range. We already know the name has to be at least one letter long so, technically, we won't really be doing work he hasn't asked for. So let's add...

validates_length_of :name, :in => 1..100

to app\models\category.rb instead. Then if Boss wants to change the minimum, we're ready for him. And after we add the new validation, we run our test case again...

ruby test\unit\category_test.rb

now checking length
Figure 37

Paul: Cool. But, before I forget, I had another question. When we want to check to make sure a Category record gets found, we use assert. When we want to check that it's not found we use assert_nil. When we want to check to make sure a Category record is successfully saved, we use assert again. But when we want to check to make sure it didn't get saved, we use assert_equal(false, ...). For me, that makes it harder to read the test. Can't we just use assert_false ?

CB: I like it! And of course we can. Except Rails doesn't include assert_false in its standard set of custom assertions. But, like I said earlier, we can create our own custom assertions.

Paul: How hard is it?

CB: Dude! This is Rails!!! By now you should be asking, "How easy is it?" ;-)

The test_helper.rb file gets automatically loaded for every test case. So any assertion method we define there can be used by any test case. So that's where we'll put ours.

def assert_false(record_being_saved)
   assert_equal(false, record_being_saved)
end

And we'll change the assertions in the test_must_have_a_valid_name and test_long_names methods, so our category test case looks like ...

test case with assert_false
Figure 38

And we run our tests again to make sure we get the same results...

test results with new helper method
Figure 39

Paul: Excellent! So that wraps it up for Categories, right?

CB: For the Unit tests it does. We're caught up on our testing of the Category model's currently defined functionality. Let's go ahead and do the same for our Recipe model. I'll put the same validations we used in the category model into the recipe model (app\models\recipe.rb), only using the appropriate field name.

validates_presence_of :title
validates_length_of :title, :in => 1..100

And then I'll add the new methods to recipe_test.rb, again changing name to title in all the appropriate places. I also need to remember to add the title when I create a new recipe in the test_create_and_destroy method, like we did with categories. So now the recipe test case looks like...


require File.dirname(__FILE__) + '/../test_helper'

class RecipeTest < Test::Unit::TestCase
  fixtures :recipes

  def test_read_and_update
    rec_retrieved = Recipe.find_by_title("pizza")
    assert_not_nil rec_retrieved
    rec_retrieved.title = "pie"
    assert rec_retrieved.save
    changed_rec = Recipe.find_by_title("pie")
    assert_not_nil changed_rec
    unwanted_rec = Recipe.find_by_title("pizza")
    assert_nil unwanted_rec
  end

  def test_create_and_destroy
    initial_rec_count = Recipe.count
    new_rec = Recipe.new(:title => "something new")
    new_rec.category_id = 1
    new_rec.save
    assert_equal(initial_rec_count + 1, Recipe.count)
    new_rec.destroy
    assert_equal(initial_rec_count, Recipe.count)
  end

  def test_must_have_a_valid_title
    rec_with_title = Recipe.new(:title => "something new")
    rec_with_title.category_id = 1
    assert rec_with_title.save
    rec_with_no_title = Recipe.new
    rec_with_no_title.category_id = 1
    assert_false rec_with_no_title.save
    rec_with_blank_title = Recipe.new(:title => '   ')
    rec_with_blank_title.category_id = 1
    assert_false rec_with_blank_title.save
  end

  def test_long_titles
    partial_title = ''
    'a'.upto('y') {|letter| partial_title << letter}
    rec_with_borderline_title = Recipe.new(:title => (partial_title * 4))
    rec_with_borderline_title.category_id = 1
    assert_equal(100, rec_with_borderline_title.title.size)
    assert rec_with_borderline_title.save
    rec_with_too_long_title = Recipe.new(:title => ((partial_title * 4) << 'z'))
    rec_with_too_long_title.category_id = 1
    assert_equal(101, rec_with_too_long_title.title.size)
    assert_false rec_with_too_long_title.save
  end

  def teardown
    recipes = Recipe.find(:all)
    recipes.each do |this_recipe|
      this_recipe.destroy
    end
  end 
end

CB: And when we run it...

now testing the full recipe unit test
Figure 40
 
Paul: Ta Da! ;-)

CB:Catchin', ain't it? ;-)

Paul: I gotta say, it definitely is. And I'm really looking forward to doing the Functional and Integration tests, so I hate to do this, but I gotta run right now. I've got a project status meeting with Boss and the Customer groups. Ought to just call these meetings the Beat on Paul meetings ;-p. Before I go, though, you said you were going to send me off with a reading list.

CB: You bet. Did you pick up the "Agile Web Development with Rails" book yet? That's got a chapter on testing in it. There's another good book you should pick up called "Rails Recipes." It's by Chad Fowler and it's got several useful testing recipes in it. There's a real good online tutorial called "Testing the Rails" at http://manuals.rubyonrails.com/read/chapter/20. And there's a fellow by the name of Bala Paranj who maintains a list of links to other online resources for testing Ruby on Rails at http://bparanj.blogspot.com/2006/11/resources-for-testing-ruby-on-rails.html. That should be enough to get you started ;-)

Next time we get together, we'll put together the Functional and Integration tests we need to get ready for our next meeting with Boss.

As Paul exits, CB can't help but think, "This could be the start of some really positive change around here!"

Articles in this series

Bill Walton is a software development/project management consultant/contractor.


Return to O'Reilly Ruby.

Copyright © 2009 O'Reilly Media, Inc.