|
Top 12 Reasons to Write Unit Testsby Eric M. Burke and Brian M. Coyner, authors of Java Extreme Programming Cookbook04/02/2003 |
Most programmers do not write tests. We all know that we should write them, but for whatever reason, most of us don't. This is unfortunate, because testing is the most powerful tool we know of to improve software quality. Tests reduce bugs, provide accurate documentation, and improve design.
In our consulting work we hear every excuse imaginable for not writing tests. For example:
- "I don't know how to write tests."
- "Writing tests is too hard."
- "I don't have enough time to write tests."
- "Testing is not my job."
- "My code is too simple for tests."
- Blah, Blah, Blah ...
We hope that by writing this article we can dispell these myths and encourage programmers to do the right thing and write tests!
Tests Reduce Bugs in New Features
We advocate writing tests as you write new code. Tests do not eliminate bugs, but they dramatically reduce the number of bugs as you add new features.
Tests Reduce Bugs in Existing Features
With well-tested code, introducing new features rarely breaks existing functionality. If a new feature breaks existing functionality, existing tests fail immediately, allowing you to pinpoint the problem and fix it. Without the tests, you may introduce a bug that is not found for days or weeks.
Tests Are Good Documentation
|
Related Reading 
Java Extreme Programming Cookbook |
A concise code example is better than many paragraphs of documentation. We see this time after time in our consulting work. Far too often, teams produce boilerplate documents that are of little practical value. When programmers need to learn an API, they search for code examples. Tests are among the best code examples because they are concise snippets of code that exercise public APIs.
Tests Reduce the Cost of Change
Tests make it easier to change software because you have confidence that changes do not break existing functionality. When you have good test coverage, you have confidence to explore new design ideas without fear of introducing new bugs.
Poorly-tested software becomes increasingly expensive to change as time goes on. Risk increases dramatically as the system becomes more complex because it becomes more and more likely that changes inadvertently break things that used to work.
Tests Improve Design
Writing tests forces you to make your code testable. Specifically, you tend to rely less on dubious patterns like singletons and global variables, instead making your classes loosely-coupled and easier to use. Code that is tightly-coupled or requires complex initialization is hard to test.
Tests Allow Refactoring
With tests, you are more able to change code throughout the lifetime of an application. Tests provide a safety net, allowing you to refactor at any time without fear of breaking existing code, so you can constantly improve the design of your program.
Tests Constrain Features
Far too often, programmers build fancy frameworks rather than deliver features customers want. When you adopt a test-first approach, you start by writing tests for the current feature. You then implement the feature. When the tests pass, you know you can stop and move to the next feature. Well-tested applications are more easily extended; therefore, you don't have to anticipate what the customer will eventually request.
Tests Defend Against Other Programmers
Textbook code is simple, but real-world problems are hard. We find that in real applications, you often encounter very subtle bugs due to Java bugs, quirky business rules, operating system differences, etc. These bugs may only manifest themselves under very peculiar scenarios.
Let's suppose you find that a payroll calculation routine removes a zero from everyone's salary, but only if the routine runs at 11:59 PM on New Year's Eve. Now, suppose that the bug fix involves a single-line code change.
Without a test, another programmer may come in and change that code. Unless they run the application at 11:59 PM on New Year's Eve, they won't know that they just re-introduced the bug and will cause countless bounced checks next year. With a test, however, you can ensure that when the programmer changes the code, the test breaks and informs the programmer of the problem.
Testing Is Fun
If you thrive on challenges, then testing is a lot of fun. Coming up with automated tests is difficult, requiring creative solutions for complex problems. Just like coding is an art, testing is an art.
In many organizations, testing is relegated to the least-experienced programmers. We often encounter the misconception that testing consists of people completing written checklists as they manually execute the application. This approach is completely unscalable, because it takes longer and longer for humans (monkeys?) to test every feature as the application grows.
Modern OO languages like Java are complex, particularly when it comes to dependencies between classes. One change can easily introduce bugs in seemingly unrelated classes. Gone are the days when each character-based screen is a standalone program. OO apps are far more complex and demand automated tests.
Writing automated tests is harder than writing the code itself, in many cases. The most expert programmers are the best testers. When faced with seemingly mundane coding tasks, coming up with creative tests provides an intellectual challenge that expert programmers thrive on.
Beginners typically need expert assistance when writing tests. This is where pair-programming helps, because experts work side-by-side with beginners as they learn the art of testing.
Testing Forces You to Slow Down and Think
When adding a new feature or refactoring an existing solution, testing forces you to think about what the code is supposed to accomplish. By writing tests first, you think about how to use the public API and what the ultimate outcome should be. Thus you end up with a clean and simple design that does exactly what you expect it to do.
Testing Makes Development Faster
On a class-by-class basis, testing slows you down. It takes time to think about and produce good tests. But as time goes on, your overall velocity increases because you are not constantly worrying about breaking existing code as you add new features.
We have also found that with good tests, we can ignore internal implementation details during the first iteration. Provided that we get the public API right, we can improve internal design and algorithms later, again without fear of breaking something. We have used this specifically for things like sorting algorithms. In the first iteration, we do something quick and dirty, like a bubble sort. We can always come back later and replace it with a better algorithm, if necessary.
Tests Reduce Fear
One of the biggest fears that programmers encounter is making a change to a piece of code and not knowing what is going to break. Having a complete test suite allows programmers to remove the fear of making changes or adding new features. We have found that we do not hesitate to change and improve well-tested code, whereas we fear changing untested code.
Eric M. Burke is an O'Reilly author and a principal software engineer with Object Computing, Inc. in St. Louis, MO.
Brian M. Coyner is coauthor of the Java Extreme Programming Cookbook and a Senior Software Engineer with Object Computing, Inc. in St. Louis, Missouri.
|
Related Reading Java Extreme Programming Cookbook |
O'Reilly & Associates recently released (March 2003) Java Extreme Programming Cookbook.
Sample Chapter 4, "JUnit," is available free online.
You can also look at the Table of Contents, the Index, and the full description of the book.
For more information, or to order the book, click here.
Return to ONJava.com.
You must be logged in to the O'Reilly Network to post a talkback.
Showing messages 1 through 27 of 27.
-
storage test case
2005-08-04 04:24:36 deanrayan [Reply | View]
How to write test case for Storage device for ex. Sun storage 5210 NAS device
-
Translation effort
2005-06-20 08:58:46 jasampler [Reply | View]
Hi, I've translated this article to spanish, and I would like if it could be published with the english version. Thanks for this IMHO really good article.
You can find me in jasampler a t yahoo d o t es
-
Regarding Unit testing
2004-11-22 02:07:57 gadesudhirreddy [Reply | View]
Hello,
Where do we write unit testing ie.,
If i write a class by name test1,in the same class we write unit testing or in another class we write unit testing code.
I don't know where to write code
please help me.
Regards
sudhir
-
Regarding Unit testing
2004-11-22 02:07:14 gadesudhirreddy [Reply | View]
Hello,
Where do we write unit testing ie.,
If write a class by name test1,in the same calss we write unit testing or in another class we write unit testing code.
Idon't know where to write code
please help me.
Regards
sudhir
-
Not Most Powerful Tool
2004-06-08 20:39:02 zz [Reply | View]
"This is unfortunate, because testing is the most powerful tool we know of to improve software quality."
The most powerful tool we actually know of is formal verification. Much more expensive but much more powerful too.
-
Wrong - most programmers do write tests!
2004-02-20 10:13:57 marstein [Reply | View]
I hold that most every programmer does write tests. I read somewhere that a big part of our work is spend writing tests. Think about it: when you write functionality you always have a main program to see if your code still works. That's a test! We change the test frequently and don't write it first. If we did that, and think more about the tests, invest some time, save them and make the results meaningful we gain a lot of safety for a little more investment in time :-)
Martin
-
"writing automated tests is hard..."
2003-04-29 05:43:26 anonymous2 [Reply | View]
The authors are right when they say:
"Writing automated tests is harder than writing
the code itself, in many cases."
Sometimes, it can seem that you would be better off spending your creative energies (+ blood, sweat and tears) on the actual problem, rather than test code. And sometimes, this is absolutely true. And, after all, if you re-architect fundamentally enough, a lot of those tests have to be thrown out, anyway...
That said, writing tests can help you focus on the result to be achieved rather than getting prematurely caught up in how to do it. Getting this perspective can save you a *lot* of time. But there are other ways to do this.
The idea of *automating* tests is an excellent one - and that's what computers are good at, isn't it? For some cases, it's obvious how to test; and for some cases, working out how to do this is very challenging, particularly as the test must be simple enough to not have bugs in it itself (such bugs can give *very* time-wasting false alarms!).
Like everything, whether to test and how to test must be a tradeoff. And such compromises can only be made by understanding what is important and what is not for your particular problem - and this is often something you don't know when you start.
Extreme Programming has other ideas in it, such as "do the simplest thing that could possibly work" and "you ain't gonna need it". Yet these very pragmatic ideas are not applied to testing, which is given a special ideological and dogmatic status.
To say that you should "always test" is simply dogma, and like all dogma, it misleads. But testing is a form of infrastructure, and it is not always worth it.
The truth is you should "sometimes test" - but you will have to work out when to do it and when not to.
-
another reason to write tests
2003-04-27 11:09:46 wrschneider99 [Reply | View]
"You're already doing most of the work anyway." Most programmers write ad hoc tests to some extent, to sanity check that new feature or new method. But these tests are generally not written to a standard framework so they can be run repeatably, at will.
It's only a little more work to take an ad hoc test and fit it into a framework so it can be re-run later.
Of course, that assumes your design is already good. Totally agree that testing forces you to improve your design. Since it's always easiest to test code that doesn't have to run in a container or through a GUI, testing encourages you to maintain layer separation.
-
Testing is a wrong way ...
2003-04-09 06:30:58 anonymous2 [Reply | View]
By tests you can only prove the existence of errors, not their absence. -
Testing is a wrong way ...
2003-04-09 06:42:56 Eric M. Burke [Reply | View]
We never said that tests prove the absence of errors. I will say that tests reduce the number of errors, improve code quality, and help avoid regressions - i.e. re-introducing bugs that were previously fixed.
A blanket statement like "Testing is a wrong way" is incredibly negative and is not constructive. What is the alternative? No tests, as is done in many organizations? I don't think so.
Tests are one piece of a much bigger puzzle - they don't solve every problem, but you'll have a lot more problems if you don't bother to write tests. -
Testing is a wrong way ...
2003-04-29 05:57:18 anonymous2 [Reply | View]
Yes, it's true that testing does not guarantee perfect code; yet it does provide a marginal benefit ("marginal" meaning that you spend a little bit of effort testing, and you get a little bit of benefit in return). And a marginal benefit is definitely worth something!
Unfortunately, the word "test" does suggest perfection to me... I wish for a better word, that conveys the idea of marginal benefit.
For some reason, when I write test code, it's very easy for me to fall into the trap of trying to make it perfect. Perhaps this is because of the problem of "false alarms" - that is, your test code has a bug in it, which makes it think your app code has a bug in it, when it doesn't really have a bug in it.
If you test code is imperfect in this particular way, it can really waste a lot of time.
BTW: While testing is obviously central to the XP canon, it doesn't seem to follow the same philophy, because of the need for perfection in this sense; and because you cannot write tests in an XP way (ie they have to work correctly and completely for them to be any use).
Regression bugs are pretty rare - the vast majority of bugs are new bugs whose cause you hadn't thought of before. In my experience, it is as miraculous as it is rare whenever an automated test actually catches a new bug.
For these reasons, they often aren't worth the effort for the marginal benefit they provide.
Of course, if you create a lot of regression bugs for some reason (eg interactions between components written by different people), then the marginal benefit is much greater. -
Testing is a wrong way ...
2003-04-29 05:55:06 anonymous2 [Reply | View]
Yes, it's true that testing does not guarantee perfect code; yet it does provide a marginal benefit ("marginal" meaning that you spend a little bit of effort testing, and you get a little bit of benefit in return). And a marginal benefit is definitely worth something!
Unfortunately, the word "test" does suggest perfection to me... I wish for a better word, that conveys the idea of marginal benefit.
For some reason, when I write test code, it's very easy for me to fall into the trap of trying to make it perfect. Perhaps this is because of the problem of "false alarms" - that is, your test code has a bug in it, which makes it think your app code has a bug in it, when it doesn't really have a bug in it.
If you test code is imperfect in this particular way, it can really waste a lot of time.
BTW: While testing is obviously central to the XP canon, it doesn't seem to follow the same philophy, because of the need for perfection in this sense; and because you cannot write tests in an XP way (ie they have to work correctly and completely for them to be any use).
-
Testing is a wrong way ...
2003-04-09 07:41:50 anonymous2 [Reply | View]
The solution is verification and analysis. As e.g. an algorithm, which was modelled with a petri net, can be examined for characteristics as deadlock, livelock, buffer capacity, ...
Testing represents a optimistic procedure: develop first and then test. A better way would be the following: permit only methods for development, which produce guaranteed (mathematically founded) correct algorithms - see petri nets
Yours sincerely,
Gabriel
-
Testing is a solution, not THE solution
2003-04-09 09:57:29 anonymous2 [Reply | View]
That's nice, but I can guarantee any software house that only implements provable algorithms will get crushed in the marketplace by a company that uses agile methodologies to produce software that is "good enough" because the agile company will ship first. Ship Or Die.
In a risk-return analysis, test-first design will beat mathematical analysis in all but a few niches.
Also, you can't be competetive in the marketplace if you have to re-prove your code every time you have to adapt your code for a new feature, or refactoring.
-
Excellent Article!
2003-04-08 20:10:02 tw33t2 [Reply | View]
I wish all developers would see the widsom in writing unit tests. For 8 years I worked in a group that did write unit tests. Most developers said the found more problems writing the tests than running them, because writing the tests would cause them to slow down and design the function for ease of testing. This made the code very easy to maintain.
I just moved to a new group recently, and as I have found out they don't write unit tests and the code STINKS! They use the excuse that they don't have time to write the tests, but they don't see all the time that is wasted (hours and hours) hunting down bugs and the poor software quality. Adding new features and maintaining the code is a nightmare! The worst part is the poor design. Unit tests force the developer to have good design. When I see a 2000 line function that has conditionals nested 10 levels deep, with code that is never executed, you know it was never unit tested.
Great article! Thanks!
-
So, how do I wirte tests?
2003-04-07 09:32:15 anonymous2 [Reply | View]
Do you have any reading suggestions or a starter on the subject? -
So, how do I wirte tests?
2003-04-08 19:39:10 tw33t2 [Reply | View]
A book I would recomend, sorry it's not O'Reilly, is Code Complete by Steve McConnell ($25 amazon). There is only chapter dedicated to unit testing, but the whole book is excellent about writing solid code. -
So, how do I write tests?
2003-04-07 20:35:54 Brian M. Coyner [Reply | View]
Our book, Java Extreme Programming Cookbook, provides an excellent start to learning numerous open source unit testing frameworks. Also included is how to integrate Ant and JUnit to create a continuous-integration development environment.
Check out www.junit.org for information on JUnit extensions.
-
Singletons are dubious?
2003-04-04 17:09:09 anonymous2 [Reply | View]
To the author: Can you please clarify your assertion that Singletons represent a "dubious pattern"? -
Singletons are dubious?
2003-04-05 07:04:29 Eric M. Burke [Reply | View]
Sure...check out this URL and search for "singleton" to see a great discussion:
http://groups.yahoo.com/group/xpstl/
The project I'm working on now uses a lot of singletons, and I can speak from experience in saying that they are hard to test, hard to refactor, and make things too tightly coupled to "global variables" disguised as singletons.
I think the discussion on the Yahoo group mentioned above says it all.
-
GUI testing
2003-04-04 04:29:10 anonymous2 [Reply | View]
Generally I'm a massive fan of unit testing, but in GUI programming this approach tends to break down. Particularly in a SWING rather than a web application.
Simple tests like "I click on the button and the main application launches" are incredibly difficult to test for with a test suite and much more straightforward with a test script.
The problems are manifold; test scripts are open to interpretation, and not always followed exactly. It's easier to write a test script than a unit test, so there is the temptation to do this where a unit test would be practical.
Does anyone have any suggestions for tackling this problem ? -
GUI testing
2003-04-15 05:44:40 anonymous2 [Reply | View]
Have a look at http://Abbot.sf.net or similar tools.
xix [nine-teen]
-
GUI testing
2003-04-07 06:42:05 anonymous2 [Reply | View]
We've experienced the same problem in our .NET Winforms apps. Initially we overcame it by coding a 'robot' class that would open a form, find the appropriate controls and then operate on them. We would interrogate the form/database to verify the outcome was what we expected.
Argh. It works, after a fashion, but we couldn't devote the time necessary to make a really good robot tester, so it's not multithreaded and that creates some problems...
I've read recently about someone who put all GUI-based methods into a separate class. When he wires up the GUI, he just calls a method in another class that does stuff. I think this is one way to go, though I'm not sure about how you'll manage verifying GUI actions based on successful method calls. I suppose that as long as your GUI reacts to values returned from the class, you'll be fine.
We've resolved that on our next project, we will avoid GUI testing (by actually loading a form, that is) like the plague, by any means necessary.
-
Write Tests == Find Bugs
2003-04-02 19:38:17 hlship [Reply | View]
When you write tests, you find bugs. Period.
I've been using unit testing in various guises for over ten years, in a variety of languages and environments. Anytime you push your code coverage numbers up, you WILL find bugs.
I think the authors of this article should emphasise combining unit tests with code coverage. Writing tests that increase code coverage is the fastest way to find those latent bugs.
It is always better to find bugs in your unit tests than in the wild and wooly environment of your code.
Had an example of this tonight; I was adding code to check that the classes being generated at runtime using BCEL actually implemented all abstract methods declared in superclasses and interfaces. I had to write a small class to represent a method signature. Completely straight forward, but from discipline, I wrote a small JUnit test suite for this class.
And the test failed. I spent some time with the debugger and some time staring at the code, before I found the tiny error in my implementation of hashCode().
Had I tried to debug this in a running Tapestry application it would have been a nightmare, because of the sheer complexity: many classes, interfaces and use of BCEL to create "enhanced" subclasses (at runtime) with additional methods and fields ... and this latent bug was sitting right in the middle of it all and would have screwed it up painfully.
So, five minutes to write the tests plus ten or fifteen minutes to chase down the hashCode() bug vs. anywhere from 45 minutes to two hours to try and track it down inside a running application. Multiply by 100's of classes in the framework ...
The other points in the article are extremely valid. For anyone who has been following Tapestry's (http://jakarta.apache.org/tapestry) light-speed reinvention lately, the root of it is the JUnit test suite built before making all the radical changes.
-
Write Tests == Find Bugs
2005-03-28 21:56:57 Saravanaprakash [Reply | View]
Hai,
I want to write a test case for an email and an web page. So can you give some sample test cases for the above. I need your help. Expect your reply.
Thanks
Saravanaprakash -
Write Tests == Find Bugs
2006-01-05 02:56:47 RahulCharan [Reply | View]
well here is Rahul Charan wrkng as Testng Engrr.
Regrdng Test Cases for E-mail
i can thnk of few like
1.Lay out Test Like hw the window of E-MAil is appearng
2.wel its functionality
i amn n t able to thjnk abt webpage
ok
Bye
Rahul
