Related link: http://www.oopsla.org
I just got back from Seattle, where I was at the 17th Annual OOPSLA conference. OOPSLA is one of those hybrid events, where the academic computer scientists and the practioners who actually build this stuff can get together and swap stories about past projects and speculate on future directions in object oriented computing, software design and web services. The proverbial brick hurled into the crowd would bounce off two software architects, somebody from IBM Global Services, and hit a tenured computer science professor. Not that I advised anyone to do this, particularly as I’m a software architect. In general, the ratio of content to product flackage was agreeably low.
So what did I learn? One of the things about writing computer books is that, if you’re a perfectionist like myself, it becomes increasingly hard to let anybody else look at your code. This seems a little bit backward, but the expectation tends to be that you’ll always get it right the first time. I’ve gotten this occasionally in my career, so you can only imagine how Kent Beck must feel. He’s credited with inventing eXtreme Programming and Agile, as well as being one of the major proponents of Test Driven Development. I just write Java books. His point, which I will now twist, was that it’s fine to just make it work, provided that the intent is to make it work better in the next round: one data manipulation doesn’t require an abstraction, but if you need to do it again, refactor. As in sin, intent matters.
Actually, you don’t need to imagine because he mentioned it in the keynote: Kent Beck leaves To-Do comments in his code. Of course, this is a natural part of XP development, not to mention most other kinds. I, and I suspect most architects, believe that good software develops out of both design and experience. Up front planning will help you get much closer to where you want to go, but iterative development, prototypes and a willingness to chop down trees (code) will get you to the best system in the most reasonable amount of time.
This issue was the subject of a panel discussion between a group of “Up Front” architects, advocating covering the walls with UML, and “Extreme Refactoring” programmers, advocating a sit-down-and-code approach. The gimmick was that panelists kept switching sides, sometimes addressing their own points. While it was fun to watch Noted Figures in Software Architecture dart across stage donning and doffing paper neckties, the point, that architecture and refactoring go hand in hand, was obvious to most of the audience in the first five minutes or so, and the panel format perhaps wasn’t the best way to address the real issues, which are the tradeoffs between design and refactoring.
A year or so ago I got the refactoring religion in a big way, and I’ve been periodically tearing through my company’s codebase dealing with the issues that had been put off for another day. It’s an interesting activity the first couple of times, because, by definition, a refactoring means that the system works the same way when you’re done as it did in the beginning. Of course, I usually cheat: I refactor when I suspect I might want to make changes in the future, and where the refactoring will make it easier for me to do that. I’ve even been known to break the cardinal rule of Doing It Right and refactor as I add new functionality, rather than before.
The interesting thing about refactoring in this case is that it’s led the product in question towards a more self-consistent architecture than it would otherwise have had, even though I did a large amount of up-front design work on the current release, which is itself the fourth major iteration of the package (and presumably I’d learned a few things on Iterations 1-3). Of course, if I hadn’t done the up-front design I wouldn’t have had a stable framework to refactor in. If I’d built the system up function by function, I suspect the overall result might have been the same, but it would have taken longer to get the initial code working, and the refactoring process would not have been trivial (although one might say the whole fourth version was just a big refactoring of the third).
So what were the big lessons? Design up front as best you can, but don’t be afraid to switch later. In addition: Don’t cling to particular technologies or languages just because, but don’t switch to the Next Big Thing either. Java and .NET are good for different things, and some of the same things. Don’t fear the TODO comment. Eclipse is cool. Web Services are almost there, but maybe not quite this minute. If you want to get a major standard adopted for billions of dollars a year worth of commerce, keep it as simple as possible. Starbucks in Seattle really is better than Starbucks back east.
I know, I know - a whole week for that? No, I also got to see Bill Gates showing off a Tablet PC and terrifying the program manager running a demo of Microsoft’s testing environment. Worth the trip right there. In all seriousness, there were some interesting presentations, and the papers behind them are available at the event web site. Next year is in Anaheim.
Anybody else go? Anybody learn anything?
refactoring begs risk analysis
Refactoring sounds like a loose version of the spiral model. The one point it's missing may be the most important: between implementation and re-design, the best practice is risk analysis.
refactoring begs risk analysis
This may not have been highlighted in this article, but refactoring can only be done when you have a comprehensive battery of unit tests. If your unit tests are not comprehensive, then making any change to the code is liable to introduce bugs. Would you suggest doing risk analysis before _any_ change to the code? The article might also give the impression that you should refactor as one big "clean up" of the code. The proponents of refactoring recommend that you should generally do it in small steps before adding new pieces of functionality. Since you are going to be changing the code at this point anyway, I don't see that there is a large increase in risk. The pay-off of having code which is maintainable over the longer term seems to far outweigh it.
refactoring begs risk analysis
The blanket assertion that refactoring requires a battery of unit tests is wrong. Refactoring is certainly aided by a good test set, just as any other development activity is, but the core activity can be applied to any project. I've seen it done very effectively with large projects that don't have any automated test infrastructure at all.
Refactoring in small steps is an intrinsic part of the Agile development process, but it's not part of all processes. Nor, for that matter, should you stop thinking about doing a refactoring when the scope gets large. Sure, it might have been easier if did it earlier, but the long term advantages are still there. If you have some time on a large product, a big cleanup operation will be helpful, just as a set of small refactorings are also helpful.
big cleanup vs. little change
Please excuse my relative ignorance in the realm of XP and refactoring. I was given the distinct impression from the article that refactoring was, as you say, one large re-engineering or code review process. I would not recommend risk analysis before just any software change. I think risk analysis gives the best returns after you have a version of your software implemented but before you embark on a significant effort to re-code. You aren't necessarily targeting the re-coding effort as a risk (although we all know changing code can be risky even with a battery of tests). Instead you should be looking for the places where your application is falling short of expectations or approaching some limit of scalability. It's also a good time to ponder alternatives or prototype new ideas. That way, you know how best to focus and prioritize your efforts once you finally put fingers to keyboard.
In my mind, the inevitable little changes fall most naturally under maintenance, not development, and I agree that a test harness can be critical here. The big changes, however, are where things are most likely to blow up in your face. They also often imply the need for a new harness.
refactoring begs risk analysis
Well Martin Fowler says ("Refactoring" start of Chapter 4):
"If you want to refactor, the essential precondition is having solid tests"
and in Chapter 2:
"In almost all cases, I'm opposed to setting aside time for refactoring. In my view refactoring is not an activity you set time aside to do. Refactoring is something you do all the time in little bursts"
refactoring begs risk analysis
Martin Fowler is a smart guy. He's made some major contributions to software engineering. However, he's not god, and when he speaks the mountains do not tremble. I've heard him speak, so I know that this is true. The first person in print on any subject would have the last word. Otherwise the world would be out at least two major religions. :)
The essential precondition for refactoring is understanding how the software works. Tests help here, as elsewhere. So do other things. I've seen successful refactorings when time was set aside to do it, and I've seen successful refactorings without batteries of tests. Scheduled refactoring is also called structural software maintenance.