Even using the right tools, in the right way, a software project can still get into trouble. One of the most pernicious ways to fail is over-specify everything up front. As the “Lean Software Development” movement has documented, well-intentioned people often add risk to their projects when they make hard decisions too early - before any research to identify any supporting facts. The best practices are Adaptive Planning, and Just-in-Time Requirements.

Another way to fail is to allow these requirements to fall into your lap by themselves. This post explores why embracing these deceptively easy requirements still adds risk.

Suppose we have a senior programmer, call him Vu, using a program that has seen better days. This “Version One” once had a good architecture, but years of adding features without refactoring have tangled it up with compromises. So one day Vu meets Karen, a junior programmer excited by the prospects of working with a new technology, “Q”.

Q is generating breathless word-of-mouth support among the seniors and thought-leaders of the programming community. Karen pitches Vu a rewrite of this new program, “Version Two”.

Two years later, Vu scraps Version Two as unusable, and Karen promotes herself out of the picture. Then Vu, using his original platform, “P”, rewrites his program from scratch as “Version Three”. He’s surprised this only takes a few months.

Vu knows better than to make too much of the biggest apparent difference between the two projects - the platform. He admits he learned a lot from Q, and that he enjoys the workarounds for the things P did not support. Yet he is left with the feeling that much of the hype regarding Q is just that - hype.

Other explanations are possible.

Let’s review what was available in Vu and Karen’s minds, at various phases of this project. When Vu started, he did not know the requirements for Version One, and he didn’t know what its design would be. In the days of “Rapid Application Development”, Vu’s thought leaders would have advised him to deliver high value features rapidly, and not worry about design.

This strategy worked, in terms of Version One’s early business viability. Vu put the program online, and it took care of business for him. But business use tends to inspire new requirements, and a program’s design quality shows when the time comes to add these features to it.

Vu’s program probably resisted new features, until the velocity of each one got lower and lower. This represented his program’s design quality losing its “Open Closed Principle”. As the program’s features exceeded its initial design, Vu recognized his program was reaching its age of retirement.

So along comes Karen, breathless with learning platform Q. Vu and Karen correctly identify Version One as a good case study for platform Q, and Karen dove into the project.

Karen thinks that one big part of the project - its requirements - are finished. And she knows that platform Q has very opinionated design ideals, so she does not expect to find designing Version Two very difficult. Version One has many pages with many buttons on each page, so Karen starts by copying each page over, then wiring up each button.

The biggest mistake here was the simplest. Karen did not triage the requirements, and start with the most important ones first. She did not determine a growth path for her program - a “phased delivery” plan. Her new program could have started by adding features to the old system.

No matter how many requirements are finished, and verified in live use as practical and appropriate, no software project should run a long time before putting features online. A software project without live users is like a startup company without customers. It is a plane using up runway without flying.

Collating requirements in order of business priority has a profound effect on design. At the simplest level, playing the odds, the code written first will experience the most tests over its lifespan. These tests come in the form of human interaction, automated tests, and the stress of adding new features. So the code representing the highest value features should enjoy the longest growth cycle. New features reinforce old features.

About the automated tests, let’s pretend we don’t know if Vu or Karen used them or not. We know platform Q provides their test rig. Tests may provide a high velocity, but they don’t strictly provide good steering. A car going very fast in the wrong direction will get into trouble very quickly.

Phased delivery and merciless refactoring are not merely ways to prepare for unknowable requirements. They are a system to find a cheap path to a valuable goal. Even if the goal is well known, the path is not.

Karen’s last big mistake was requesting a big-bang delivery, complete with one huge data migration, and retraining. Platform Q absolutely did not require these things - in fact both Q and all its user community endlessly advocate and support incremental releases and reversible migrations. A big-bang delivery would have lead to glitches, bugs, and down-time. Vu declined.

Version Three was Vu’s attempt to learn from the situation and recover his project. When he started, he knew a lot more about the requirements than he ever did. And he also knew the best design ideals. So, even using hoary old platform P, his job is almost as simple as transcribing a program from one language to another. Vu had two complete systems to draw from.

The vaunted technique of “Big Requirements Up Front”, hallowed in software engineering books written to this day, is hit-or-miss. If your project’s working cycle is only a few months, with a familiar platform, and if you already have complete prototypes with working designs, you have very good odds of success.

However, if you attempt to squeeze all those requirements, sideways, thru your pipe, you will get into trouble.

Just don’t blame platform Q. (-;