Saturday, May 1, 2010

Running Tested Features (RTF)

Agile Projects: Early Features

An Agile project really does focus on delivering completed features from the beginning. Assuming that the features are tested as you go, the RTF metric for an agile project could look like this:
This is a very simple metric, often called the “burn-up” chart. Real live features, done, tested, delivered consistently from the very beginning of the project to the very end. Demand this from any project, and they must become agile in response to the demand.

Waterfall: Features Later On

Waterfall-style projects “deliver” things other than RTF early on. They deliver analysis documents, requirements documents, design documents, and the like, for a long time before they start delivering features. The theory is that when they start delivering features, it will be the right features, done right, because they have the right requirements and the right design. Then they code, which might be considered Delivered Software, but not Running Tested Features, because the features aren’t tested, and often don’t really run at all. Finally they test, which mostly discovers that the RTF progress wasn’t what they thought. It looks like this:
Look at all the fuzzy stuff on this picture. There’s blue fuzzy unpredictable overhead in requirements and design and testing. We can time-box these, and often do, but we don’t know how much work is really done, or how good it is.
And the apparent feature progress itself may also be fuzzy, if the project is planning to do post-hoc testing. We think that we have features done, but there is some fuzzy amount of defect generation going on. After a bout of testing — itself vaguely defined — we get a solid defect list, and do some rework. Only then do we really know what the RTF curve looked like.
The bottom line is that a non-agile project simply cannot produce meaningful, consistent RTF metrics from day one until the end. The result, if we demand them, is that a non-agile project will look bad. Everyone will know it, and will push back against the metric. If we hold firm, they’ll have no choice but to become more agile, so as to produce decent RTF.