Software Reality

Test Smarter, Not Harder

A culture of 100% test coverage has grown up around TDD. Code quality “measuring” tools such as Clover are used as a pugil stick to beat developers over the head with, if their code doesn’t measure up to an arbitrary set of metrics: and one of these is a measure of how much product code is covered by unit tests.

(Clover does have some pretty awesome features by the way, such as test optimization; but institutionalising test coverage as a measure of code quality isn’t one of them).

TDD itself can become a highly elaborate, labour-intensive way of creating code. It’s not uncommon to end up with far more test code (which the customer never gets to see, and which arguably doesn’t really “do” anything) than product code, which is what the customer is really paying for. Yes, the customer is also paying to have product code of a sufficient quality, and that’s what TDD addresses. But there has to be a better way of improving code quality with tests: a way that doesn’t soak up developers’ time and turn the tests into the main software deliverable, in the developers’ minds.

There are three main shortcomings to TDD:

1. It’s too time-consuming
2. You end up with too much test code, with code paths duplicated by different tests
3. The final amount of test code and product code that you see is disproportionate to the amount of refactoring/evolutionary design that went on to get there

To sum these three issues up: TDD is Too Damn Difficult; it takes too much effort to produce a bit of “production code” result.

Not using TDD – just writing tests after you’ve written some code – also has problems, in particular the aimlessness of this approach. It’s difficult to know when you’re done writing enough tests – in practice there won’t be enough time to hit 100% coverage, so the team must stop at some arbitrary timeboxed stage somewhere along the road. Most dissatisfying.

So here’s the idea. These shortcomings can all be addressed by first doing a small amount of structured up-front analysis and design, and identifying which parts of the code you’ll gain the most benefit from testing. We call this approach Design Driven Testing or DDT (the linked tutorial illustrates how to use an add-in for Enterprise Architect to automate much of DDT).

Take this sequence diagram, an overview of the code path for a hotel search (click the image for the full-size version):

Design Driven Testing - New book on the way

Just what the world needs, another book on Test Driven Development (TDD)??

Well not exactly... Along with co-author Doug Rosenberg we've recently signed up with Apress to write a book about automated testing (i.e., unit testing, acceptance testing, new-kid-on-the-block behavioural testing, and requirements testing).

But... this won't be your everyday "toe the party line" automated testing volume that regurgitates the TDD doctrine. Instead, we're making the point that TDD is, to put it mildly, ass-backwards. Tests shouldn't be the sole design driver; instead the design can be used to identify the key areas of the code to cover with tests. In other words the design should drive the unit tests, just as requirements drive the customer acceptance tests.

We've found that, while it has much in its favour, TDD represents too much work for what you get out of it. For example, we definitely don't recommend that you go all-out for 100% test/code coverage, because you'll never achieve it in the time available... so what's the measure of success? How do you know when you've written enough tests and it's time to stop? Instead, we believe you should "test smarter, not harder" - in other words, identify the key areas of the code that will benefit the most from tests.

There's a chapter on Design Driven Testing in our previous book, Use Case Driven Object Modeling with UML: Theory and Practice. However, in this new book we'll be exploring in much more detail the ideas and the practicality of automated testing, using a real-world, non-trivial project ("Mapplet 2.0") being developed at ESRI for a travel website. This project will use a Flex front-end hooked up to Java server code, which in turn connects to a hotel search XML database and ESRI's ArcGIS Server. So we'll be exploring how to go about unit testing an enterprise application with a mix of languages and technologies (just like the majority of real-life projects out there).

To top it all, Sparx Systems has developed an ICONIX Process add-in for their UML modeling tool, Enterprise Architect (EA), which automates many of the steps that we'll cover in the book. You don't have to use EA to use Design Driven Testing, but after seeing what's possible with the emerging tools support for the process, we believe you'll want to!

Stay tuned, I'll post more about the ideas behind Design Driven Testing over the next few days.