A culture of 100% test coverage has grown up around TDD. Code quality “measuring” tools such as Clover are used as a pugil stick to beat developers over the head with, if their code doesn’t measure up to an arbitrary set of metrics: and one of these is a measure of how much product code is covered by unit tests.
(Clover does have some pretty awesome features by the way, such as test optimization; but institutionalising test coverage as a measure of code quality isn’t one of them).
TDD itself can become a highly elaborate, labour-intensive way of creating code. It’s not uncommon to end up with far more test code (which the customer never gets to see, and which arguably doesn’t really “do” anything) than product code, which is what the customer is really paying for. Yes, the customer is also paying to have product code of a sufficient quality, and that’s what TDD addresses. But there has to be a better way of improving code quality with tests: a way that doesn’t soak up developers’ time and turn the tests into the main software deliverable, in the developers’ minds.
There are three main shortcomings to TDD:
- It’s too time-consuming
- You end up with too much test code, with code paths duplicated by different tests
- The final amount of test code and product code that you see is disproportionate to the amount of refactoring/evolutionary design that went on to get there
To sum these three issues up: TDD is Too Damn Difficult; it takes too much effort to produce a bit of “production code” result.
Not using TDD – just writing tests after you’ve written some code – also has problems, in particular the aimlessness of this approach. It’s difficult to know when you’re done writing enough tests – in practice there won’t be enough time to hit 100% coverage, so the team must stop at some arbitrary timeboxed stage somewhere along the road. Most dissatisfying.
So here’s the idea. These shortcomings can all be addressed by first doing a small amount of structured up-front analysis and design, and identifying which parts of the code you’ll gain the most benefit from testing. We call this approach Design Driven Testing or DDT (the linked tutorial illustrates how to use an add-in for Enterprise Architect to automate much of DDT).
Take this sequence diagram, an overview of the code path for a hotel search (click the image for the full-size version):
The “story” behind the diagram is that HotelSearchClient has just made a call to a search service, and received an XML document containing any number of matching hotels. The task is to now parse this XML and turn it into a nice collection of Hotel objects that the client code can then play with.
I’ve colour-coded the diagram to show which method calls are on which object. What we’re mainly interested in is that, given an XML document containing a bunch of hotels, if we parse the document to produce a HotelCollection object (literally a collection of Hotels), are the hotels all present and correct?
Following along with TDD, you would write tests for the three constructors (Hotel, HotelCollection and HotelSearchXmlResponse), and for the methods Hotel.populateFrom(node), HotelCollection.add(node), HotelCollection.queryXPath(..), and HotelSearchXmlResponse.parse(). Each of these might end up with more than one test case, to cover all the permutations and ins and outs. That’s rather a lot of test code, and ultimately what does all of that actually give you?
But take another look at the sequence diagram. There’s basically one entry and exit point: the parse() method on HotelSearchXmlResponse. The point of the whole interaction is to generate a collection of Hotels from the XML document. What you’re interested in ensuring is that, given a known input document, the correct number of Hotel objects gets created, and each Hotel object is correctly populated.
So, at a minimum, you could write a test that uses a test XML document, fires off a call to parse(), and asserts that the HotelCollection returned is what you’d expect. That would pretty much cover it. If there are any parsing bugs along the way, this one test would break.
Some people might be uncomfortable with the idea of leaving some areas of quite complex code apparently uncovered; e.g. looking again at the sequence diagram, Hotel.populateFrom(node) is a significant method because it parses an XML fragment and populates the Hotel object. It’s a key method, in fact. So it would make sense to write one or more tests for this method. However, this is duplicating some of our “main” test, which is already set to assert that the Hotel objects are correctly populated.
In this case, simply moving these tests to Hotel.populateFrom(node) instead would cover it – basically moving the tests closer to the code they’re really testing.
So you’d end up with two tests:
- HotelSearchXmlResponseTest.testParse() – this will assert that the correct number of hotels is returned
- HotelTest.testPopulateFrom() – this will assert that the Hotel is populated correctly
So two tests give you everything that you’d really want to achieve from the unit tests; and a quick interaction diagram (which should only take 10mins or less to sketch out) gives you the overview needed to make the call on which are the most important methods to test, before the code has been written.
This technique is a fundamental part of Design Driven Testing (DDT). DDT does go further; in addition to unit tests, there are also behavioural tests, which are driven from “controllers” (aka logical software functions) that are identified from use cases; and acceptance tests, which are end-to-end, integration tests, each one based on a single use case.
Comments