Guard test, my brand new friend

November 1 2008

ever had to work with external subsystems? even worse, had to write an automated test which has to communicate with those subsystems? it can be a nightmare, and as soon as you don’t expect, you find yourself in a mess trying to update the test suite with new test data, in order to have the test pass.

assuming the contract it’s clear, the simplest solution is to write a unit test on our code using a test double implementation of the system (where “test double” stands for fake, stub or mock – choose one). that’s fine, and best of all, you can reuse that double inside an acceptance test too. you decide to focus on the business rule and isolate from real subsystems. [ Matteo wrote a great essay on this, telling a story from one of our current customers. ]

yep, but i’m still waiting for a green bar on real subsystem calls! sometimes you really need an automated test glueing together application logic and external infrastructure, such as a DataAccessObject (aka gateway or broker) to a stored-procedure written and maintained by another team. you are in charge of passing correct values to the dao, depending on some business scenarios, so you really wants to know when (not if) something stops working: database not tuned, high data volumes, network troubles, and so on.

wait, you can use an integration test, don’t you? that’s true: but, insted of testing our code calling external system, why don’t we test that system itself? we can use real data, the same we’re going to use inside an acceptance test, on a real scenario. this is something we’re using since a while in our team, and it seems it’s working. we call these guard tests: when an acceptance test starts failing, there’s always another test broken, a guard test, telling us “hey, relax! it’s not so bad”.

now, it’s time for an example. the first time we introduced a guard test was for a story like “user selects a market sector. when news on companies from that sector arrive, the user is notified”. we had to integrate with an existing dao for sector -> companies association. so, we found ourselves writing a characterization test for that dao, something like “given sector General-Financial, we expect to find at least two companies: company X and company Y”. this is our guard test. having clear how to use the dao, we unit tested the code in our application using it – an adapter. finally, we wrote an acceptance test for the whole story, using real data: we simulated a news for company X and added and assertion on user being notified.

to recap, we had:

  • a unit test for how our adapter uses the dao, with a test double
  • an integration test for the dao itself – the guard test
  • an acceptance test using the real dao

everything worked fine, at least for the first few weeks. then, as expected, data migration happened, and the build was broken. [ yes, it’s really strange to see market sectors composition change! we finally found out that earlier database data were fake – manually added by developers. ]

the happy part of the story is that we had both acceptance and guard test failing: good news, it’s just a matter of data! we spent a bit investigating the database content (while waiting for an automatically-deployed-development-database with dbdeploy) and fixing both tests.

so, what’s the morale? it goes like this: try to understand business rules with acceptance tests isolated from infrastructure details. good. anyway, if you need to write AT using real systems, add a guard test too: two red bars are better than one!


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: