November 7, 2009
Following up on my post from yesterday I re-read the revised version of Fowler’s Mocks Aren’t Stubs article. With the whole article being a good read, this is the part that I consider the most important to me:
It’s at this point that I should stress that whichever style of test you use, you must combine it with coarser grained acceptance tests that operate across the system as a whole. I’ve often come across projects which were late in using acceptance tests and regretted it.
Most projects I worked on, most teams I worked with, did some kind of testing. None TDD. None came even close to what the grand masters are doing in this area. But we all tried our best.
However, what most of us never realized is that unit testing is really just that. *Unit* testing. I remember when back in 2003 (plus minus a year) I worked on a Java (Micro Edition) navigation software and I wrote tests (which I called “system tests” back then) for running the whole app in a ‘headless’ emulator and logging all navigational actions. The test then compared this to the expected output. Running time was up to a minute for some of these tests. In another corner of this project I wrote “graphical tests” which rendered the 2D and 3D view of the navigation system and compared the images to the expected output. After I left the project, the colleagues quickly dropped most of the tests. (In fact, most of the unit tests were dropped, too. They considered them a burden. But that, I guess, is a different story.) I, however, knew how helpful these tests were. More than once did I make a simple change in an ‘unrelated’ class and it would magically affect the output of the ‘advisor’ component – not drawing the correct direction sign anymore. The tests didn’t necessarily help me to spot the source of the bug. But they made sure I did not commit breaking changes.
On a more recent project, a proxy server written in Ruby, I applied three levels of “system tests”: I used “spike tests” to drive a set of objects from all layers, but with a SQLite3 database instead of the real thing. Then I added “black box tests” running the system as a whole and hitting it with certain HTTP requests. Still using the SQLite3 database setup and running the tests on the local developer machine or the build server. Plus I added “deployment tests”. More or less the “black box tests”, but running against the deployed system. Because the proxy was the place where I had to add quick changes (because the proxied system was to hard to change), these tests allowed me to move very quickly from changes to deployment (to the test system) and on to production.
To make this long story short: Two things are important for me. 1. I am all with Uncle Bob in that you should always consider the database just an implementation detail. Make sure from the very beginning that you can replace it with an in-memory or SQLIte3 database. Or flat file. Or fake. Or whatever. 2. Use fine-grained unit tests. Plus, use coarse-grained acceptance tests. (What I called “system tests” or “spike tests” above.)
I’ll continue reading up on mocking and testing etc.. I feel I have a lot more catching up to do.. :/