Wednesday, January 11, 2012

To Unit Test or Not To Unit Test


Here's a link to Tom Fischer debunking the two biggest arguments for adopting unit testing:
1. Unit tests cost less in the long run: the author is right, this is a weak argument. By the time the benefits kick in the system is in the hands of maintenance engineers in a lot of cases.
2. Unit tests cause less bugs: he's right again. Unit tests mostly catch the errors developers can predict.

I agree with the author's general point: unit testing is can be overrated. In my experience unit tests have increased the quality of code in tangible SOLID ways, but decreased my overall speed of producing working code due to "going it alone" which I regret. They slow you down for a couple reasons:
1. The maintenance of unit tests. A example scenario: change your method signature to add a parameter. Each consuming test needs that new parameter, which can be dozens of tests. Even if you refactor your tests and use optional parameters to minimize impact of this exact issue, this still eats time and effort and happens routinely as part of healthy refactoring.
2. As a team, you need full buy-in on the tests because other developers will change your methods and cause broken tests. Without full buy-in the original unit test author (you) finds the broken tests. What to do then? rewrite? drop them? Investigate why they are broken and rewrite? This will happen every morning to the lone TDD developer on a team project. Code changes happen and the uninitiated will leave you in a sea of failing tests rather than fix your failing tests.
3. You end up writing tests after the fact when you could just hammer out working code. Sometimes you just know what you need to write and should simply get it done.

The TDD purists would say "you're doing it wrong" to #3. "Write tests and code as one" is the argument I hear, and I'm sympathetic to that, but dont have enough experience with it. Mostly because you need full team buy-in or else the time to face up and obliterate a sea of red X's each morning.
Also I do a fair bit of web UI programming; I use tools like Telerik to enhance productivity and get the front end users expect without the styling overhead. These activities are immune to TDD (how do you write test-first grid columns?).
I might encourage TDD for the benefits of better code and a nice suite of developer-only tests for more confidence refactoring (at the cost of maitenance). But if I were signing the cheques, I'd recommend against it (unless the majority of the team is from a TDD background, it depends), and spend the testing dollars on reliable UI-first testing and real repeatable test suites that actually test functionality the users care about. And as for quality code, I see code reviews as a more reliable way to impart code quality plus it has many more incidental benefits. More on code reviews and testing strategies some other time.
Last, unit tests can cause of false sense of security in developers and managers. I have seen this mistake propigate first hand. "Code coverage" of tests shows nothing. It is a sincerely useless statistic unless your inputs are fixed. Unit tests as a test suite is also a flawed notion; as if their mere existance proves the Completeness of the application. Obviously proving the correctness of many methods is a Good Thing, but its only the complete picture when you've tested how those parts work when put together.
Correctness aside, a flood of green checkmarks seems to have a numbing effect on managers brains. The overconfidence leads to mind numbingly impossible statements like "we're 90% done this project, and we'll have the users touch the system for the first time next week!" This fantasy 90% is based on how many requirements are marked done. But if your users havent OK'ed each one, then nothing is really done. That project is actually at 0%. Painful thought for the managers of little software background but that's the real truth. That project turned out to actually be at less than 50% complete. Not funny for the guy signing the cheques. (Of course, reliable done criteria would help this hypothetical manager avoid this predictable pitfall.)

TL;DR; unit testing is a personal development style that has to be adopted by an entire team. It is not free, even with TDD because of the maintenance burden tests carry with them. It can create a false sense of security causing essential User-focused testing to fall to the wayside. TDD/Not TDD is a significantly less important question than code review processes and user-focused testing processes.

No comments:

Post a Comment