Thoughts on Unit Testing

Unit testing is a topic that brings up a lot of very strong opinions, especially in the Mac community. But even with all these opinions, the fact is that a lot of developers don't do it. I've started using unit tests in Lighthouse Keeper 1.1 so I thought I would give an overview of how and why I've used them.

One of the most annoying parts of software is testing. It takes forever and is boring and repetitive. And worse of all, you often miss stuff, often stuff that is hard to track down. One way to make part of this easier is to use unit tests.

There are two extremes to the argument over whether you should write unit tests:

- "You should write tests for absolutely every single method and/or function in your code"

- "Unit tests are boring, a waste of coding time and don't test everything anyway so don't write any"

As with anything, the truth is often found in the middle ground. Some of the 2nd quote is true: unit tests ARE boring (at least for most devs) and they don't test everything. However, they may not be a waste of coding time if used properly as they can save you a huge amount of testing and debugging time.

The 1st quote is also completely true in some cases. If you are writing software for planes, nuclear power stations, rockets etc I would expect you to be writing unit tests for everything (as well as various other sorts of tests). If something goes wrong in these situations due to a program error it potentially lead to the loss of human life.

Unit Tests in Lighthouse Keeper

But let's assume you aren't writing software that could potentially kill someone. Do you need to cover every part of your code with unit tests? No. In Lighthouse Keeper I just have tests for the backend that works with the Lighthouse API. There are two reasons for this:

1. In an MVC app, the model layer is the easiest layer to tests as it should work completely in isolation from the rest of the application.

2. This is where most of the complex code that is.

Your time is precious so you want to get the biggest benefit from the time you spend writing tests. As such you should target the parts of your code that are most likely to gain bugs, especially ones that may not be immediately noticeable in normal use.

In the backend of Lighthouse Keeper I have a lot of code that parses data, or filters content based on certain logic. These can easily be broken by accident while I'm working on something else. These are ideal candidates for writing unit tests. But the objects that simply store data don't have any tests related to them at all as they only have simple accessor methods. These methods are unlikely to gain bugs and if they do they'll be pretty easy to detect and fix.

Future Unit Testing

While this is where my unit testing is now, I'm wanting to expand it further. While there is a lot of complex code in my model classes, there is also some easily breakable code in other layers of my code. Unfortunately this is much harder to test.

One of the big problems is designing for testing. Some parts of this are sensible and result in better, cleaner code. But I've found some parts actually lead to more code as you're building to make things easy to tests, not to make things simple for you. Model classes are very often designed well for testing by their nature, with controllers you have to put more effort in.

One thing I won't be heading towards is Test Driven Development. This is where you write your tests first and then code later. To me I find that a far too rigid form of development. It requires a lot of thought and planning before you start coding which goes against the way that I work, which involves a lot of chopping and changing. It leads to a more haphazard way of developing but I feel it helps me find the best solutions to problems. Ironically, while this form of development isn't ideal for doing Test Driven Development, it is ideal for automated tests such as unit tests as they help prevent me breaking stuff I know works.

OCRunner

One of the big problems with unit testing in Cocoa is that the tools are very weak. Xcode uses the same UI for unit tests failures as for compiler errors and warnings. It also requires you to go into the raw build log to see individual test details, multi-line errors, timings and other information.

To solve this I spent a few days coding up a replacement UI and this resulted in OCRunner. It's far from complete but it's already very useful for getting more information about your unit tests than Xcode provides in its UI. And best of all, it's released under the MIT licence so you can play around with it yourself. The source is available on Launchpad.

Unfortunately other commitments (aka my degree) has prevented me from working on it for a while, but when I next have some time I'm hoping to add some more features and polish and get a built version online for people to download. If you have any feature requests of find any bugs, then be sure to file them in the bug tracker.

(7) Comments

---