How to use new unit testing tools & techniques to improve software quality

In the past, the sting came from not being able to simply lift a software unit from its development environment, compile and run it " let alone supply it with test data.

For that to happen, you need a harness program acting as a holding mechanism that calls the unit, details any included files, "stubs" written to handle any procedure calls by the unit, and offers any initialization sequences which prepare data structures for the unit under test to act upon.

Not only was creating that process laborious, but it took a lot of skill. More often than not, the harness program required at least as much testing as the unit under test.

Perhaps more importantly, a fundamental requirement of software testing is to provide an objective, independent view of the software. The very intimate code knowledge required to manually construct a harness compromised the independence of the test process, undermining the legitimacy of the exercise.

The legacy from high integrity systems
In developing applications for the medical, railway, aerospace and defence industries, unit test is a mandatory part of a software development cycle - a necessary evil.

For these high integrity systems, unit test is compulsory, and the only question is how it might be completed in the most efficient manner possible. It is therefore no coincidence that many of the companies developing tools to provide such efficiency have grown from this niche market.

In non-safety-critical environments, perceived wisdom is that unit testing is a nice idea in principle, but commercially unjustifiable. A significant factor in that stance is the natural optimism which abounds at the beginning of any project. At that stage, why would anyone spend money on careful unit testing? There are great engineers in the team, the design is solid, sound management is in place. What could possibly go wrong?

However, things can, and do go wrong, and while unit test can't guarantee success, it certainly helps minimise failure. So, if we look at the tools designed and proven to provide quick and easy unit tests in high integrity systems, it makes sense that the same unit tests would provide a solid solution for those working on commercially developed software as well.

When is unit test justifiable?
Unit testing cannot always be justified. And, sometimes it remains possible to perform unit test from first principles, without the aid of any test tool at all.

There are pragmatic judgements to be made.

Sometimes that judgment is easy. If the software fails, what are the implications? Will anyone be killed, as might be the case in aircraft flight control? Will the commercial implications be disproportionately high, as exemplified by a continuous plastics production plant? Or are the costs of recall extremely high, perhaps in a car's engine controller? In these cases, extensive unit testing is essential and any tools that aid in that purpose make sense.

On the other hand, if software is developed purely for internal use or is perhaps a prototype, then the overhead in unit testing all but the most vital of procedures would be prohibitive.

As you might expect, there is a grey area. Suppose the application software controls a mechanical measuring machine where the quantity of the devices sold is low and the area served is localized. The question becomes: Would the occasional failure be more acceptable than the overhead of unit test?

In these circumstances, it's useful to prioritize the parts of the software which are either critical or complex. If a software error leads to a strangely coloured display or a need for an occasional reboot, it may be inconvenient but not in itself justification for unit test. On the other hand, the unit test of code which generates reports showing whether machined components are within tolerance may be vital.