What is code coverage?
Code coverage, in short, is all about how thoroughly your tests exercise your code base. The intent of tests, of course, is to verify that your code does what it's expected to, but also to document what the code is expected to do. Taken further, code coverage can be considered as an indirect measure of quality -- indirect because we're talking about the degree to what our tests cover our code, or simply, the quality of tests. In other words, code coverage is not about verifying the end product's quality.
So, how does code coverage differ from other types of testing techniques? Code coverage can be classified as white-box testing or structural testing because the "assertions" are made against the internals of our classes, not against the system's interfaces or contracts. Specifically, code coverage helps to identify paths in your program that are not getting tested. It is because of this distinction that code coverage shows its usefulness when testing logic-intensive applications with a lot of decision points.
Needs for Code Coverage
To be able to use a code coverage tool we need a number of requirements, they are:
- Command line driven
- Xml reports
- Usable within any unit testing framework
- Support for up to .Net 4.0 framework
- Stable and supported
There are a number of tools available for code coverage, mainly in the Java world however here are a few in the .NET world
Question is which tool and why?
- The quickest answer would be dotCover as it is from JetBrains, however dotCover is currently only in Early Access Program, and perhaps in the future this would fill our needs.
- PartCover, does not appear to be well supported and a little tricky to get working.
- TestDriver.NET is perfect for checking out your code coverage from within Visual Studio and works well with Gallio too, however is not well supported and the documentation is very sparse.
- Which leaves us with NCover, although it used to be free, the owners have now produced a commercial version, which is very well supported, supporting all .NET languages, 32 and 64 bit support, no special builds required, Sequence point coverage reporting and Branch coverage reporting, XML coverage data output for integration to CI servers, any many more. It would be nice for each developer to have NCover, however the cost is quite high ($199.00 per user), and if budgets are restricted then a single version would be sufficient for the CI server.
I have not gone in to great detail about how to perform code coverage, as there is ample information on the internet, what I have tried to do is provide a simple justification to why code coverage is an integral part of the development process.
My recommendation would be NCover, mainly because it appear to be the only fully supported Code Coverage tool for .NET
Another Code Coverage tool, which is built in to Visual Studio Team edition, I have not covered here, as it was not available for me at the time of writing. Thanks Nathan Gloyn for point this out.