Although definitions can be tricky, unit testing has been pretty intrinsic to extreme programming methods, as well as agile development styles that have come along subsequently. While unit testing in and of itself can help drive a better development process, some believe it does not necessarily support an agile test-driven development (TDD) process.
But test tools such as csUnit, MbUnit and TestRunner, some claim, can have special use for agile development of Visual Studio applications. Meanwhile, some of these same individuals question whether Visual Studio Team System, or VSTS, adequately supports a true Test-Driven Development process.
Manfred Lange, one of the csUnit Lead Developers, explained, "There are a lot of people who believe that unit testing is one of the agile practices (XP in particular), and that when unit testing is used, a team has already adopted a first agile technique. Unfortunately, this is not quite true."
"There are two battles going on and most people only see one. One is getting developers to practice unit testing and two is getting them to practice test-driven development," said Jay Flowers, a design lead at Northrop Grumman. "There is a lot of legacy code out there -- legacy code as defined by [noted programmer and consultant] Michael Feathers as 'any code without unit tests.'
"I think the key to winning the first battle is to make it easy to get developers' legacy code under test. Getting their legacy code under test, and under test easily, will flow into adoption of TDD," Flowers continued. "Getting legacy code under test is a technical issue where our adoption of TDD is a behavioral change." A success in the first battle will give momentum to attacking the second."
Test first, ask questions later
In TDD the test always comes first, while in unit testing this is not always the case. From this perspective, TDD programmers are not able to use the automated test generation features available in VSTS, according to some.
Lange said, "The basic idea of TDD is: Test a little, code a little, refactor a little -- in that sequence! And the rhythm here is not in hours or days."
He continued: "The rhythm for the entire sequence is seconds or minutes. Run tests frequently, preferably as often as possible. With TDD the test always comes first, while with 'unit testing' in the broadest sense this is not clear. Generating tests after-the-fact is not considered to be TDD. Sometimes I even prefer the term 'Test-Driven Design' as it makes clearer what this really is about."
On the other hand, unit testing after the code is developed is not necessarily bad, it just does not serve the goals of TDD. Lange explained, "Some testing is always better than no testing. However, software that is implemented using a test-driven approach, if done properly, will result in simpler systems, which are easier to understand, easier to maintain and with a near-zero-defect quality. The design will look differently, and, if the developer has sufficient courage, even the architecture can emerge over time using this approach (instead of up-front architecture or up-front design)."
The main benefit of the automated test generation features in VSTS is to help get the legacy code under test, Flowers noted. Although he suggested it is not a very smart code generator, and is writing a better one called Doubler.
Will Ballard, Owner of Mailframe, which makes the TestRunner tool for Visual Studio , said, "With VSTS, the unit testing tools are most useful in running a large battery of tests, and not well suited to single test case iteration in TDD. I like that it is integrated, but I dislike the 'list view' style visualizations in VSTS: Too much scrolling and shuffling as you get a real-size set of tests in an assembly, which for me are often in excess of 1,000 test cases."
In contrast, other tools might be more suited for TDD. "With TestRunner," claimed Ballard, "the focus is on the in-editor experience that is central to test-driven or test-first development, where the coding takes place as a continuous test-code-test sequence of activities."
He continued: "You can run from the 'test tree' view, but you can get at all the test functionality right there in place as you type a test case or change your application code. Integrating profiling and code coverage with unit testing was the real innovation, as you can see how well your tests are exercising your code. I found that folks using external coverage tools simply gave up on them after extended use, as it became a cumbersome additional step."
Another testing framework is MbUnit, which Flowers has been working on. He said it is easy to extend, has more types of test fixtures and supports an add-in called TestDriven.NET.
He noted, "TestDriven.NET can run test written with just about any framework. So in the end the only reason to buy VSTS is to enforce your process."
"These other tools do a better job of encouraging users to follow the process than VSTS does. In the end I see MbUnit, Doubler, TestDriven.NET, NCover, NAnt and CCNET as a better toolset than VSTS, and this toolset is free and will continue to evolve and release new features at a much faster rate than VSTS can," Flowers suggested.