News Stay informed about the latest enterprise technology news and product updates.

Microsoft unit testing effort grows, still faces NUnit challenge

The big unit testing trend in the .NET space this year seems to be the rollout of unit test software on Visual Studio 2008 workstations. But NUnit, recently released as NUnit 2.4.6, remains a challenging presence, and the open-source mbUnit framework continues to find adherents.

Microsoft continues to upgrade its offerings for developers who are intent on unit testing. But NUnit tools remain a strong alternative. In fact, NUnit may be the greatest open-source software success to date in the world of .NET development.

Unit testing has gained in use ever since the arrival of Extreme Programming and Agile processes in the late 1990s. Its antecedents go as far back as Smalltalk, which had a tester known as SUnit. This work later saw new life as JUnit, which became a popular Java unit testing framework that in part begat NUnit, a .NET open-source tester.

NUnit itself is at a crossroads, as its stewards look at a roadmap for future versions. The latest NUnit release is NUnit 2.4.6, released at the end of 2007.

Microsoft's unit test alternatives are also at a crossroads of sort. The company initially rolled out its unit testing framework -- referred to as VSTest, VSTS Test or MSTest -- as part of its Visual Studio Team System product line but has recently released unit testing software on the desktop as part of Visual Studio 2008 Professional Edition.

Will high visibility of NUnit within the more elite ranks of .NET developers translate into majority use? There is no question that NUnit support is vast. A broad survey of .NET developer ranks that this site undertook (along with sister site TheServerSide.NET) in 2007 shows that 33.2% of respondents used NUnit.

But Microsoft's decision to pass over NUnit as it created its Visual Studio Team System for Test certainly did not spur NUnit along. Moreover, good testing has always required special efforts – special efforts in the face of an overriding drive to get software out the door. Some developers' advocacy of unit testing is suspect. Tests can be fashioned to ensure code will pass. The segment is still new.

Why didn't Microsoft opt to support NUnit? Some might say because it was not invented there. But there is another, slightly different take. ''The problem for them was that NUnit is open source,'' said Chris Menegay, trainer-consultant with notion Solutions. ''They need to own the code.''

He indicated that Microsoft did look at what was good in NUnit and made an effort to build some of that into its tools.

More on unit testing in the .NET Framework
VSTS meets unit testing, part 1: Unit testing basics

VSTS meets unit testing, part 2: Integration and automation

VSTS meets unit testing, Part 3: TDD tools
Microsoft's initial unit testing tool efforts did not appreciably diminish NUnit interest because, in Menegay's words, it remained a "better-performing" tool. Some people had thousands of test cases and they took too long to run on the new Microsoft offering, he said.

''Now, with Visual Studio 2008, Microsoft did a lot of performance improvements, so more customers are looking at it more seriously, '' he said. He also noted improvements to the developer interface.

Clearly a move to any new test framework would take time, at least for organizations that have been testing aggressively. Moves away from established NUnit practices are not taken lightly. But various tools are offered to allow Visual Studio users to continue to use NUnit.

Today, NUnit remains the main standard, with mbUnit gaining traction as well, due to ease of extensibility and useful build-in attributes, said Microsoft MVP Roy Osherove.

MSTest is useful as Microsoft's way of bringing the idea of unit testing to the .NET masses, said Osherove in a recent e-mail.

''It's not perfect but the fact it exists is great and I think we'll see a nice parallel life situation between the two or three major frameworks out there,'' he said.

''I think NUnit and mbUnit will coexist for a long time until we'll see some real changes or merges between them,'' said Osherove. (When he speaks of a long time, he is not talking eras. A long time here is "at least a year or two.)

Long-time MVP Osherove, who is writing a book on The Art of Unit Testing, available now in an early-access edition from Manning Publications, recently joined Typemock. The company makes the Typemock Isolator for clearly separating components under test.

NUnit gets the job done, but the library testing tools in Visual Studio makes creating tests extremely easy, according to John Robbins, Wintellect instructor, consultant and co-founder. ''The integration with the IDE [for] creating, debugging, and running tests is extremely good," he wrote via e-mail.

We asked Robbins if NUnit, mbUnit and VSTS Unit Test -- or whatever Microsoft eventually calls it -- would continue to coexist.

He answer was yes -- absolutely.

''In fact,'' Robbins pointed out, ''the Microsoft Patterns and Practices Group has been releasing its source code for a while where they take advantage of some neat tricks to allow the code to run under either NUnit or VSTS Unit Test depending on a compilation define.''

He continued, ''mbUnit is a great tool, especially with its outstanding multithreaded testing. What I've been recommending in our consulting business is to use VSTS Unit Testing tools for the ease of use and IDE integration and use mbUnit for all your multithreaded and advanced data passing tests.'' That way, he suggested, you get the best of everything.

Dig Deeper on .NET Framework application performance management

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.