Legacy Documentation. View NUnit 3 Documentation

Release Notes

View Changes Separately for Each 2.5 Pre-Release

NUnit 2.5 Final Release - Version 2.5.0.9122 - May 2, 2009

General

  • NUnit now uses a new version numbering scheme. The first three dot-separated values represent the release level, as before, while the fourth is a build number tied to the date of the release.
  • This release includes pNUnit, an extended NUnit runner for distributed parallel tests. The pNUnit program was developed at Codice Software for use in testing the Plastic SCM and has been contributed to NUnit. For more info about using pNUnit see the pNUnit site.
  • The installer now offers Typical, Complete and Custom options. Selecting Typical gets you the NUnit framework, core, console runner and Gui. To install pNUnit or the NUnit tests select Complete or Custom.
  • There are no longer separate distributed packages for .NET 1.1 and 2.0. Both the binary zip and msi packages contain subdirectories for .NET 1.1 and 2.0. The Gui is now built for .NET 2.0 only. Tests may still be run under .NET 1.0 o 1.1 by running in a separate process. In the case of the msi, the user may elect to install either or both of them.
  • The Visual Studio solutions and projects are now in a directory tree that is parallel to the source tree. A VS2008 project has been added and the VS2003 project was removed. Certain components are still built for .NET 1.0 and 1.1 through the NAnt script.
  • Most of the code for Assert and for elements of the constraint syntax is now generated. This allows us to more rapidly deploy new constraints with their corresponding syntax. The file SyntaxElements.txt contains the specifications used in generating the code and the source of a tool for generating the source is included in the tools directory. The generated files continue to reside in the NUnit source at this time, so that those working in other areas of the code don't have to regenerate them each time they make changes.
  • The nunit.framework.extensions and nunit.core.extensions assemblies have been removed from the build.
  • NUnit is now built using NAnt 0.86 beta 1. The windows installer is created with WiX 2.0.5085

Framework - Attributes

  • NUnit 2.5 features parameterized tests, which are defined using a number of new attributes:
    • TestCaseAttribute allows the programmer to specify the arguments and a number of optional parameters inline.
    • TestCaseSourceAttribute identifies a property, field or method that to provide the arguments and other parameters.
    • ValuesAttribute is used on a method parameter to specify a set of values to be supplied as arguments inline.
    • RangeAttribute works like ValuesAttribute but lets you specify a range to be used for the arguments.
    • RandomAttribute works like ValuesAttribute but lets you use random values for the arguments.
    • ValueSourceAttribute is placed on a method paramter to specify the property, field or method that will supply arguments.
    • CombinatorialAttribute, PairwiseAttribute and SequentialAttribute indicate how data supplied for individual parameters is combined to form test cases. The default is Combinatorial, which supplies all possible combinations.
    • The presence of either TestCaseAttribute or TestCaseSourceAttribute on a method is now sufficient to identify it as a test even if it is not decorated with TestAttribute.
    • Test Cases specified using TestCaseAttribute or TestCaseSourceAttribute may include a Description, ExpectedException, Categories and Properties in addition to the arguments themselves.
    • Parameterized test methods may be generic. NUnit will deduce the correct implementation to use based on the types of the parameters provided. Generic test methods are supported in both generic and non-generic clases.
  • NUnit now includes an implementation of Theories, similar to what is found in JUnit. Support for Theories is provided by the Theory, Datapoint and Datapoints attributes and by the Assume.That method. For more information and further links, see the TheoryAttribute documentation page.
  • Other new attributes provided in NUnit 2.5 are:
    • RequiresThreadAttribute forces creation of a new thread and may optionally indicate the desired ApartmentState for the thread.
    • RequiresSTAAttribute causes the test to run in the STA. A new thread is created only if the parent is not in the STA. On methods, the .NET-provided STAThreadAttribute may also be used.
    • RequiresMTAAttribute causes the test to run in the MTA. A new thread is created only if the parent is not in the MTA. On methods, the .NET-provided MTAThreadAttribute may also be used.
    • TimeoutAttribute is used to set the timeout for tests. When used on classes or assemblies, the timeout value is used as the default timeout for all subordinate test cases. Test cases with a timeout value run on a separate thread and return a failure message if the timeout is exceeded.
    • The MaxTimeAttribute specifies a miximum elapsed time for a test case. If the test takes longer, it is reported as a failure. This attribute was previously available as an extension. Unlike TimeoutAttribute, MaxTimeAttribute does not cause the test to be cancelled, but merely times it.
    • RepeatAttribute causes a test case to be executed multiple times. This attribute was previously available as an extension.
    • RequiredAddinAttribute may be used to mark an assembly, indicating the name of any addin that is required to run the tests. If the addin is not present, the test assembly is marked NotRunnable.
  • TestFixture classes may now be generic. They must be marked with one or more instances of TextFixtureAttribute using the new constructor that takes an array of Types. NUnit will instantiate the fixture multiple times, using the type arguments provided.
  • Use of the TestFixtureAttribute is now optional in designating non-generic classes that contain tests marked with either TestAttribute or TestCaseAttribute.
  • More than one method may now be marked with the SetUp, TearDown, TestFixtureSetUp and TestFixtureTearDown attributes. Setups in a base class are executed before those in a derived class and teardowns are executed in the reverse order. If there are multiple setups or teardowns defined at the same level, the order is unspecified so this practice is not generally recommended.
  • PropertyAttribute has been modified internally to use a dictionary of name/value pairs rather than just a single name and value. This feature is not exposed for direct use, but may be accessed by derived attributes that want to apply multiple named values to the test. For a simple example of usage, see the code for RequiredThreadAttribute.
  • PlatformAttribute has been extended to accept the new keywords NT6, Vista and Win2008Server.
  • The MessageMatch enum used with ExpectedExceptionAttribute has been extended with a new value StartsWith, indicating that the exception message must start with the string provided.
  • TestCaseAttribute now supports a MessageMatch property.

Framework - Constraints

  • New constraints and corresponding syntactic constructs are provided:
    • Is.InRange tests that a value lies within a specified range.
    • Has.Attribute() tests for the presence of a specified attribute on a type and optionally applies further tests to that attribute.
    • Is.InstanceOf replaces Is.InstanceOfType, which is now deprecated.
    • Is.AssignableTo allows reversing the operands of AssignableFrom for increased clarity in the code and in any error messages when the actual value is of the derived type.
    • Throws.Exception allows testing the exception thrown by a delegate in place and provides the ability to make arbitrary tests on the caught exception. Throws.TypeOf() and Throws.InstanceOf() are provided as shorthand for the commonly used Throws.Exception.TypeOf() and Throws.Exception.InstanceOf.
    • Throws.Nothing provides for testing that a delegate does not throw. While it doesn't do much, it serves the purpose of converting an unexpected error into a test failure.
    • Is.Ordered allows testing a collection is ordered according to a supplied comparison criterion. (contributed by Simone Busoli)
    • Is.SamePath(string) tests that two paths are equivalent, without requiring that they exist in the file system.
    • Is.SamePathOrUnder(string) tests that one path is under another, without requiring that either path exists in the file system.
    • DelayedConstraint and the After syntactic element allow delaying the application of a constraint until a certain time has passed. See DelayedConstraint for features and limitations.
  • The parsing of constraint expressions written using the fluent interface has been reorganized internally, with the following benefits:
    • Meaningless sequences like "...Null.Null..." or "...And.Or..." will no longer compile - the NUnit tests actually verify this by attempting to compile them.
    • Syntax modifiers like Within and IgnoreCase are now only accepted - and shown by intellisense - on constraints that actually make use of them.
    • New And and Or infix operators are provided.
    • The With provides some ability to group constraints.
  • The "syntax helper" classes, Is, Has, Text, List and Throws, have been moved to the NUnit.Framework namespace, so that they may be used more easily.
  • PropertyConstraint now works with Types when testing for the existence of a named property.
  • EqualConstraint has been enhanced with several new modifiers, which may be used immediately after the Within(...) modifier to indicate how a numeric tolerance value should be interpreted.
    • Ulps = as units in the last place (floating point only)
    • Percent = as a percentage of expected value
    • Days = as a TimeSpan in days
    • Hours = as a TimeSpan in hours
    • Minutes = as a TimeSpan in minutes
    • Seconds = as a TimeSpan in seconds
    • Milliseconds = as a TimeSpan in milliseconds
    • Ticks = as a TimeSpan in ticks
  • The comparison constraints (GreaterThan, LessThan, etc.), RangeConstraint and CollectionOrderedConstraint may now be used with objects that implement IComparable<T>.
  • The syntax used for specifying that a collection is ordered has changed. Is.Ordered is now a property. The property name to use for ordering is specified using Is.Ordered.By(name).
  • The following constraints now accept a Using modifier to indicate that a user-specified comparer should be used:
    • EqualConstraint
    • GreaterThanConstraint
    • GreaterThanOrEqualConstraint
    • LessThanConstraint
    • LessThanOrEqualConstraint
    • RangeConstraint
    • UniqueItemsConstraint
    • CollectionContainsConstraint
    • CollectionEquivalentConstraint
    • CollectionSubsetConstraint
    • CollectionOrderedConstraint
    The provided comparer may be any of the following:
    • IComparer
    • IComparer<T>
    • Comparison<T>
    In addition, EqualConstraint may use:
    • IEqualityComparer
    • IEqualityComparer<T>
    Using C# 3.0, lambda expressions are also accepted.
  • A new syntax element, Matches(Constraint), allows use of custom constraints, predicates or lambda expressions in constraint expressions.

Framework - Asserts

  • New Assert methods:
    • Assert.IsOrdered allows checking that a collection is in a particular order. (contributed by Simone Busoli)
    • The DirectoryAssert class provides tests on directories:
      • AreEqual(DirectoryInfo, DirectoryInfo)
      • AreEqual(string, string)
      • AreNotEqual(DirectoryInfo, DirectoryInfo)
      • AreNotEqual(string, string)
      • IsEmpty(DirectoryInfo, DirectoryInfo)
      • IsEmpty(string, string)
      • IsNotEmpty(DirectoryInfo, DirectoryInfo)
      • IsNotEmpty(string, string)
      • IsWithin(DirectoryInfo, DirectoryInfo)
      • IsWithin(string, string)
      • IsNotWithin(DirectoryInfo, DirectoryInfo)
      • IsNotWithin(string, string)
    • Assert.Throws(Type expectedException, TestDelegate code) gives more control over tests of expected exceptions. TestDelegate is a delegate, which may of course be supplied as an anonymous delegate or lambda expression. If the correct exception type is thrown, the actual exception is returned from the method, so that further verification may be performed.
    • Assert.Throws(Constraint, TestDelegate) allows passing a constraint or constraint expression as the first argument, which is then applied to the resulting exception.
    • Assert.DoesNotThrow method tests that a delegate does not throw an exception.
    • Assert.Pass allows early termination of a test with a successful result.
    • Assert.Inconclusive terminates a test, returning the new Inconclusive result state.
  • Assert.That has been extended to allow a delegate or a reference as the argument. By default, these are evaluated before being used by the constraint supplied but some constraints may delay evaluation. The new DelayedConstraint is an example.
  • AssertionHelper has been updated so that the Expect overloads now include the signatures newly introduced for Assert.That.
  • NUnit now includes added functionality in the .NET 2.0 build of the framework. The following additional features are supported:
    • All Asserts and Constraints work with nullable types.
    • Some Asserts allow an alternate generic syntax for convenience:
      • Assert.IsInstanceOf<T>(object actual);
      • Assert.IsNotInstanceOf<T>(object actual);
      • Assert.IsAssignableFrom<T>(object actual);
      • Assert.IsNotAssignableFrom<T>(object actual);
      • Assert.Throws<T>(TypeSnippet code);

Framework - Miscellaneous

  • The following obsolete interfaces, classes and methods have been removed:
    • The IAsserter interface
    • The AbstractAsserter class
    • The Assertion class
    • The AssertionFailureMessage class
    • The old NUnit.Framework.TestCase class used for inheriting test classes
    • The Assert.DoAssert() method
    • Two ExpectedExceptionAttribute(Type, string) constructor
    • Two ExpectedExceptionAttribute(string, string) constructor

Core

  • NUnit now supports running tests in a separate process or in multiple processes per assembly. In addition, tests run in a separate process may use a different runtime framework from that under which NUnit is running.

    Note: In the Beta release, execution of tests under Mono from a copy of NUnit that is running under .NET is not yet supported.
  • NUnit now allows use of static methods as tests and for SetUp, TearDown, TestFixtureSetUp and TestFixtureTearDown.
  • Failures and Errors are now distinquished internally and in summary reports. Methods that are not run because they are invalid are also reported separately.
  • NUnit 2.5 is able to recognize, load and run NUnitLite tests.

Console

  • The console runner now supports a /framework option, which allows running the tests under a different version of the CLR.
  • The new /process:xxxxx command line option is used to run tests in a separate process or in multiple processes per assembly.
  • A new commandline option, /timeout:nnnn allows you to specify a default timeout value, which is applied to each test case in the run without a Timeout specified.
  • The summary report now displays Errors, Failures, Inconclusive, Ignored and Skipped tests separately. More detailed information on non-runnable tests and setup failures is provided.
  • The console summary report is no longer created using a stylesheet, which renders the /transform option meaningless. The option has been removed.

Gui

  • The Gui is now able to display the source code for test or production code from the stack trace, provided the assemblies contain source code information and the source code is available. Contributed by Irénée Hottier.
  • Reloading the tests after changing settings in a way that modifies the shape of the tree is now handled correctly.
  • The Settings Dialog now opens to the page that was last viewed.
  • The default gui display now uses a single tab for all text output. For users upgrading from an earlier release, the current settings are honored. If you wish to change to the new default, use the Restore Defaults button on the Text Output settings dialog.
  • The tree display shows non-runnable tests in red and inconclusive tests in orange. Inconclusive tests are temporarily listed on the Tests Not Run tab for this release.
  • The final test run display shows a more detailed summary: passed tests, errors, failures, inconclusive, non-runnable, skipped and ignored.
  • The Project Editor now allows setting the ProcessModel and DomainUsage for a project to control how that project is loaded and run. It also supports setting a target runtime framework for each configuration. If the target runtime is different from the runtime under which NUnit is running, the tests will be run automatically in a separate process under the target runtime.
  • The TestLoader settings dialog provides default settings for running tests in a separate process or in multiple processes if not specified in the NUnit project file.
  • The Project Editor no longer reloads the tests as each individual change is made. Tests are reloaded after the editor is closed and only if changes have been made to the overall project configuration or to the active configuration.
  • The Addin Dialog now shows an error message for any addin that fails to load. (from 2.4.8)
  • The Assembly Info display now uses a scrolling text box and has been enhanced to show information about the Process and AppDomain in which the test is running. It also shows two runtime versions for eachtest assembly: the one for which it was built and the one under which it is currently loaded.
  • The status bar now displays errors and failures separately.
  • Properties with a collection for a value are now displayed displayed as a comma-separated list in the properties window.

Extensibility

  • The implementation of constraints has been simplified so that it is now possible to provide modifier properties on a constraint without creating any additional classes. Such custom constraints can be integrated into the NUnit syntax by rebuilding the framework assembly.

    Note: The ability to create new syntactic elements without rebuilding the framework is planned for a future release.
  • A simpler method of providing new data point extensions based on attributes applied to the parameter itself is now available. Such attributes may be derived from ValuesAttribute and do not require any special addin in order to work.
  • New extension points TestCaseProviders and DataPointProviders allows addins to provide data for parameterized tests.
  • The following extensions are included in the nunit.core.extensions and nunit.framework.extensions assemblies:
    • The XmlConstraint extension, which allows comparing two xml files

Bug Fixes

  • Loading a single assembly and then adding another assembly using the Add Assembly menu item was not working correctly.
  • Reloading tests after settings changes was failing when the new settings changed the shape of the tree in such a way that the new tests didn't match the old ones correctly.
  • The Reload Project menu item was followed by a prompt asking if the current project state should be saved first and making the reload work in an unexpected way if you answered yes.
  • A class without a TestFixture attribute, containing only TestCase methods, without any Tests, was not recognized as a test fixture.
  • Assert.DoesNotThrow was failing to display a user message.
  • Xml documentation for Assert.IsNotEmpty and Assert.AreEqual was incorrect.
  • CollectionAssert.AreEqual and EqualConstraint were not working with IEnumerables that were not also Collections.
  • PlatformAttribute now distinguishes correctly between Vista and Windows 2008 Server.
  • The Gui output panes were failing to use a fixed font. (from 2.4.8)
  • NUnit tests of AssertThrows were not working on systems using non-English cultures.
  • Domains were not unloading correctly in some circumstances. Unloading is now done on a separate thread.
  • An NUnitSettings.xml file of zero length was causing a crash. (from 2.4.8)
  • Invoking the gui with /exclude:XXX, where XXX is a non-existent category, was causing all tests to be excluded. (from 2.4.8)
  • Categories were not working properly on repeated tests. (from 2.4.8)
  • A serious memory leak was fixed in the NUnit test runners. (from 2.4.8)
  • Static SetUp and TearDown methods were not being called in a SetUpFixture.
  • The current directory used when executing addins that load tests was not correctly set.

Earlier Releases