Tolerances

A tolerance lets an assertion pass when the actual value is close enough to the expected value rather than exactly equal to it. Tolerances are essential for floating point arithmetic, for values that pass through storage with lower precision (a database that keeps microseconds, a JSON serializer that rounds), and for timestamps that are taken a moment apart.

NUnit expresses a tolerance with the Within modifier, optionally followed by a modifier that says how the amount should be interpreted:

Assert.That(actual, Is.EqualTo(expected).Within(0.01));          // plus or minus 0.01
Assert.That(actual, Is.EqualTo(expected).Within(1).Percent);     // plus or minus 1 percent of expected
Assert.That(actual, Is.EqualTo(expected).Within(2).Ulps);        // within 2 representable floating point values
Assert.That(actual, Is.EqualTo(expected).Within(5).Seconds);     // plus or minus 5 seconds

This page describes how tolerances work across NUnit, which types support them, what happens when a tolerance cannot be applied, and how to get tolerant comparisons for types NUnit does not know about.

Where tolerances can be used

Constraint or API Syntax Notes
EqualConstraint Is.EqualTo(x).Within(t) Supports all tolerance modes. Applies to elements of arrays, collections, dictionaries, tuples.
GreaterThan, LessThan and the OrEqual forms Is.GreaterThan(x).Within(t) Linear and Percent tolerances. See Comparison constraints.
Properties comparer Is.EqualTo(x).Within(t).UsingPropertiesComparer() Applies the tolerance to every property that can take one.
Properties comparer configuration .UsingPropertiesComparer(o => o.Within(t)) From NUnit 4.4. Separate tolerances for numeric and time properties.
Item constraints such as Has.Some, Has.Exactly(n) and Is.All Has.Some.EqualTo(x).Within(t) The tolerance belongs to the per-item EqualTo.
Assert.AreEqual (classic) Assert.AreEqual(expected, actual, delta) Linear tolerance for double only.

The following do not accept a tolerance:

Tolerance modes

Linear

A plain Within(amount) is a linear (absolute) tolerance. The assertion passes when the difference between the actual and expected values, in either direction, is no greater than the amount. The tolerance and the values may be of different numeric types.

[Test]
public void Tolerance_Linear()
{
    // The actual value may differ from the expected value by at most the tolerance, in either direction
    Assert.That(4.99, Is.EqualTo(5.0).Within(0.01));
    Assert.That(5.01, Is.EqualTo(5.0).Within(0.01));
    Assert.That(5.02, Is.Not.EqualTo(5.0).Within(0.01));

    // Integers accept an integer tolerance
    Assert.That(98, Is.EqualTo(100).Within(2));

    // Numeric types may be mixed freely
    Assert.That(5.5, Is.EqualTo(5).Within(0.5));
    Assert.That(10m, Is.EqualTo(10.4).Within(0.5));
}

Percent

Within(amount).Percent interprets the amount as a percentage of the expected value. Because the allowed deviation is relative to the expected value, an expected value of zero only matches an actual value of zero.

[Test]
public void Tolerance_Percent()
{
    // 5 percent of the expected value 100 means anything from 95 to 105 passes
    Assert.That(97.0, Is.EqualTo(100.0).Within(5).Percent);
    Assert.That(104, Is.EqualTo(100).Within(5).Percent);
    Assert.That(106, Is.Not.EqualTo(100).Within(5).Percent);

    // The percentage is relative to the expected value, so an expected value of zero only matches zero
    Assert.That(0.0, Is.EqualTo(0.0).Within(5).Percent);
    Assert.That(0.001, Is.Not.EqualTo(0.0).Within(5).Percent);
}

Ulps

Within(amount).Ulps interprets the amount as a number of units in the last place: the number of representable float or double values between the expected and actual values. One ULP is the smallest step a value of that magnitude can take, so an ULP tolerance automatically scales with the size of the numbers being compared. For numerical work this is often safer than a fixed tolerance, which is either too loose for small values or too strict for large ones.

[Test]
public void Tolerance_Ulps()
{
    // 0.1 + 0.2 is not exactly 0.3 in binary floating point...
    Assert.That(0.1 + 0.2, Is.Not.EqualTo(0.3));

    // ...but it is the very next representable double, one "unit in the last place" away
    Assert.That(0.1 + 0.2, Is.EqualTo(0.3).Within(1).Ulps);

    // A fixed tolerance that suits small numbers is meaningless for large ones; ULPs scale with the value
    Assert.That(20000000000000004.0, Is.EqualTo(20000000000000000.0).Within(1).Ulps);
}

ULP tolerances apply to float and double values only.

Time units

For DateTime, DateTimeOffset and TimeSpan values the tolerance is a TimeSpan. You can pass one directly, or pass a number followed by one of the unit modifiers Days, Hours, Minutes, Seconds, Milliseconds or Ticks.

[Test]
public void Tolerance_Time()
{
    DateTime expected = new(2024, 1, 1, 12, 0, 0, DateTimeKind.Utc);
    DateTime actual = expected.AddSeconds(45);

    // Pass a TimeSpan...
    Assert.That(actual, Is.EqualTo(expected).Within(TimeSpan.FromMinutes(1)));

    // ...or a number followed by a unit
    Assert.That(actual, Is.EqualTo(expected).Within(1).Minutes);
    Assert.That(actual, Is.EqualTo(expected).Within(60).Seconds);
    Assert.That(actual, Is.Not.EqualTo(expected).Within(30).Seconds);

    // TimeSpan and DateTimeOffset values work the same way
    Assert.That(TimeSpan.FromMinutes(59), Is.EqualTo(TimeSpan.FromHours(1)).Within(1).Minutes);

    DateTimeOffset expectedOffset = new(expected);
    Assert.That(expectedOffset.AddMilliseconds(400), Is.EqualTo(expectedOffset).Within(500).Milliseconds);
}
Note

From NUnit 4.3, Is.EqualTo has dedicated overloads for DateTime, DateTimeOffset and TimeSpan. On those, a numeric tolerance must be followed by a unit: Is.EqualTo(someDateTime).Within(5) is not a usable constraint and the assertion does not compile. Conversely, the unit modifiers are not available on numeric values, so Is.EqualTo(5).Within(1).Seconds does not compile either. If the expected value is typed as object, these mistakes are caught at run time instead, as described below.

Supported types

NUnit can apply a numeric tolerance to the built-in numeric types: byte, sbyte, short, ushort, int, uint, long, ulong, decimal, float and double. Values of different numeric types may be compared with each other and the tolerance may be of yet another numeric type. char values are compared exactly, even though they convert to numbers.

A TimeSpan tolerance can be applied to DateTime, DateTimeOffset and TimeSpan values.

Arrays, collections, dictionaries and tuples

EqualConstraint compares arrays, collections, dictionaries and tuples recursively, and the tolerance travels with it. Each pair of corresponding elements is compared using the tolerance. Dictionary keys are always compared exactly.

An array, collection or dictionary has a single element type, so the tolerance must apply to all of its elements or to none of them: a numeric tolerance on a DateTime[] is an error, just as it is on a single DateTime. Tuples are different, because each member has its own type. A tuple member that cannot take the tolerance, such as a string, is compared exactly while the numeric members use the tolerance.

[Test]
public void Tolerance_Collections()
{
    // The tolerance is applied to each pair of corresponding elements
    double[] expected = [1.0, 2.0, 3.0];
    double[] actual = [1.01, 1.99, 3.0];
    Assert.That(actual, Is.EqualTo(expected).Within(0.05));

    // This includes values nested in dictionaries and tuples
    var expectedPrices = new Dictionary<string, decimal> { ["apple"] = 1.00m, ["pear"] = 1.50m };
    var actualPrices = new Dictionary<string, decimal> { ["apple"] = 1.01m, ["pear"] = 1.49m };
    Assert.That(actualPrices, Is.EqualTo(expectedPrices).Within(0.02m));

    // Tuple members have their own types: the double uses the tolerance and the string is compared exactly
    Assert.That((1.001, "a"), Is.EqualTo((1.0, "a")).Within(0.01));

    // An array has one element type, so the tolerance must suit all elements or the assertion errors
    DateTime[] expectedDates = [new(2024, 1, 1)];
    DateTime[] actualDates = [new(2024, 1, 1)];
    Assert.That(() => Assert.That(actualDates, Is.EqualTo(expectedDates).Within(0.01)),
        Throws.TypeOf<NotSupportedException>());
}

Objects compared property by property

When you use UsingPropertiesComparer, a tolerance on the constraint is applied to every property that can accept it and ignored for properties that cannot. From NUnit 4.4 you can also set tolerances in the comparer's configuration, which allows one tolerance for numeric properties and another, a TimeSpan, for time properties.

private sealed record Measurement(string Sensor, double Value, DateTime TakenAt);

[Test]
public void Tolerance_PropertiesComparer()
{
    DateTime now = new(2024, 1, 1, 12, 0, 0, DateTimeKind.Utc);
    var expected = new Measurement("probe-1", 21.5, now);
    var actual = new Measurement("probe-1", 21.52, now);

    // The tolerance applies to every property that can accept it; the others are compared exactly
    Assert.That(actual, Is.EqualTo(expected).Within(0.05).UsingPropertiesComparer());

    // Numeric and time properties can each have their own tolerance
    var later = actual with { TakenAt = now.AddMilliseconds(300) };
    Assert.That(later, Is.EqualTo(expected).UsingPropertiesComparer(
        o => o.Within(0.05).Within(TimeSpan.FromSeconds(1))));
}

See Properties Comparer for the rest of the configuration options.

Default tolerance for floating point values

When a float or double comparison specifies no tolerance, NUnit uses the default floating point tolerance in effect for the test, which is exact equality unless you change it. Apply DefaultFloatingPointToleranceAttribute to a test, a fixture or the assembly to set a different default. An explicit Within always wins over the default.

When a tolerance cannot be applied

Equality for many types is defined by the type itself, through IEquatable<T> or an Equals override, and NUnit has no way to make such a comparison approximate. This covers strings, chars, enums, Stream, DirectoryInfo, your own classes and structs (including records), and third-party value types such as NodaTime's Instant or Duration. The same is true when you supply your own comparer with Using: the comparer decides what is equal and NUnit cannot inject a tolerance into it.

Before NUnit 4.0 a tolerance in these situations was silently ignored, so a test such as

Assert.That(secondInstant, Is.EqualTo(firstInstant).Within(Duration.FromSeconds(2)));

compared the two values exactly and failed for a reason that was hard to spot. From NUnit 4.0 the assertion errors instead with a NotSupportedException whose message starts with Specified Tolerance of type ... not supported for instances of type ....

private readonly record struct Celsius(double Degrees);

[Test]
public void Tolerance_NotSupportedForCustomTypes()
{
    var expected = new Celsius(20.0);
    var actual = new Celsius(20.4);

    // Celsius defines its own equality, so NUnit has no way to apply a numeric tolerance to it.
    // Rather than silently ignoring the tolerance, the assertion errors.
    Assert.That(() => Assert.That(actual, Is.EqualTo(expected).Within(0.5)),
        Throws.TypeOf<NotSupportedException>().With.Message.Contains("Tolerance"));
}

An error rather than a failure tells you that the assertion itself is wrong, not the code under test. The NUnit analyzers report the same situation at compile time as warning NUnit2047 when the types are known to the compiler.

Tolerances for your own types

When NUnit cannot apply a tolerance to a type, you have several options:

  1. Put the tolerance in a comparer and pass it with Using. Do not also use Within; the comparer is responsible for the whole comparison, and combining the two is an error. A reusable IEqualityComparer<T> works for single values and for collections of the type alike.
  2. Compare the underlying value that carries the precision, for example a double property or the result of a subtraction, with a normal Within.
  3. Use the properties comparer so that NUnit compares the object's numeric and time properties one by one, applying the tolerance to each.
  4. Write a custom constraint when the comparison is complex enough to deserve its own name and failure message.
private sealed class CelsiusComparer(double tolerance) : IEqualityComparer<Celsius>
{
    public bool Equals(Celsius x, Celsius y) => Math.Abs(x.Degrees - y.Degrees) <= tolerance;

    // Any two values might be equal, so they must all produce the same hash code
    public int GetHashCode(Celsius obj) => 0;
}

[Test]
public void Tolerance_CustomType()
{
    var expected = new Celsius(20.0);
    var actual = new Celsius(20.4);

    // Option 1: build the tolerance into the comparer passed to Using
    Assert.That(actual, Is.EqualTo(expected).Using<Celsius>((a, b) => Math.Abs(a.Degrees - b.Degrees) <= 0.5));
    Assert.That(actual, Is.EqualTo(expected).Using(new CelsiusComparer(0.5)));

    // A reusable comparer also works for collections of the type
    Celsius[] expectedReadings = [new(20.0), new(21.0)];
    Celsius[] actualReadings = [new(20.4), new(20.7)];
    Assert.That(actualReadings, Is.EqualTo(expectedReadings).Using(new CelsiusComparer(0.5)));

    // Option 2: compare the underlying value
    Assert.That(actual.Degrees, Is.EqualTo(expected.Degrees).Within(0.5));

    // Option 3: let the properties comparer apply the tolerance to the numeric members
    Assert.That(actual, Is.EqualTo(expected).Within(0.5).UsingPropertiesComparer());

    // Combining Using with Within is an error: the comparer alone decides what is equal
    Assert.That(() => Assert.That(actual, Is.EqualTo(expected).Using(new CelsiusComparer(0.5)).Within(0.5)),
        Throws.TypeOf<NotSupportedException>());
}
Note

A tolerant comparer is not transitive: a may be within tolerance of b, and b of c, while a and c are not. That is harmless for Is.EqualTo, which only ever compares corresponding pairs of values, but it matters for the unordered collection constraints discussed next.

Unordered collections

Is.EquivalentTo, Is.SubsetOf, Is.SupersetOf and Is.Unique have no Within modifier. They match items by grouping equal values, which only gives a well-defined answer when equality is transitive. You can pass a tolerant comparer with Using, and NUnit will not stop you, but the result then depends on which items happen to be paired first and may differ from what you expect.

When order does not matter but the values are approximate, either sort both collections and compare them in order with Is.EqualTo(...).Within(...), or assert each expected value individually with an item constraint:

[Test]
public void Tolerance_UnorderedCollections()
{
    double[] expected = [3.0, 1.0, 2.0];
    double[] actual = [1.01, 2.0, 2.99];

    // Is.EquivalentTo has no Within modifier. Sort both sides and compare them in order instead...
    Assert.That(actual.Order(), Is.EqualTo(expected.Order()).Within(0.05));

    // ...or check that each expected value is present exactly once
    foreach (double value in expected)
    {
        Assert.That(actual, Has.Exactly(1).EqualTo(value).Within(0.05));
    }
}

Comparison constraints

Is.GreaterThan, Is.GreaterThanOrEqualTo, Is.LessThan and Is.LessThanOrEqualTo accept a linear or Percent tolerance. The tolerance relaxes the bound in the actual value's favour: Is.GreaterThan(10).Within(1) passes for any value greater than 9, and Is.LessThan(10).Within(1) for any value less than 11. From NUnit 4.1 the bound may be a DateTime or TimeSpan with a TimeSpan tolerance.

[Test]
public void Tolerance_ComparisonConstraints()
{
    // The tolerance relaxes the bound: 9.95 counts as greater than 10.0 when the tolerance is 0.1
    Assert.That(9.95, Is.GreaterThan(10.0).Within(0.1));
    Assert.That(10.05, Is.LessThan(10.0).Within(0.1));
    Assert.That(9.5, Is.GreaterThanOrEqualTo(10).Within(5).Percent);

    // Time values accept a TimeSpan tolerance
    DateTime deadline = new(2024, 1, 1, 12, 0, 0, DateTimeKind.Utc);
    Assert.That(deadline.AddSeconds(2), Is.LessThanOrEqualTo(deadline).Within(TimeSpan.FromSeconds(5)));
}

Rules

  • A tolerance amount must not be negative. From NUnit 4.3 a negative amount throws an ArgumentException.
  • Within may appear only once in a constraint expression.
  • A mode (Percent, Ulps or a time unit) must follow a Within amount, and only one mode may be used.
  • Percent, Ulps and the time units require a numeric amount.
  • A tolerance of zero is an exact comparison. In particular, Within(0) on a floating point comparison overrides a non-zero default floating point tolerance.
[Test]
public void Tolerance_Rules()
{
    // A tolerance must not be negative
    Assert.That(() => Is.EqualTo(5.0).Within(-0.1), Throws.ArgumentException);

    // Within may appear only once in a constraint
    Assert.That(() => Is.EqualTo(5.0).Within(0.1).Within(0.2), Throws.InvalidOperationException);

    // A mode must follow Within, and only one mode may be used
    Assert.That(() => Is.EqualTo(5.0).Percent, Throws.InvalidOperationException);
    Assert.That(() => Is.EqualTo(5.0).Within(1).Percent.Ulps, Throws.InvalidOperationException);
}

See also