Question

I m writing some NUnit tests for database operations. Obviously, if Add() fails, then Get() will fail as well. However, it looks deceiving when both Add() and Get() fail because it looks like there s two problems instead of just one.

Is there a way to specify an order for tests to run in, in that if the first test fails, the following tests are ignored?

In the same line, is there a way to order the unit test classes themselves? For example, I would like to run my tests for basic database operations first before the tests for round-tripping data from the UI.

Note: This is a little different than having tests depend on each other, it s more like ensuring that something works first before running a bunch of tests. It s a waste of time to, for example, run a bunch of database operations if you can t get a connection to the database in the first place.

Edit: It seems that some people are missing the point. I m not doing this:

[Test]
public void AddTest()
{
    db.Add(someData);
}

[Test]
public void GetTest()
{
    db.Get(someData);
    Assert.That(data was retrieved successfully);
}

Rather, I m doing this:

[Test]
public void AddTest()
{
    db.Add(someData);
}

[Test]
public void GetTest()
{
    // need some way here to ensure that db.Add() can actually be performed successfully
    db.Add(someData);
    db.Get(somedata);
    Assert.That(data was retrieved successfully);
}

In other words, I want to ensure that the data can be added in the first place before I can test whether it can be retrieved. People are assuming I m using data from the first test to pass the second test when this is not the case. I m trying to ensure that one operation is possible before attempting another that depends on it.

As I said already, you need to ensure you can get a connection to the database before running database operations. Or that you can open a file before performing file operations. Or connect to a server before testing API calls. Or...you get the point.

Answer 1

NUnit supports an "Assume.That" syntax for validating setup. This is documented as part of the Theory (thanks clairestreb). In the NUnit.Framework namespace is a class Assume. To quote the documentation:

/// Provides static methods to express the assumptions
/// that must be met for a test to give a meaningful
/// result. If an assumption is not met, the test
/// should produce an inconclusive result.

So in context:

public void TestGet() {
    MyList sut = new MyList()
    Object expecting = new Object();
    sut.Put(expecting);
    Assume.That(sut.size(), Is(1));
    Assert.That(sut.Get(), Is(expecting));
}

Answer 2

Tests should never depend on each other. You just found out why. Tests that depend on each other are fragile by definition. If you need the data in the DB for the test for Get(), put it there in the setup step.

Answer 3

I think the problem is that you re using NUnit to run something other than the sort of Unit Tests that NUnit was made to run.

Essentially, you want AddTest to run before GetTest, and you want NUnit to stop executing tests if AddTest fails.

The problem is that that s antithetical to unit testing - tests are supposed to be completely independent and run in any order.

The standard concept of Unit Testing is that if you have a test around the Add functionality, then you can use the Add functionality in the Get test and not worry about if Add works within the Get test. You know Add works - you have a test for it.

The FIRST principle (http://agileinaflash.blogspot.com/2009/02/first.html) describes how Unit tests should behave. The test you want to write violates both I (Isolated) and R (Repeatable).

If you re concerned about the database connection dropping between your two tests, I would recommend that rather than connect to a real database during the test, your code should use some sort of a data interface, and for the test, you should be using a mock interface. If the point of the test is to exercise the database connection, then you may simply be using the wrong tool for the job - that s not really a Unit test.

Answer 4

I don t think that s possible out-of-box.

Anyway, your test class design as you described will make the test code very fragile.

Answer 5

MbUnit seems to have a DependsOnAttribute that would allow you to do what you want.

If the other test fixture or test method fails then this test will not run. Moreover, the dependency forces this test to run after those it depends upon.

Don t know anything about NUnit though.

Answer 6

You can t assume any order of test fixture execution, so any prerequisites have to be checked for within your test classes.

Segregate your Add test into one test-class e.g. AddTests, and put the Get test(s) into another test-class, e.g. class GetTests.

In the [TestFixtureSetUp] method of the GetTests class, check that you have working database access (e.g. that Add s work), and if not, Assert.Ignore or Inconclusive, as you deem appropriate.

This will abort the GetTests test fixture when its prerequisites aren t met, and skip trying to run any of the unit tests it contains. (I think! I m an nUnit newbie.)

Answer 7

Create a global variable and return in the test for Get unless Add set it to true (do this in the last line of Add):

public boolean addFailed = false;
public void testAdd () {
    try {
        ... old test code ...
    } catch (Throwable t) { // Catch all errors
        addFailed = true;
        throw t; // Don t forget to rethrow
    }
}
public void testGet () {
    if (addFailed) return;
    ... old test code ...
}

友情链接