English 中文(简体)
Is Pex (Test generation) really useful tool?
原标题:

Yes, it is possible to generate tests on boundary values for functions like "Sum" or "Divide". Pex is a good tool here.

But more often we create tests on business behaviour. Let s consider example from classic Beck s tdd book:

[Test]
public void ShouldRoundOnCreation()
{
  Money money = new Money(20.678);
  Assert.AreEqual(20.68,money.Amount);
  Assert.AreEqual(2068,money.Cents);
}

Can this test be generated? No :) 95 % of tests in my projects check business logic, and can not be generated.

Pex (Especially in pair with Moles) can give 100% code coverage, but a high code coverage rate of a test suite does never indicate, that code is well tested - It only gives false confidence that everything is tested. And this is very dangerous.

So, the question is - Is Pex really useful tool?

最佳回答

I think you are mistaken in the way Pex should be used: we highly recommend users to write assertions in their parameterized unit tests.

If you write assertions, Pex will try systematically to invalidate them - that s where the power of Pex comes in: the more you write assertions, the more it tries to find bugs for you.

If you use Pex to get a high code coverage test suite without writing assertions, you only get what you asked for: code coverage and guaranteed runtime exceptions. Pex only tries to cover branches (1 assertion = 1 branch), if there are no branches to cover (no assertion), it won t generate interresting test cases.

In general, writing assertions in parameterized unit tests are harder to write because... well they are more general. Let s start with the rounding behavior: there is certainly a bound on which you allow rounding to occur, this should translate naturally into a parameterized unit test:

[PexMethod]
public void RoundInBounds(double value) 
{
    var money = new Money(value);
    // distance between value and amount should be at most 0.01/2
    Assert.AreEqual(value, money.Amount, 0.005);
}

There are also a number of patterns that can be used to write those. For example, your Money class is probably idempotent: if you feed the value of Amount back in a Money instance, you get Amount back. This translate elegantly into a parameterized unit test:

[PexMethod]
public void RoundIsIdempotent(double value) 
{
     var first = new Money(value).Amount;
     var second = new Money(first).Amount;
     Assert.AreEqual(first, second, 0.0001);
}

Note also, that parameterized unit tests definitely belong in the TDD world. Just write the parameterized unit test first, Pex will find the failing bug, fix the bug, Pex finds the next failing bug, etc...

Does this make Pex a useful tool? You be the judge.

问题回答

There s a few things useful in Pex.

  1. Code coverage. Let s put Pex aside for a second. In general, 100% code coverage doesn t necessarily mean you have good code coverage. It just means ever path is executed, but program has data-flow, and testing that code different, additional inputs, gives you best "test coverage", if not code coverage. Here s, I m just reiterating your point. 100% code coverage is not necessarily good, but you can say for sure that 25% code coverage is bad, so that s how code coverage is useful. When you have low code coverage, you know for sure you have low test coverage.

    When you use Pex to get 100% code coverage, it s not really helping you get better test coverage per se, but what it does do is give ever piece of the production code some test, which can be used for debugger. In fact, a Pex presentation as a conference shows the use of Pex for this very purpose. The programmer said, "Gee, look at this method in NHibernate. I d like to step through it in a debugger to see what it does, but how do I even invoke that method through the normal "business-entry point" into the library? Without knowing anything about the library, you can t. So he ran Pex, and was able to step through the code with all sorts of parameters. Interesting, Yes. Useful, maybe so, maybe no.

  2. In addition to automatically created tests, Pex is also useful for parameterizing your tests. It s much better to create unit tests that are data driven. Why write the same code over and over, with different parameters. Write it once, and feed the parameters from a data source.

  3. Pex is also useful as a simple stubbing framework. It s probably the easiest way to create mock objects, using the new lambda expression, which is much easier to understand than complex frameworks like RhinoMocks, etc. With Moles, you can also stub not just interface, but concrete non-virtual methods in classes.

  4. I would also be careful with the "testing at the business logic" layer, too much. You could easy get to doing Behavioral Driven Development, which is not for unit testing. There, you are testing against a spec. If that s all you did, how would you test, say, custom data structures, etc., that have no business value, but are internal libraries used throughout the application.





相关问题
Manually implementing high performance algorithms in .NET

As a learning experience I recently tried implementing Quicksort with 3 way partitioning in C#. Apart from needing to add an extra range check on the left/right variables before the recursive call, ...

Anyone feel like passing it forward?

I m the only developer in my company, and am getting along well as an autodidact, but I know I m missing out on the education one gets from working with and having code reviewed by more senior devs. ...

How do I compare two decimals to 10 decimal places?

I m using decimal type (.net), and I want to see if two numbers are equal. But I only want to be accurate to 10 decimal places. For example take these three numbers. I want them all to be equal. 0....

Exception practices when creating a SynchronizationContext?

I m creating an STA version of the SynchronizationContext for use in Windows Workflow 4.0. I m wondering what to do about exceptions when Post-ing callbacks. The SynchronizationContext can be used ...

Show running instance in single instance application

I am building an application with C#. I managed to turn this into a single instance application by checking if the same process is already running. Process[] pname = Process.GetProcessesByName("...

How to combine DataTrigger and EventTrigger?

NOTE I have asked the related question (with an accepted answer): How to combine DataTrigger and Trigger? I think I need to combine an EventTrigger and a DataTrigger to achieve what I m after: when ...

热门标签