Question

I m using decimal type (.net), and I want to see if two numbers are equal. But I only want to be accurate to 10 decimal places. For example take these three numbers. I want them all to be equal.

0.1123456789
0.11234567891
0.11234567899

The 10 decimal limit is coming from my database, so I have to assume that the first number was already rounded, and therefore I can t simply round the others because the last one will round up.

I really just want to truncate the to 10 decimal places, but can t see how to do this either.

Answer 1

What about multiplying by 10^10 and dropping the fractional part?

decimal x2 = Math.Truncate(x * 10000000000);
decimal y2 = Math.Truncate(y * 10000000000);
Assert.Equals(x2, y2);

EDIT: Changed to Math.Truncate by Aaron s suggestion. Thanks.

Answer 2

The same way you d compare floating point numbers. Here s some pseudocode, because I don t know the .NET call for absolute value, but it will essentially look like this (modify the constant for the precision needed):

if( Math.Abs( value1 - value2 ) < 0.0000000001 )
{
  // blah blah
}

Answer 3

Multiply by 10000000000, convert to an int, then divide by the same number.

That way you truncate the the excess digits.

You may want to add .5 * 10^-11 in order to round properly before truncating.

Answer 4

Multiply by 10^10 Rather than converting to an integer (which is 32 bit) it might be worth using an int64. An int 32 has a limit of approximatly 2 billion which when multipled out gives a range between +2.1 and -2.1 on your decimal value, an int64 has a much larger range that, when multiplied out gives you a range of approximatly +922,000,000 to -922,000,000 on the decimal value

http://msdn.microsoft.com/en-us/library/system.int32.aspx

http://msdn.microsoft.com/en-us/library/system.int64.aspx

Answer 5

Maybe this works for you:

If Decimal.Round(yourDec1, 10, MidpointRounding.AwayFromZero) = Decimal.Round(yourDec2, 10, MidpointRounding.AwayFromZero) Then

Bobby

友情链接