By markzzz


2019-02-05 09:41:03 8 Comments

Due to the floating point "approx" nature, its possible that two different sets of values return the same value.

Example:

#include <iostream>

int main() {
    std::cout.precision(100);

    double a = 0.5;
    double b = 0.5;
    double c = 0.49999999999999994;

    std::cout << a + b << std::endl; // output "exact" 1.0
    std::cout << a + c << std::endl; // output "exact" 1.0
}

But is it also possible with subtraction? I mean: is there two sets of different values (keeping one value of them) that return 0.0?

i.e. a - b = 0.0 and a - c = 0.0, given some sets of a,b and a,c with b != c??

4 comments

@plugwash 2019-02-06 04:28:24

Unfortunately the answer is dependent on your implementation and the way it is configured. C and C++ don't demand any specific floating point representation or behavior. Most implementations use the IEEE 754 representations, but they don't always precisely implement IEEE 754 arithmetic behaviour.

To understand the answer to this question we must first understand how floating point numbers work.

A naive floating point representation would have an exponent, a sign and a mantissa. It's value would be

(-1)s2(e – e0)(m/2M)

Where:

  • s is the sign bit, with a value of 0 or 1.
  • e is the exponent field
  • e0 is the exponent bias. It essentially sets the overall range of the floating point number.
  • M is the number of mantissa bits.
  • m is the mantissa with a value between 0 and 2M-1

This is similar in concept to the scientific notation you were taught in school.

However this format has many different representations of the same number, nearly a whole bit's worth of encoding space is wasted. To fix this we can add an "implicit 1" to the mantissa.

(-1)s2(e – e0)(1+(m/2M))

This format has exactly one representation of each number. However there is a problem with it, it can't represent zero or numbers close to zero.

To fix this IEEE floating point reserves a couple of exponent values for special cases. An exponent value of zero is reserved for representing small numbers known as subnormals. The highest possible exponent value is reserved for NaNs and infinities (which I will ignore in this post since they aren't relevant here). So the definition now becomes.

(-1)s2(1 – e0)(m/2M) when e = 0
(-1)s2(e – e0)(1+(m/2M)) when e >0 and e < 2E-1

With this representation smaller numbers always have a step size that is less than or equal to that for larger ones. So provided the result of the subtraction is smaller in magnitude than both operands it can be represented exactly. In particular results close to but not exactly zero can be represented exactly.

This does not apply if the result is larger in magnitude than one or both of the operands, for example subtracting a small value from a large value or subtracting two values of opposite signs. In those cases the result may be imprecise but it clearly can't be zero.

Unfortunately FPU designers cut corners. Rather than including the logic to handle subnormal numbers quickly and correctly they either did not support (non-zero) subnormals at all or provided slow support for subnormals and then gave the user the option to turn it on and off. If support for proper subnormal calculations is not present or is disabled and the number is too small to represent in normalized form then it will be "flushed to zero".

So in the real world under some systems and configurations subtracting two different very-small floating point numbers can result in a zero answer.

@markzzz 2019-02-06 09:15:31

So basically this problem interess only subnormal numbers? i.e. if I don't work with subnormals I'll never have two different floating point numbers that can result in a zero?

@plugwash 2019-02-06 15:01:17

The problem interests numbers where the result of the subtraction is subnormal. Subtracting two small normal numbers can produce a subnormal result which some implementations will flush to zero.

@markzzz 2019-02-06 17:48:05

So disabling DAZ and FTZ will show two different results?

@plugwash 2019-02-06 18:13:22

The difference between "FTZ" and "DAZ" modes on Intel processors seems to be where the flushing happens. The former operates on the outputs of an operation while the latter operates on the inputs. So if i'm reading the documentation right with FTZ the subtraction could produce a false-zero while with DAZ the subtraction would produce a correct denormal result, but the comparision would then treat that denormal result as zero.

@Eric Postpischil 2019-02-05 10:40:59

The IEEE-754 standard was deliberately designed so that subtracting two values produces zero if and only if the two values are equal, except that subtracting an infinity from itself produces NaN and/or an exception.

Unfortunately, C++ does not require conformance to IEEE-754, and many C++ implementations use some features of IEEE-754 but do not fully conform.

A not uncommon behavior is to “flush” subnormal results to zero. This is part of a hardware design to avoid the burden of handling subnormal results correctly. If this behavior is in effect, the subtraction of two very small but different numbers can yield zero. (The numbers would have to be near the bottom of the normal range, having some significand bits in the subnormal range.)

Sometimes systems with this behavior may offer a way of disabling it.

Another behavior to beware of is that C++ does not require floating-point operations to be carried out precisely as written. It allows “excess precision” to be used in intermediate operations and “contractions” of some expressions. For example, a*b - c*d may be computed by using one operation that multiplies a and b and then another that multiplies c and d and subtracts the result from the previously computed a*b. This latter operation acts as if c*d were computed with infinite precision rather than rounded to the nominal floating-point format. In this case, a*b - c*d may produce a non-zero result even though a*b == c*d evaluates to true.

Some C++ implementations offer ways to disable or limit such behavior.

@eerorika 2019-02-05 10:44:52

Gradual underflow feature of IEEE floating point standard prevents this. Gradual underflow is achieved by subnormal (denormal) numbers, which are spaced evenly (as opposed to logarithmically, like normal floating point) and located between the smallest negative and positive normal numbers with zeroes in the middle. As they are evenly spaced, the addition of two subnormal numbers of differing signedness (i.e. subtraction towards zero) is exact and therefore won't reproduce what you ask. The smallest subnormal is (much) less than the smallest distance between normal numbers, and therefore any subtraction between unequal normal numbers is going to be closer to a subnormal than zero.

If you disable IEEE conformance using a special denormals-are-zero (DAZ) or flush-to-zero (FTZ) mode of the CPU, then indeed you could subtract two small, close numbers which would otherwise result in a subnormal number, which would be treated as zero due to the mode of the CPU. A working example (Linux):

_MM_SET_FLUSH_ZERO_MODE(_MM_FLUSH_ZERO_ON);    // system specific
double d = std::numeric_limits<double>::min(); // smallest normal
double n = std::nextafter(d, 10.0);     // second smallest normal
double z = d - n;       // a negative subnormal (flushed to zero)
std::cout << (z == 0) << '\n' << (d == n);

This should print

1
0

First 1 indicates that result of subtraction is exactly zero, while the second 0 indicates that the operands are not equal.

@plugwash 2019-02-06 03:41:13

"The smallest subnormal is (much) less than the smallest distance between normal numbers" no, the smallest denomal is exactly the same as the distance between normal numbers with the lowest allowed exponent.

@Joseph Ireland 2019-02-05 10:40:48

Excluding funny numbers like NAN, I don't think it's possible.

Let's say a and b are normal finite IEEE 754 floats, and |a - b| is less than or equal to both |a| and |b| (otherwise it's clearly not zero).

That means the exponent is <= both a's and b's, and so the absolute precision is at least as high, which makes the subtraction exactly representable. That means that if a - b == 0, then it is exactly zero, so a == b.

Related Questions

Sponsored Content

12 Answered Questions

[SOLVED] What is the difference between float and double?

22 Answered Questions

[SOLVED] Limiting floats to two decimal points

34 Answered Questions

[SOLVED] How to deal with floating point number precision in JavaScript?

28 Answered Questions

[SOLVED] Is floating point math broken?

9 Answered Questions

[SOLVED] How dangerous is it to compare floating point values?

16 Answered Questions

[SOLVED] Difference between decimal, float and double in .NET?

13 Answered Questions

1 Answered Questions

0 Answered Questions

Why the float values are different from double values when set precision?

  • 2017-06-23 09:03:53
  • Hatem Safwat
  • 61 View
  • 0 Score
  • 0 Answer
  • Tags:   c++ c++11

3 Answered Questions

Sponsored Content