Home › Forums › kdb+ › Help Understanding Precision › Re: Help Understanding Precision
-
That number cannot be represented exactly as a floating point number i.e. it has a infinitely repeating representation (if you look at the attached pic you’ll see the binary representation is infinite). Therefore, by adjusting display precision, you’ll see different outputs. Some numbers can be represented as a floating point number e.g.
q)P 7 q)1.125 1.126 1.125 1.126 q)P 0 q)1.125 1.126 1.125 1.1259999999999999
https://www.exploringbinary.com/why-0-point-1-does-not-exist-in-floating-point/