math - Strange subtraction in c# -
i doing c# program.. must simple mathematical operation.
72057594037927936.0 - 255.0 = ..... both numbers double... obtain
72057594037927680.0 instead of
72057594037927681.0 can explain me please how possible?
thanx
the exact result of subtraction,
72057594037927681 = 0xffffffffffff01 needs 56 bits of precision, double has 53, hence result rounded nearest representable number.
Comments
Post a Comment