Bug in rounding functions? Who can explain? :)

1 visualización (últimos 30 días)
Pavel
Pavel el 11 de Nov. de 2015
Editada: Stephen23 el 11 de Nov. de 2015
>> format long
>> 4.1*808e6
ans =
3.312800000000000e+09
>> fix(4.1*808e6)
ans =
3.312799999000000e+09
>> fix(4*808e6 + .1*808e6)
ans =
3.312800000000000e+09

Respuesta aceptada

Walter Roberson
Walter Roberson el 11 de Nov. de 2015
>> 4.1*808e6 - 3312800000
ans =
-4.76837158203125e-07
That is, due to floating point roundoff, 4.1*808e6 falls just short of an integer. fix() is the same as floor() for positive numbers so fix() removes the fraction after 3312799999 that is not quite 1.0, exactly like how floor(1.99999999) is 1.0 not 2.0

Más respuestas (0)

Categorías

Más información sobre Logical en Help Center y File Exchange.

Etiquetas

Productos

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by