trailing 01s and 9s

Hi guys,

I'm having rounding issues in my answers:

freq_in = 1.499;
h = 0:1:H-1;
f_harm = (h+1)*freq_in
f_harm =

     1.499000000000000   2.998000000000000   4.497000000000000   5.996000000000000   7.495000000000001

f_gdf = mod(f_harm,40/8);
f_gdf =

     1.499000000000000   2.998000000000000   4.497000000000000   0.996000000000000   2.495000000000001

a = (2*mod((1:I)',2)-1) * f_gdf; 

   f_gd = repmat((Fs*floor((g+1)/2)/I)',1,H) + a

f_gd =

     1.499000000000000   2.998000000000000   4.497000000000000   0.996000000000000   2.495000000000001
     3.501000000000000   2.002000000000000   0.503000000000000   4.004000000000000   2.504999999999999
     6.499000000000001   7.998000000000000   9.497000000000000   5.996000000000000   7.495000000000001
     8.500999999999999   7.002000000000000   5.503000000000000   9.004000000000000   7.504999999999999
    11.499000000000001  12.998000000000001  14.497000000000000  10.996000000000000  12.495000000000001
    13.500999999999999  12.001999999999999  10.503000000000000  14.004000000000000  12.504999999999999
    16.498999999999999  17.998000000000001  19.497000000000000  15.996000000000000  17.495000000000001
    18.501000000000001  17.001999999999999  15.503000000000000  19.003999999999998  17.504999999999999

The trailing 9s are causing problems in my script later on when I use the floor function. Is there a method for rounding it to whatever it would take to flip the 9s into 0s? Thanks.

Answer 1

[Jan]

The numbers are stored in the binary format inside the computer. For the display on the screen they are converted to the decimal format, which leads to the observed effects of trailing ...0001 and ...999. There is simply no "exact" equivalence between binary and decimal representations.

If your program depends on the value of certain digits in the numerical representation, you have to use integer values smaller than 2^53, because they can be repesented exactly. So think of multiplying your data with the needed factor, which might be 1000 in your case.

See also: FAQ: Why is 0.3 - 0.2 ~= 0.1

[Leor Greenberger]

Thanks Jan and Walter. I actually posted a second question on the subject, which might help me clear up this issue.

Answer 2

[Walter Roberson]

Not really, no. I suggest you read http://matlab.wikia.com/wiki/FAQ#Why_is_0.3_-_0.2_-_0.1_.28or_similar.29_not_equal_to_zero.3F