significant figures from h5read() output

I have an issue where the output of h5read() is being rounded, and I wish to increase the number of significant figures that are being saved. I need it to be more precise, because these values are used as indices for more data within the file, which is then used to group the data.

The issue is that, because of the rounding, I am not able to use these values to access the data that I need. For example, a value in the h5 file is "315.059998", which is then being stored as "315.060", which then cannot access the bin for "/bins 315.059998".

I have checked the help centre page for the h5read() function, and cannot see anything that solves my issue (though I may have missed it), and I have also made sure that the values really are stored in the h5 file at the correct precision.

If anybody knows how to increase the precision that these numbers are saved as, I would be very grateful.

A reduced example is shown below:

vals = h5read('filename.h5','/values');
for i = 1:length(vals)
    output(i,:) = h5read('filename.h5',strcat('/bins',num2str(vals(i))));
end

Answer 1

[Walter Roberson]

str2num(vals(i)) would be a number -- an error if vals(i) is not character vector or string scalar. If it is a character vector or string scalar but cannot be converted to number then double precision [] will be returned.

When you strcat() between a character vector and the str2num() result, then the result would be as if char() had been called on the numeric value -- for example

strcat('ABC', str2num('49'))

ans = 'ABC1'

because str2num('49') is going to return 49, and char(49) is '1'

If your actual code uses num2str() instead of str2num(), then notice in the documentation that num2str() with a single parameter choses the output precision based upon the magnitude of the inputs.

[Walter Roberson]

For example, a value in the h5 file is "315.059998", which is then being stored as "315.060", which then cannot access the bin for "/bins 315.059998".

Well, you are wrong about the file in the file being 315.059998 . HDF5 stores floating point numbers as either IEEE 754 Single Precision or else IEEE 754 Double Precision.

format long g

val_single = single(315.059998)

val_single = single
          315.06

fprintf('%.999g\n', val_single)

315.05999755859375

val_double = double(315.059998)

val_double = 
                315.059998

fprintf('%.999g\n', val_double)

315.05999800000000732325133867561817169189453125

Notice that neither of the fprintf() outputs end in 8 .

If you have a '/bins 315.059998' element in your file, then you should be using h5info to read the attributes of the file, locate the /bins entry, convert the embedded number to floating point, and use ismembertol to compare to the targer value.

[Max]

Hi Walter,

Thanks for the quick responses. Yes, the actual code uses num2str(). I have corrected that in my question.

I think, from your response, I am getting a bit more context to my issue and filling some holes in my knowledge. The value is output from a physical device, where some points are saved with different significant figures than others. One point has a precise saved value of: 131.30599999999998 (more decimal places than the example I gave, but the issue still stands). When I access this initial value in MATLAB this is then stored as 131.3060, which then cannot be used to access the binning information. There are a few more points from the device where this would be a problem, and other points where the values from the device have been saved to fewer decimal points.

The bit that is confusing me the most is that I have similar code from a colleague that is written in Python, and this issue doesn't happen. The values are read and stored as they have been saved in the original file.

I will try your suggestions, to see if I can get a working solution.

[Walter Roberson]

One point has a precise saved value of: 131.30599999999998

format long g

val_single = single(131.30599999999998)

val_single = single
         131.306

fprintf('%.999g\n', val_single)

131.305999755859375

val_double = double(131.30599999999998)

val_double = 
                   131.306

fprintf('%.999g\n', val_double)

131.30599999999998317434801720082759857177734375

You can see from this that IEEE 754 Single Precision and IEEE 754 Double Precision both do not have any way of storing exactly 131.30599999999998 .

There are two possibilities here:

1. The value 131.30599999999998 might have been stored as text. If so the num2str(vals(i)) would have required that vals was a string() array so that vals(i) could refer to a group of characters instead of a single character, and the result of the num2str() would in this case be the string scalar converted to a character vector with exactly the same characters; OR
2. You are mistaken about the exact value of the numeric value, and you are confusing how MATLAB displays with what it stores

format long g
val_double

val_double = 
                   131.306

num2str(val_double)

ans = '131.306'

fprintf('%.999g\n', val_double)

131.30599999999998317434801720082759857177734375

num2str(val_double, 15)

ans = '131.306'

num2str(val_double, 16)

ans = '131.306'

num2str(val_double, 17)

ans = '131.30599999999998'

format short
val_double

val_double = 131.3060

num2str(val_double)

ans = '131.306'

fprintf('%.999g\n', val_double)

131.30599999999998317434801720082759857177734375

num2str(val_double, 15)

ans = '131.306'

num2str(val_double, 16)

ans = '131.306'

num2str(val_double, 17)

ans = '131.30599999999998'

The output of num2str() does not depend upon the current setting of format -- I just show repeat the num2str() commands here to explicitly show that the output is independent of format setting. Likewise the output of fprintf() does not depend upon the format setting

You can see that the precise value stored is 131.30599999999998317434801720082759857177734375 not 131.30599999999998 . And you can further see from the fact that you include the trailing 0 on 131.3060 that you have format short in effect, not format long g . format short generally displays 4 digits after the decimal point for non-integers with absolute magnitude between 1e-3 and 1e+4 that do not happen to be integers; for other values it switches to scientific notation (and when displaying a non-scalar, the display mode depends upon the range of the value with the largest absolute magnitude.)

format short does not display "precisely" what is stored in the numbers: it displays approximately what is stored, to a number of decimal places that is often useful to get a general idea of what is going on with the values. It is intended to display an "often useful" amount of information about the numbers, not the precise values.

[Stephen23]

"One point has a precise saved value of: 131.30599999999998 (more decimal places than the example I gave, but the issue still stands). When I access this initial value in MATLAB this is then stored as 131.3060, which then cannot be used to access the binning information."

It sounds like you are confusing how numeric data are displayed with what data are stored in memory.

val = 131.30599999999998

val = 131.3060

fprintf('%.999g\n',val)

131.30599999999998317434801720082759857177734375

There is no such double binary floating point "precise value" exactly equal to 131.30599999999998, so MATLAB uses the nearest double value (which most likely is also what the physical device is doing, before it exports that value with e.g. max 17** sig figs). Do NOT confuse the representation of some numeric value with what is stored in memory! Thinking in terms of "precise values" will not when working with binary floating point numbers: in contrast understanding that they have a fixed, finite precision will help you to work with them.

** this being sufficient to exactly specify the nearest double value without writing all of its decimal digits. This is a big clue that your "physical device" also uses double floating point values.

[Walter Roberson]

fprintf('%.999g\n', 131.306)

131.306000000000011596057447604835033416748046875

fprintf('%.999g\n', 131.306*(1-eps))

131.30599999999998317434801720082759857177734375

so the 131.30599999999998 is the next closest representable number less than 131.306 .

It would not astonish me if the device "intended" the value to be exactly 131.306 but there was some arithmetic issues in the conversion between internal sensor values and double precision. For example it might be internally using an unsigned 16 bit sensor reading of 65653 with a resolution of 1/500 units, so 65653/500 to get the floating point representation. Although in full double precision that should give 131.306000000000011596057447604835033416748046875 it would not be astonishing if a device such as an Arduno did not get a division bit-for-bit exact to the "recommended" IEEE value.

[Max]

Hi Walter,

Thanks for all the information. You are correct that the device was supposed to record a value of 131.306. Using what you said about the values either being single precision or double precision, I was able to create kind of a "dirty" solution with a try, catch statement to account for the values which haven't been recorded as expected. The code now works correctly and is binning data as it should.

One thing that is still confusing me, though, is why this doesn't appear to be an issue in Python code that I tested. The values saved from the Python code were then able to be used to access the bins with no issues from the precision.

Thanks again for all the help!

[Walter Roberson]

@Max

In MATLAB, you used num2str(), which has a default conversion precision that does not happen to match your needs. You can explicitly use a different conversion precision such as num2str(val_double, 17)

The equivalent conversion that you are doing in Python uses a different default conversion precision than MATLAB does.