HDF5 V1.10

Can Matlab read/write HDF5 v1.10 files? If not, are there plans to add that support? I'm particularly interested in SWMR (Single Write Multiple Read) functionality, but I have not found any mention of that in the Matlab docs.

Answer 1

[Talmo Pereira]

MATLAB as of R2018a cannot read v1.10 HDF5 files and therefore support SWMR. The HDF5 library version used in either R2017b or R2018a:

>> [ver_major,ver_minor,ver_patch] = H5.get_libversion
ver_major =
   1
ver_minor =
   8
ver_patch =
  12

From the HDF5 SWMR Users Guide:

"Word of Caution: Please note that HDF5 files created with SWMR access cannot not be read by the tools based on the HDF5 1.8 libraries due to the file format changes in the HDF5 1.10 version of the library. [...] One can use the h5repack or h5format_convert tools to convert an HDF5 file created by the HDF5 Library version 1.10.0 to be accessible by the HDF5 Library version 1.8. The h5repack tool rewrites the whole file; h5format_convert modifies the file “in place” by rewriting only metadata information according to the 1.8 format while leaving raw data intact."

Here's what happens if you try to query a SWMR-enabled file:

>> h5disp('swmr_test.h5')
Error using h5infoc
The filename specified was either not found on the MATLAB path or it contains unsupported characters.
Error in h5info (line 102)
    hinfo = h5infoc(filename,location, useUtf8);
Error in h5disp>display_hdf5 (line 112)
    hinfo = h5info(options.Filename,options.Location);
Error in h5disp (line 90)
display_hdf5(options);

Using the commandline tool provided with the official library:

>> !h5ls swmr_test.h5
x                        Dataset {64, 64, 1, 1000/Inf}

It would be terrific if MathWorks considered reaching feature parity with the HDF5 library or leading wrappers like h5py within at least 1-2 years of breaking changes :)...

[Jasmin Jestel]

Dear Talmo,

I encountered the same problem as Norton when i was trying to read a recently created hdf5 file (.h5) in matlab. When searching the internet for a way to solve this problem, i found your reply. I downlaoded the hdf5 software and tried to call the mentioned h5format_convert function from the windows command line. Unfortunately, it was unsuccessful, probably because i specified the option and/or parameter wrong. Could you possibly share an example of how you call the function? Or would you advise me to download Anaconda and use the Python work package?

Best regards, JJ

[Talmo Pereira]

Yeah the Python solution is what I've been doing. I use something like this SO answer using h5py.Group.copy and create the output file with:

h5py.File("output.h5", "w", libver="earliest")

Reference: http://docs.h5py.org/en/latest/high/file.html#version-bounding

[Jasmin Jestel]

Thank you for your help, it is much appreciated!

[Paul Ryan]

I still can't get this to work in python. Would you be able to share exactly what you did?

Answer 2

[Alexander Neumann]

MATLAB 2021b (Prerelease)

>> [majnum,minnum,relnum] = H5.get_libversion()
majnum =
     1
minnum =
    10
relnum =
     7

so it has finally been updated

Answer 3

[Jarom Nelson]

h5repack and h5format_convert are available with the HDF5 1.10.x software to update a file to be compatible with HDF5 1.8.x.

From the SWMR user guide:https://support.hdfgroup.org/HDF5/docNewFeatures/SWMR/HDF5_SWMR_Users_Guide.pdf

One can use the h5repack or h5format_convert tools to convert an HDF5 file created by the HDF5 Library version 1.10.0 to be accessible by the HDF5 Library version 1.8. The h5repack tool rewrites the whole file; h5format_convert modifies the file “in place” by rewriting only metadata information according to the 1.8 format while leaving raw data intact.