Exporting to .mat file in C# (result is corrupt)

Hi!

For experiments I am going to make I need a .mat file export function for my C# program, so I went ahead and wrote one based on this document (Level 5): http://www.mathworks.com/help/pdf_doc/matlab/matfile_format.pdf

What I want is:

• Export double precision vectors and matrices.
• Multiple vectors/matrices in one file.

Problem: When I try to import my .mat file i get the error:

 Caught "std::exception" Exception message is:
 Unknown exception

Could someone please help me with finding the problem? I have look through the .mat "standard" multiple times and I cannot find my fault.

Best Regards Emil Fresk

My internal file structure is like this:

 All data MUST be 8 byte aligned!
       ________________________________________________
      |        4 bytes        |        4 bytes         |  Data in the following tables
      |------------------------------------------------|

       ________________________________________________
      |        Descriptive text (116 bytes)            |
      |                        ------------------------|
      |                       |   Subsys data offset   |
      |------------------------------------------------|
      |   Subsys data offset  |   version  | endianess |
      |------------------------------------------------|

       ________________________________________________
 Tag: |   miMATRIX (4 bytes)  | Size of file excluding |
      |                       |    Tag (4 bytes)       |
      |------------------------------------------------| -- ---------------
      |                  data header                   |   |               |
      |------------------------------------------------|   | Sub Element   |
      |                     data                       |   |               |
      |------------------------------------------------| --                |
      |                  data header                   |                   | Size in Tag
      |------------------------------------------------|                   |
      |                     data                       |                   |
      |------------------------------------------------|                   |
      |                       :                        |                   |
      |                       :                        |                   |
      |------------------------------------------------|-------------------

 Sub Element (each part of the sub element must be padded to 8 bytes):
       ________________________________________________
      |        miUINT32       | Size of Array Flags (8)| --
      |------------------------------------------------|   | Sub Element header
      |   Array Flags/Class   |        reserved        |   | 
      |------------------------------------------------| --
      |         miINT32       | Number of dimensions   |   |
      |                       |     n * miINT32        |   | Array size information
      |------------------------------------------------|   |
      |         Dim1          |        Dim2 ....       |   |
      |------------------------------------------------| --
      |         miINT8        |   Array Name length    |   |
      |------------------------------------------------|   | Array Name
      |          Array Name (max 127 bytes)            |   |
      |------------------------------------------------| --
      | Data type (miXXXXX)   |  Data length in bytes  |   |
      |------------------------------------------------|   | Real data (saved column wise)
      |                   Real data                    |   | Called pr in the defintion
      |------------------------------------------------| --
      | Data type (miXXXXX)   |  Data length in bytes  |   | Imaginary data (saved column wise,
      |------------------------------------------------|   | optional - dependent of the Array Flags)
      |                 Imaginary data                 |   | Called pi in the defintion
      |------------------------------------------------| --

Valued example:

• Starting text: "My very long descriptive text\By Emil Fresk (my-mail)"
• endianess = "MI"
• Array Flags = 32 (global)
• Array Class = mxDOUBLE_CLASS
• Dimension 5 x 1
• Data type: miDOUBLE
• Array name: "Test"
• Data is 5 doubles: 0.0, 1.0, 2.0, 3.0, 4.0

Hex dump of the file (232 bytes):

        00 01 02 03 04 05 06 07
 000000 4D 79 20 76 65 72 79 20  My very 
 000008 6C 6F 6E 67 20 64 65 73  long des
 000010 63 72 69 70 74 69 76 65  criptive
 000018 20 74 65 78 74 0A 0A 42   text..B
 000020 79 20 45 6D 69 6C 20 46  y Emil F
 000028 72 65 73 6B 20 28 65 6D  resk (em
 000030 69 6C 2E 66 72 65 73 6B  il.fresk
 000038 40 67 6D 61 69 6C 2E 63  @gmail.c
 000040 6F 6D 29 00 00 00 00 00  om).....
 000048 00 00 00 00 00 00 00 00  ........
 000050 00 00 00 00 00 00 00 00  ........
 000058 00 00 00 00 00 00 00 00  ........
 000060 00 00 00 00 00 00 00 00  ........
 000068 00 00 00 00 00 00 00 00  ........
 000070 00 00 00 00 00 00 00 00  ........
 000078 00 00 00 00 00 01 4D 49  ......MI <- Version and endian
 000080 0E 00 00 00 60 00 00 00  ....`... <- miMATRIX and size (96 bytes)
 000088 05 00 00 00 08 00 00 00  ........ <- miUINT32 and array flags size (8 bytes)
 000090 00 00 20 06 00 00 00 00  .. ..... <- Flags: Global, mxDOUBLE_CLASS and 4 bytes reserved
 000098 06 00 00 00 08 00 00 00  ........ <- Dimension miINT32, size 8 bytes
 0000A0 05 00 00 00 01 00 00 00  ........ <- 5 time 1 in dimension
 0000A8 02 00 00 00 04 00 00 00  ........ <- miINT8 and length on name (4)
 0000B0 54 65 73 74 00 00 00 00  Test.... <- Vector name "Test" and 4 bytes padding
 0000B8 09 00 00 00 28 00 00 00  ....(... <- miDOUBLE and length of data (40 bytes = 5 doubles)
 0000C0 00 00 00 00 00 00 00 00  ........ <- 0.0
 0000C8 00 00 00 00 00 00 F0 3F  .......? <- 1.0
 0000D0 00 00 00 00 00 00 00 40  .......@ <- 2.0
 0000D8 00 00 00 00 00 00 08 40  .......@ <- 3.0
 0000E0 00 00 00 00 00 00 10 40  .......@ <- 4.0 and EOF

Answer 1

[José-Luis]

Please, save yourself a headache and use the Matlab C++/Fortran API. You would need to be comfortable mixing that with C#, of course, but it would sure save you a world of hurt and might spare you the need to reinvent the wheel.

http://www.mathworks.com/help/matlab/calling-matlab-engine-from-c-c-and-fortran-programs.html

Additionaly, it is very hard to help you since we cannot see what code generated your .mat file.

[Emil Fresk]

Hi,

I was looking to this, but I have zero experience when it comes embedding other code. When it comes to the code, full project here in Visual Studio 2013: https://www.dropbox.com/sh/2mx03e7jstoz0d7/AABd6dDgXDaQQQQ5Vk8G_FjAa?dl=0

Best Regards Emil

[José-Luis]

Ledsen Emil, but I'm not a Windows/Visual studio user. If you don't want to embed code, there are workarounds. You could always call Matlab directly from C#. That should have been my first suggestion, but it's been years since I left Windows.

[Emil Fresk]

I looked into this however this does not allow to be used completely stand alone. Thanks for the tip though!

[José-Luis]

No worries. Why do you want to save as .mat? Speed? If so a binary file would work as well. Readability? Then a text file might be enough. Matlab can easily import text files.

[Guillaume]

For what it's worth, I downloaded your source code, compiled it (VS2013), ran it, and successfully loaded the generated test file in matlab 2014b (Win64) without it generating any exception.

The only issue is that the data in the array is complex with the same real and imaginary part which I don't think you intended:

>> load('test.mat')
>> Test

Test =

     0.0000 + 0.0000i
     1.0000 + 1.0000i
     2.0000 + 2.0000i
     3.0000 + 3.0000i
     4.0000 + 4.0000i

Answer 2

[Emil Fresk]

Okey, I did what I probably should have done from the start: I saved a .mat file from Matlab and compared the results.

Conclusion: Matlab does NOT follow its own standard.

• The Flag field (figure 1-6) is Byte and Bit Reversed.
• The Flag field should be 0x00 when real data (not global as in the document).
• The Variable name type and Variable name length is 16-bit not 32-bit as in the document.
• The padding in the "text part" of the header should be 0x20.
• Writing "MI" for endianess is incorrect, it's the other way around (IM).

Question: Could MathWorks answer as to why it does not follow its own standard? Or have I misunderstood something fundamental?