How I can I save large tables without using save(data, '-V7.4')?

I'm saving data in large tables whilst fitting a model to a small set of data. This is producing tables which are quite small. I am repeating this process for 25 data sets.

There are several tables nested within the cells of other table. There's a summary table with each row containing a set of grid parameters each with a measure of fit. Within this subtable are the iterations of fminsearch for the specific combination of parameters.

I have a major problem saving this data. For around 20 data sets the save function works. The file it produces is nice and small. After another data set I get an error during telling me that matlab is 'Out Of Memory during serialization of the Subsystem Data. Error using save. Error closing file'.

After much searching I've saved the file using save(data, '-V7.4') but the filesize is 4.5 GB which is far larger than the 4MB file created before the error message. What is going on and can I save using the normal save function? Or can matlab compress the 4.5GB file it is creating?

Many thanks,

Dr James Tripp

[dpb]

I'd suggest file bug report/contact TMW tech support directly on this one.

www.mathworks.com

[per isakson]

"'-V7.4'" . Is that an undocumented argument value? See

save, Save workspace variables to file, Input Arguments

[Titus Edelhofer]

I guess James meant 7.3 ...

Answer 1

[Titus Edelhofer]

Hi James,

generally speaking, the v7.3 is an uncompressed format, whereas the default format (7.0) is a compressed format. It looks like the following happens:

• Your data are large (what does "memory" in MATLAB tell you or the taskmanager, is this true?)
• Your data compress very well.

Therefore the file in 7.0 is small, but for creating it, memory is needed for the compression process. In format 7.3 it's as large as before in memory, but no additional memory for compressing is used, therefore you don't get the Out Of Memory error.

Titus