Clearing a cell array does not release memory held by cell contents

I have a script that creates some cell arrays and then populates the cells at various levels with arrays of different sizes. At the end of this script, memory is tight and so I try to clear a few of the larger cell arrays that are not needed. When I try to clear the cell on the command line like so:

clear acell

this seems like it releases the header information only for the cell array. When I type

memory

it does not show a significant amount of memory released. Even when I go through a loop and manually release each array in the cell structure like this:

acell{idx} = [];

before clearing the cell structure, the memory is not freed. The OS shows that matlab is using less memory, but I cannot allocate arrays without running out of memory. I am running Matlab R2012a on Windows XP, 32 bit.

I would appreciate any insight that you have. Thanks for your help!

Alan

[James Tursa]

Can you post a short script that builds & clears the memory to show the apparent problem?

[Alan]

Sure.

If I check my memory available for all arrays before I do the following it is about 1400MB.

Then I do:

C = cell(2^15,1);

r = floor(rand(2^15,1)*100);

for idx=1:2^15

C{idx} = randn(r(idx));

end

clear C;

I'm not sure how to make the above look like code in this comment window, but when I perform the above "clear" operation, it shows that the memory used by Matlab is back down to where it was, but there is only 420 MB available to all arrays. I have lost 1 GB of available memory. As far as I can tell, I can never access the memory that matlab has freed until I exit matlab and restart it.

Answer 1

[Alan]

I think this question is answered by the documentation for memory, basically blaming it on the Windows Heap Manager (and I quote):

"MATLAB, by default, uses the standard Windows heap manager except for a set of small preselected allocation sizes. One characteristic of this heap manager is that its behavior depends upon whether the requested allocation is less than or greater than the fixed number of 524,280 bytes. For, example, if you create a sequence of MATLAB arrays, each less then 524,280 bytes, and then clear them all, the MemUsedMATLAB value before and after shows little change, and the MemAvailableAllArrays value is now smaller by the total space allocated.

The result is that, instead of globally freeing the extra memory, the memory becomes reserved. It can only be reused for arrays less than 524,280 bytes. You cannot reclaim this memory for a larger array except by restarting MATLAB."

So it seems that the memory becomes reserved, even though it seems like this memory is available to the rest of the system. :S IMHO this makes the process of actually using all or most of the memory available extremely awkward. From my 18 years of using Matlab, I expected better.

[Walter Roberson]

Hmmm, memory fragmentation indeed.

Answer 2

[Walter Roberson]

Suppose you do

Z = zeros(1e5,1e5);
A{1} = Z;
A{2} = Z;

Then because of MATLAB's copy-on-write semantics, A{1} and A{2} would point to the same memory block as Z points to, and will continue to point to that block until they are altered. If you were to now clear A{1}, then very little memory would be saved, as most of the work would just be in reducing the "number of times in use" counter for that memory block. Not until the last user of the block releases it (the count reaches 0) is the memory actually freed.

[Alan]

That makes sense except that these arrays are all allocated separately. Even if I "clear all" there is a lot of memory that is gone until I restart Matlab.

[James Tursa]

Again, I would request that you please post a short code segment that reproduces the apparent problem.

[Walter Roberson]

I'm wondering if this is an issue of memory fragmentation rather than memory leak ?

[James Tursa]

I am wondering if there is ever going to be some code for us to look at ...

[Daniel Shub]

I am guessing that Alan has moved on.