Matlab crashes with large matrix
14 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
valentino dardanoni
el 25 de Oct. de 2023
Comentada: Bruno Luong
el 25 de Oct. de 2023
I need to create a large matrix F=[F1, F2, ... , FN] where each Fn has the same number of rows (about 12,000) but different number of columns. What I am doing now is to create F as a cell array using a loop with element F{n} = Fn. The problem is that when the number of columns of F is very large, Matlab crashes: at some point it shuts down (I run this on a server, and I find matlab closed). I suspect that it shuts down when I convert the cell array into a single matrix, F=cell2mat(F);
Notice that everything works ok when F has a number of columns less than say, say 1,000,000,
Any help is much appreciated. Thanks!
0 comentarios
Respuesta aceptada
Walter Roberson
el 25 de Oct. de 2023
Editada: Walter Roberson
el 25 de Oct. de 2023
Well, that is certainly possible... if you make the arrays large enough then there might just not be enough memory available.
Is it possible to predict the number of columns that will be needed for each Fn before doing the actual callculations? If so then you could consider doing that calculation and totaling them, and pre-allocating the output array, then writing directly into the output array instead of building the cell array. If you pre-build the sizes then you can use cumsum() to create an array that will tell you which indices to write at
As a side note: it is more efficient to use
F = horzcat(F{:});
than F = cell2mat(F); -- but this would not solve your memory problems.
2 comentarios
Bruno Luong
el 25 de Oct. de 2023
No it's not normal. But when all memory are consumed, many tasks are slowing down and requesred memory get swap to hard drive. The PC are in a "non standard" functioning mode and things can get slippy from there. Even pop up a warning message or windows become a hercules task.
Más respuestas (1)
Bruno Luong
el 25 de Oct. de 2023
Editada: Bruno Luong
el 25 de Oct. de 2023
For 1e6 column the memory required in Gb for one array is
memgGb = 1e6*12e3*8 / 1e9
And you have the cell array + the concatenate array (single contiguous emory blockà, so you need 192 Gb of free memory.
You might consider using single-class array, write directly to the allocated matrix as Walter suggests or using tall array assuming you store the transposed of the original intended array.
2 comentarios
Bruno Luong
el 25 de Oct. de 2023
licols is Matt's FEX, but yeah I quite participate to popular the idea. Thanks
Ver también
Categorías
Más información sobre Loops and Conditional Statements en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!