Out of memory. Type HELP MEMORY for your options.

Hi, Please find below code.It gives Out of memory. HELP MEMORY for your options.Error in ndgrid (line 66) x = reshape(x(:,ones(1,prod(s))),[length(x) s]); . Please help me resolving this.

Thanks, Sita

clc; clear; tic p = haltonset(1) %'Skip',1e3,'Leap',1e2);

i1=1; j1=10;

i2=1; j2=10;

i3=1; j3=10;

i4=1; j4=10;

i5=1; j5=10;

i6=1; j6=10;

i7=1; j7=10;

i8=1; j8=10;

i9=1; j9=10;

i10=1; j10=10;

x1s(:,:)=p(i1:j1,:); x2s(:,:)=p(i2:j2,:); x3s(:,:)=p(i3:j3,:); x4s(:,:)=p(i4:j4,:); x5s(:,:)=p(i5:j5,:); x6s(:,:)=p(i6:j6,:); x7s(:,:)=p(i7:j7,:); x8s(:,:)=p(i8:j8,:); x9s(:,:)=p(i9:j9,:); x10s(:,:)=p(i10:j10,:);

    [aa bb cc dd ee ff gg hh ii jj]=ndgrid(x1s,x2s,x3s,x4s,x5s,x6s,x7s,x8s,x9s,x10s);
overnested = arrayfun(@(x1,x2,x3,x4,x5,x6,x7,x8,x9,x10)[x1,x2,x3,x4,x5,x6,x7,x8,x9,x10],aa,bb,cc,dd,ee,ff,gg,hh,ii,jj,'un',0);
celldisp(overnested)
x = permute([aa(:), bb(:),cc(:),dd(:),ee(:),ff(:),gg(:), hh(:),ii(:),jj(:)],[ 11 10 9 8 7 6 5 4 3 2 1]);%replaced below code

 x1temp=x(1,1,1,1,1,1,1,1,1,1,:)
 x2temp=x(1,1,1,1,1,1,1,1,1,2,:)
 x3temp=x(1,1,1,1,1,1,1,1,1,3,:)
 x4temp=x(1,1,1,1,1,1,1,1,1,4,:)
 x5temp=x(1,1,1,1,1,1,1,1,1,5,:)
 x6temp=x(1,1,1,1,1,1,1,1,1,6,:)
 x7temp=x(1,1,1,1,1,1,1,1,1,7,:)
 x8temp=x(1,1,1,1,1,1,1,1,1,8,:)
 x9temp=x(1,1,1,1,1,1,1,1,1,9,:)
 x10temp=x(1,1,1,1,1,1,1,1,1,10,:)

[Jan]

You forgot to mention in which line this error occurres.

[sita]

Hi,

 This is error description.

Out of memory. HELP MEMORY for your options.Error in ndgrid (line 66) x = reshape(x(:,ones(1,prod(s))),[length(x) s]);

Thanks, Sita

[sita]

Hi, For this error if i could store one row of overnested output values and use them for calculating function and clear memory followed by next row of overnested. Please find below code.If these two functions could give one row of output once then i think i will be able to resolve this error.Please help me in doing this.

Thanks, Sita.

[aa bb cc dd ee ff gg hh ii jj]=ndgrid(x1s,x2s,x3s,x4s,x5s,x6s,x7s,x8s,x9s,x10s); overnested = arrayfun(@(x1,x2,x3,x4,x5,x6,x7,x8,x9,x10)[x1,x2,x3,x4,x5,x6,x7,x8,x9,x10],aa,bb,cc,dd,ee,ff,gg,hh,ii,jj,'un',0);

x = permute([aa(:), bb(:),cc(:),dd(:),ee(:),ff(:),gg(:), hh(:),ii(:),jj(:)],[ 11 10 9 8 7 6 5 4 3 2 1])

[Jan]

I do not understand the problem. What is the goal of the arrayfun?

[José-Luis]

The usefulness of using a 10 dimensional ndgrid has to be questioned. Storing a double requires 64 bits or 8 bytes of memory. To give you an idea, assuming all dimensions are of equal length (all you x's have the same size), here are the storage requirements for 10-dimensional arrays:

• size 2: 2^10 * 8 = 8192 bytes -> 8kB
• size 5: 78125000 bytes -> 78MB
• size 10: 8e10 bytes -> 80GB (out of range for a desktop nowadays)
• size 20: 8.19e13 bytes -> 81TB (way out of theoretical range for 64bit computers)

I strongly suggest you revise your approach. Creating an ndgrid might not only be unnecessary, it is also probably a bad idea.

[sita]

Hi, clear; clc; p = haltonset(1)

x1s=net(p,3) x2s=net(p,4) x3s=net(p,5) [a b c ]=ndgrid(x1s,x2s,x3s); overnested = arrayfun(@(x1,x2,x3)[x1 x2 x3],a,b,c,'un',0); celldisp(overnested) x = permute([a(:), b(:),c(:)],[4 3 2 1])

x1t=x(1,1,1,:) x2t=x(1,1,2,:) x3t=x(1,1,3,:) in above code x1s,x2s,x3s are my input vectors.I am trying to take all possible combinations of all the elements of 3 vectors.X is my vector with all possible combinations and i stored all three variables in x1t,x2t, x3t. After this i calculate my function where my inputs are x1t,x2t,x3t.To overcome from this memory problem, If i could store each row at a time(i.e. x1t,x2t,x3t) and calculate my function and clear that variable and repeat the same. Please help to fix this problem. Thanks, Sita

Answer 1

[José-Luis]

All possible permutations in a loop:

 for ii = 1:numel(x1)
     for jj = 1:numel(x2)
        for kk = 1:numel(x3)
            your_input = [x1(ii) x2(jj) x3(kk)]
        end
     end
 end

This could help with your out of memory error. Depending on the sizes of the vector it might take a long time. Please accept an answer if it helps you.

[sita]

Thanks for your answer.Help full..

[José-Luis]

Please edit your code.

http://www.mathworks.com/matlabcentral/answers/13205-tutorial-how-to-format-your-question-with-markup

"Is not working correctly" is not an helpful description of your problem.

[sita]

for i = 1:numel(x3s) x3=x3s(i) for j = 1:numel(x2s) x2=x2s(j) for k = 1:numel(x1s)

             x1=x1s(k)
             yal =fc10(x1,x2,x3)
            your_input= [x1 x2 x3]

        end

     end

end hi, I edited my code and it is working.Though nested loops way solves memory problem it is very slow .If i want to do the same thing for ten variables i.e x1 ...x10. nested loops is very difficult.it takes very very long time.Please suggest me if there is any other method.

If i find a way of using permute instead of nested for loops with out memory problem,it helps to solve my problem.Please suggest me some method.

x = permute([aa(:), bb(:),cc(:),dd(:),ee(:),ff(:),gg(:), hh(:),ii(:),jj(:)],[ 11 10 9 8 7 6 5 4 3 2 1]);

[José-Luis]

You are trying to perform 10^10 operations. Depending on the complexity of your function, that is bound to take some time. As a quick test:

 tic;for ii=1:10^10;end;toc

Time elapsed = 19 seconds. Just going through the loop counter takes a while. I would image that even the simplest of function could take a very long time.

There might be a trade-off between speed and memory consumption. That being said, you have several alternatives to make your code faster. The first step is to use the profiler to find bottlenecks in your code and optimize the functions that are taking the most time. After doing that, you have several alternatives

• Parallelize your code (doc parfor, parallel computing toolbox and others)
• Get a faster computer
• Use a lower level language, such as C/C++ or Fortran that are bound to be much faster than Matlab.

Answer 2

[Jan]

"Out of memory" means, that you try to create an array, which does not fit into the available free memory.

The solutions are trivial:

1. Install more memory (twice as much is a good strategy)
2. Because RAM is cheap even more memory is fine
3. Switch off other applications, which occupy memory
4. Clear unused variables in Matlab
5. Use smaller data types if possible (a double needs 8 bytes, a uint16 only 2)
6. Use a 64-bit system and Matlab, such that you can use the bunch of gigabytes you have installed according to the points 1. and 2.

You find exactly the same answers, when you search in the forum for "Out of memory".

[EDITED]

Another problem might be, that you try to create a 10 dimensional ndgrid. I'm not sure why you use matrices as input to ndgrid, because I'd expect vectors -- but this might be meaningful for reasons I do no know. But 10 dimensional regular grids tends to be extremely huge. If each dimension has 10 elements, Matlab needs to store 10^10 elements. So I guess, that there is a design problem and you want to do something else.