Out of memory error

43 views (last 30 days)
Yagiz Dereboy
Yagiz Dereboy on 9 Aug 2022
Answered: Yagiz Dereboy on 21 Aug 2022
I am trying to run a optimzation problem but I am keep getting out of memory error. It's a 20k variable 300 constraint problem. I dont understand why this happens. I'll give the constraint functions and the main script respectively. This is a modified version of the p-median problem but thats not very important I guess for my question at hand.
function constraints = constraintFcnEq(x,y)
for i = 1:150
constraints(i) = sum(x(i,:)) == 1;
end
constraints(151) = sum(y) == 3;
end
function constraints = constraintFcnIneq(x,y)
for i = 1:150
constraints(i) = sum(x(:,i)) - 150*y(i) <= 0;
end
end
A = readmatrix("iris.csv");
A = A(:,2:5);
A = normalize(A);
dist_mat = pdist(A);
dist_mat = squareform(dist_mat);
x0x = zeros(150,150) + 1/150;
x0y = zeros(150,1) + 3/150;
x = optimvar("x",150,150,"LowerBound",0,"UpperBound",1);
y = optimvar("y",150,1,"LowerBound",0,"UpperBound",1);
initialPoint.x = x0x;
initialPoint.y = x0y;
problem = optimproblem;
problem.Objective = sum(dist_mat.*x,...
'all')+ 100*sum(10.*(y.^2)./(10.*(y.^2) + 0.01));
problem.Constraints.constraint1 = constraintFcnIneq(x,y);
problem.Constraints.constraint2 = constraintFcnEq(x,y);
options = optimoptions("fmincon","Display","iter","PlotFcn","optimplotfval");
show(problem);
[solution,objectiveValue,reasonSolverStopped] = solve(problem,initialPoint,...
"Solver","fmincon","Options",options);
  2 Comments
Yagiz Dereboy
Yagiz Dereboy on 9 Aug 2022
Thank you for the answer. I don't know what do you mean by means to run my code. I can send you the iris.csv. But in my mind I sent the constraint functions and the main script. Please tell me what else I can do to help you help me:)

Sign in to comment.

Accepted Answer

John D'Errico
John D'Errico on 9 Aug 2022
Edited: John D'Errico on 9 Aug 2022
You have 20000 variables. So fmincon will be computing a Hessian matrix internally of size 20000*20000 = 4e8. But that matrix is double precision. So it will alone require 8*4e8=3.2e9. So roughly 3.2 GB of RAM will be required. And if a copy is ever made, or say, a factorization of the matrix is ever computed, then you need to immediately double the memory required. Fmincon will indeed need to factor that matrix, so you are starting to get in trouble. And that is only for that one array. Worse, MATLAB stores arrays in CONTIGUOUS blocks of memory. So you need to have enough RAM around so that MATLAB will be happy.
These days, it is still not uncommon for people to try to use MATLAB to solve problems like this on a computer with 4GB of RAM, Even 8GB of RAM would not be even close to enough here.
  1 Comment
Yagiz Dereboy
Yagiz Dereboy on 9 Aug 2022
Thank you so much for your answer. May I ask you an another question if it is not too much. May I do a lot better job with programming language like c++

Sign in to comment.

More Answers (1)

Yagiz Dereboy
Yagiz Dereboy on 21 Aug 2022
Just a heads up for everyone with memory problems, use L-BFGS instead of BFGS

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by