Analytical hessian for large scale optimization in fmincon
4 views (last 30 days)
Chaitanya Awasthi on 24 Apr 2021
I have currently set up an optimization problem in Matlab which stems from an inverse problem in design. I tested my setup with small test cases (decision space ~ 80 variables) and fmincon works like a charm. I did some calculations to get rough estimates on my actual optimization problem and here is what it looks like:
Decison space: ~ 20,000 variables
Nonlinear equality constraints: ~15.000
Linear inequality constraints: ~5,000
Clearly, its a large scale optimization problem and, per Matlab's recommendations, I should use the interior-point method. I would also like to note that I do have access to the gradient of the cost function and possibly Hessian (likely going to be ridiculously large, and I haven't coded it up yet). Here are a few questions that I would very much appreciate responses to:
- Can fmincon efficiently handle a problem such as mine, or am I better off using a dedicated commercial solver?
- I think I would absolutely need to use l-bfgs for my hessian approximation (when I don't pass an analytical hessian). Because l-bfgs approximates hessian using limited memory, does passing an analytical hessian worth it? The reason I ask is because I am under the impression that if I pass an analytical hessian, I will not be able to use limited memory approximation of hessian and that the interior-point algorithm is going to use the full hessian per iteration to converge to a solution.
- Does fmincon work if I use the sparse function to pass an analytical hessian? (I am fairly certain that my Hessian is going to be sparse)
Thank you for time. Please feel free to ask for any clarification.
Matt J on 24 Apr 2021
Can fmincon efficiently handle a problem such as mine, or am I better off using a dedicated commercial solver?
The only way to find out is to try...
Does fmincon work if I use the sparse function to pass an analytical hessian? (I am fairly certain that my Hessian is going to be sparse)
Yes. You can also use a Hessian Multiply Function to conserve memory,