Which algorithm converges faster?

Hello, everyone. I'm having a large scale unconstrained optimization problem. If I treat the unconstrained problem as a constrained problem with infinity constraints, I should be able to use both the fminunc and fmincon function. When using the fminunc function, I should provide the gradient and the sparse pattern of the Hessian. While using the fmincon function, I can choose the l-bfgs method to approximate the Hessian. So, here is my question. Which one converges faster? Are there any good reasons to choose one over another? Or any principles to guide such choice?

Answer 1

[Alan Weiss]

It is hard to know in advance which algorithm will be faster. Generally, the more information you can give to an algorithm (explicit gradient, Hessian sparsity, etc.), the faster the computation, but there are counterexamples. The fmincon trust-region-reflective algorithm is essentially the same as the fminunc trust-region algorithm, so there is no point trying both. But the fmincon sqp and interior-point algorithms are different, though sqp is not large-scale.

One of the most important things is to give a good initial guess, and that is often very difficult to obtain.

If the problem has no constraints, but you can prove that the solution lies in a bounded region, then it is almost certainly better to give the bounds explicitly and to use fmincon.

Good luck,

Alan Weiss

MATLAB mathematical toolbox documentation