Same optimization algorithm but different results?

I noticed that when I run an optimization method in MATLAB with a fmincon function, for example active-set method, the resulting coefficients and computation time is different every time I run the algorithm. I do not change the starting points, I just run the method one after another.

I understand this is the case for global optimization algorithms as they are using stochastic algorithms in the background. But why the same thing happens for the local minimum methods?

Answer 1

[Bjorn Gustavsson]

We ran into similar results using fminsearch. This was for matlab-versions from around 2010-2011, and we boiled our problems down to parallelization of the sum-function - in our error-function we had something like

V = sum(V)/sum(V)*V;

(not that explicitly), and due to that parallel implementation the two calls to sum resulted in minor differences between them, this made the result very different for some starting-points...

Some matlab-version later the sum implementation were again robust.

What problem you have is very difficult to guess given the small amount of information you've given...

HTH HTH

[Luce Ca]

Hi, what kind of info do you need?

The difference is very small, for example, once the coefficient is 0.4567 and the second time is 0.4569

[Bjorn Gustavsson]

Oh, I'm mostly done here...

...but if you want someone else to chime in you'd need to give the bare-bone code reproducing the outcome.

One more thing you should do first: set the display option to fmincon to 'iter', then you'll see where things start to differ.

HTH