Obtaining close results with and without using MultiStart and GlobalSearch for fitting data to a non-linear model

I am trying to fit an experimental data to a nonlinear model. I tried curvefitting tool and used "trust-region" algorithm but by changing the startingpoints the result was not changing and I was getting fitting parameter values inconsistence with the physics of the problem. Then, I tried "Levenberg-Marquardt" algorithm and I see the obtained fitting parametes change even with small changes in the startingpoints. So, I thought it might be helpful to see how things change by using multistart and globalsearch which use different startpoints. Also, I thought because I do have access to more algorithms in the optimization toolbox that might be helpful.

Now, I have written my codes in optimization toolbox and I have used "lsqcurvefit", "lsqnonlin" and also "fmincon" with multistart and also "fmincon" with globalsearch. I have tried almost all the algorithms of each solver for both local solver and also in the multistart and globalsearch. What I see is by using the same startingpoints I see close or same results for both local solver and multistart (globalsearch). And the estimated error doesn't change that much by using multistart (globalsearch). I am wondering does this mean that for my function the local and global minima are the same? If yes, so why I see my result is changing by changing startpoints when using "levenberg-Marquardt" algorithm in curvefitting tool (Also by using some other algorithms like "sqp" in optimization tool) or is there a problem that I don't figure it out?

Also, I see the result of the multistart changes with changing the startpoints which is not expected because multistart searches for global optima and so the result should not be dependent on the startingpoint similar to local solvers. I am wondering why is that?

I have attached the code for my "lsqcurvefit" together with "multistart" trial and two series of my data for reference.

Thank you in advance!

Answer 1

[Matt J]

I am wondering does this mean that for my function the local and global minima are the same?

It might, but there's never any guarantee of that. Both the local and global solvers can fail. You haven't mentioned what exitflags are reported, but even the exitflag is the solver's best guess at whether a solution was found.

If yes, so why I see my result is changing by changing startpoints when using "levenberg-Marquardt" algorithm in curvefitting tool

There may be multiple solutions. Or, you might be getting stuck in a startpoint-dependent local minimum.

Also, I see the result of the multistart changes with changing the startpoints which is not expected because multistart searches for global optima and so the result should not be dependent on the startingpoint similar to local solvers.

Again, there's no guarantee of that when you have multiple global minima. Consider the function f(x)=1. Here, any startpoint is a global minium, so changing the startpoint(s) will give you different solutions every time.

[Shaily_T]

Thank you so much for your answer!

It might, but there's never any guarantee of that. Both the local and global solvers can fail. You haven't mentioned what exitflags are reported, but even the exitflag is the solver's best guess at whether a solution was found.

The exitflags in most of the cases where 1 or 2 and I am sure the solvers converged in these cases. I have also attached my function. I am wondering is there a way to check if the local and global minima are the same in my case? I mean by evaluating the function.

There may be multiple solutions. Or, you might be getting stuck in a startpoint-dependent local minimum.

Is there a way that I can figure out which of these probabilities is the case here?

And does the fact that when using "levenberg-Marquardt" algorithm in curvefitting toolbox changing startingpoints leads to change in the obtained value for fitting parameters mean I cannot rely on the answers? If yes, do you have a suggestion? Because in the other algorithm "trust-region" the result doesn't change with the change of startingpoints which I think doesn't make sense either because sometimes it leads to unphysical results. And also in the optimization algorithms I do have similar problem that for some of them the reults are not dependent on the startingpoints while for others it depends a lot on the value of the startingpoints.

Again, there's no guarantee of that when you have multiple global minima. Consider the function f(x)=1. Here, any startpoint is a global minium, so changing the startpoint(s) will give you different solutions every time.

Does that mean if here I see my results for multistart are changing with the startingpoints, so my function have multiple global minima? Does thinking the other way round make sense here?

Thank you in advance!

[Matt J]

The only real way to verify anything is to do a grid search for the global minima. Don't know if that's possible for your problem.

[Shaily_T]

Thank you! I am not familiar on how to do a grid search. Do you know a link for that?

[Matt J]

Do you know what a grid search is?

[Shaily_T]

No, I tried to search for that, but I am still struggling.

This is the best link I've found:

https://www.mathworks.com/help/gads/isolated-global-minimum.html

Section:

MultiStart with a Regular Grid of Start Points

[Matt J]

It's just a parameter sweep. You discretize your parameter space into a discrete grid of points and do brute force evaluation of the objective values at all the points. If your parameter space is two dimensional, for example, you could make a surface plot to see where the local and global minima are. It usually isn't tractable for greater than 3 dimensions. Also, when you do it, you want to try to vectorize the function evaluations so that you don't have to explicitly loop over the grid points. Here is a thread where a 2D example of this is being discussed:

https://www.mathworks.com/matlabcentral/answers/1704920-problems-with-fminsearch-giving-startvalues-as-result#answer_951130

[Shaily_T]

Thank you so much!

I just had a look there.

I have already tried parameter sweep in my case (not using grid search but by changing the parameters manually in the function to see the results) and indeed I have not seen an improvement in the fitted curve. I am wondering do you think even in this case it might be helpful to do a grid search for the initial points of my function?

I have posted a question here about my actual problem for fitting these data to my model function and in the comment I have elaborated more on my function and problem: https://www.mathworks.com/matlabcentral/answers/1704170-getting-inconsistent-results-for-fitting-different-series-of-data-to-a-nonlinear-function?s_tid=mlc_ans_email_view#answer_950110

I apprecitae your time and thank you in advance!

[Shaily_T]

Hi again @Matt J

For your suggestion of doing a grid search for the global minima, I tried to plot the function that I want to optimize vs two of my fitting parameters (so, the other two are constant) to have a sense of what is going on. I have attached the obtained surfplot (SSE is the function to be minimized, a, d, sigma are parameters that I want to obtain by minimization). I am wondering do you think if a grid search would be helpful in my case?

I apreciate your comment!