Should lsqcurvefit reshape the initializing parameter vector?

When I run the following simple test of lsqcurvefit in R2018a,

function test

    ctrue=[1;2;3];
    xdata=rand(3);

    ydata=F(ctrue,xdata);

    lb=-inf(3,1); ub=+inf(3,1);
    lb(1)=1; ub(1)=1;

    opts=optimoptions(@lsqcurvefit,  'SpecifyObjectiveGradient',true);

    c=lsqcurvefit(@F,[0;0;0],xdata,ydata,lb,ub,opts);

with this model function,

function [out,Jac]=F(c,xd)

   assert( isequal(size(c),[3,1]) , ...
          ['c is expected to be 3x1, but it is ',...
          num2str(size(c,1)),'x' num2str(size(c,2)) ])

   out=xd*c;

   Jac=xd;

the assert is triggered and I get,

 Error using test>F (line 21)
 c is expected to be 3x1, but it is 1x3

So lsqcurvefit is passing c as a 1x3 row vector, even though the initial c that I provided was a 3x1 column vector [0;0;0]. Should this occur? Shouldn't I be able to count on the initial vector's shape to determine the shape of the parameter arrays that get passed to the model function?

[Bruno Luong]

In the doc it witten "If the user-defined values for x and F are arrays, they are converted to vectors using linear indexing (see Array Indexing (MATLAB))."

So obviously the vector conversion is under lsqcurvefit control, and it obviously prefers a row-vector.

[Matt J]

Not sure which part of the doc you are citing, but here, it says,

When x0 is a matrix, solvers pass x as a matrix of the same size as x0 to both the objective function and to any nonlinear constraint function.

So either there is a bug in lsqcurvefit or there is a bug in the documentation. Or, I suppose, row vectors are not considered "matrices".

[Bruno Luong]

R2018b

>> doc lsqcurvefit

In the part 'Input Arguments fun — Function you want to fit"

10 lines down

Not it's not a BUG as I understood, that means user can pass X0 as matrix; but lsqnonlin converts it to vector. Same for F.

[Matt J]

I think that section of the documentation is mis-written. It is inconsistent with both the other documentation page that I cited for you, as well as the modification of my test below. Now, F(c,xd) expects c to be a 3x3 matrix. In this case, I find that the assert() is not triggered and the optimization completes successfully. In other words, c is being successfully passed as intended in 3x3 form.

function test

    ctrue=eye(3);
    xdata=rand(3);

    ydata=F(ctrue,xdata);

    c=lsqcurvefit(@F,zeros(3),xdata,ydata);

function out=F(c,xd)

     assert( isequal(size(c),[3,3]) , ...
            ['c is expected to be 3x3, but it is ',...
            num2str(size(c,1)),'x' num2str(size(c,2)) ])

     out=xd*c;

[Alan Weiss]

Both documentation sections are correct, but I can now see that the section on linear indices is not helpful. What happens is that, internally, solvers use linear indices for computing values, but for nonlinear objective and constraint functions the arguments are reshaped back to the x0 shape when used.

I will update the doc.

FYI, linear indices are crucial for linear constraint matrices A and Aeq, because for those arguments the x argument is reshaped to a column vector no matter the shape of x0.

Alan Weiss

MATLAB mathematical toolbox documentation

Answer 1

[Matt J]

After a bit of investigation, I think I've found where the problem is coming from and now that I understand it better, I'm pretty convinced it's a coding bug.

The problem occurs when one of the input upper/lower bounds pairs are equal, lb(i)=ub(i), meaning x(i) is known a priori. To handle this, lsqcurvefit tries to sift out those variables using the function snlsFixedVar, so that the optimization can be done in terms of the free variables only.

function [xcurr, fvec, LAMBDA, JACOB, EXITFLAG, OUTPUT, msgData] = ...
    snlsFixedVar(funfcn, xstart, l, u, verb, options, defaultopt, ...
    fval, JACval, caller, Jstr, computeLambda, mtxmpy, ...
    detailedExitMsg, optionFeedback, finDiffFlags, varargin)
%

%SNLSFIXEDVAR  Sparse nonlinear least squares solver with fixed variables
%
%   Locate a local solution to the box-constrained nonlinear least-squares
%   problem:
%
%              min { ||F(x)||^2 :  l <= x <= u }
%
%   where F:R^n -> R^m, m > n, and || || is the 2-norm. Also, at least one
%   element of x(i) is fixed, that is l(i) = x(i) = u(i).
%
%   This function first removes the fixed variables from the problem. The
%   reduced problem is passed to SNLS to be solved. Once the reduced
%   problem solution has been found, the fixed variables are added back to
%   the solution.

From comments in the first line of this function, it is clear the code intends to take control of the shape for the purpose of evaluating the user function:

 % Save shape of xstart so we can reshape for user function and output
 sizeX = size(xstart);

However, the saved shape sizeX is never passed to the reduced objective function, so this intent is never fulfilled.

% Restrict objective function to "free" variables.
funfcn = i_createObjectiveFcn(funfcn, l, idxFree, hasJacobMult);

Aside from this, I have to believe it is a bug if for no other reason than because the error only occurs when there is an lb(i)=ub(i) condition. In all other cases, the objective function appears to receive input in the shape of the initializing array, x0, consistent with existing documentation.

[Alan Weiss]

Thank you for reporting the results of your investigation. I have forwarded your comments to the development team.

Alan Weiss

MATLAB mathematical toolbox documentation

[dong]

I had the same problem and have been looking for solutions.

It seems that solver of lsqcurvefit reshape the xstart when there is an lb(i)=ub(i) condition. As a result, the size of reshaped xstart is small than the orignal xstart. Due to the reduction of xstart, it causes mass mess when the funfcn indexes parameter with a warning "Index out of matrix dimension".

I'm so depressed that I don't know how to solve the problem. Hope to get your help,thanks.

[Bruno Luong]

Three year after and the bug is still not corrected?

EDIT: it has been corrected in R2021a, dong seems to use older MATLAB version

[dong]

Bingo,the bug has been corrected in R2021a.Thank you very much.

[Matt J]

Unfortunately, there still seem to be some other issues when setting lb(i)=ub(i), see

https://www.mathworks.com/matlabcentral/answers/423334-should-lsqcurvefit-reshape-the-initializing-parameter-vector?s_tid=srchtitle

Answer 2

[Alan Weiss]

It is possible that the behavior of solvers is inconsistent, so when the vector orientation matters, I suggest that you be proactive and force the dimensions you like. For example,

function [out,Jac]=F(c,xd)
c = c(:); % for column c
c = c(:).'; % for row c
c = reshape(c,3,1); % another possibility

Alan Weiss

MATLAB mathematical toolbox documentation

[Matt J]

OK, thanks Alan, but does that then mean that this page has a documentation bug? The part that says,

When x0 is a matrix, solvers pass x as a matrix of the same size as x0 to both the objective function and to any nonlinear constraint function.

seems to be making a claim for all solvers.

[Alan Weiss]

Yes, I'll have to look at this again. When I wrote that documentation, I was thinking only of fmincon and fminunc, which do behave that way. But for least-squares solvers, well, I wasn't thinking of them, and you have discovered that they don't behave the same way, which is why I put the sheepish comment "It is possible that the behavior of solvers is inconsistent..."

Alan Weiss

MATLAB mathematical toolbox documentation

[Matt J]

But for least-squares solvers, well, I wasn't thinking of them, and you have discovered that they don't behave the same way

I see. But oddly, they do seem to behave the same way most of the time. My modified example, verifies that the shape of a 3x3 parameter matrix is preserved in lsqcurvefit, just as it would be in fmincon. So, it remains a bit murky what the intended behavior is.

[Bruno Luong]

Conclusion: it's a big mess. Just take control over the shape and fo not rely on MATLAB or the doc.

[dong]

How to deal with the solver of lsqcurvefit when there is an lb(i)=ub(i) condition.

issure:

when there is an lb(i)=ub(i) condition is the xstart, the function of snlsFixedVar first removes the fixed variables from the problem, it means the size of xstart is changed,as a results, the function of funfcn cannot index the correct parameter form the xstart.

[Bruno Luong]

You can do like this (I use Matt's example)

ctrue=[1;2;3];
xdata=rand(3);
ydata=F(ctrue,xdata);

lb=-inf(3,1);
ub=+inf(3,1);
lb(1)=1; ub(1)=1;

opts=optimoptions(@lsqcurvefit,  'SpecifyObjectiveGradient',true);

% This is original code, that throw error
%c=lsqcurvefit(@F,[0;0;0],xdata,ydata,lb,ub,opts);

% Use a wrapper
c=lsqcurvefit(@(varargin) Fexpand(lb < ub, varargin{:}), ...
    [0;0;0],xdata,ydata,lb,ub,opts);

% Wrarpper to expand back decision variables
function varargout = Fexpand(b, varargin)
c = varargin{1};
try % newer MATLAB has fix the bug
    c(b) = c;
end
varargout = cell(1,nargout);
[varargout{:}] = F(c, varargin{2:end}); % <- Call  original model
end

% This is the original model, intended to used with lsqcurvefit
function [out,Jac]=F(c,xd)
out=xd*c;
Jac=xd;
end

[dong]

Thank you very much!

It is appreciate that you offer me the function Fexpand. it is useful.

% Wrarpper to expand back decision variables
function varargout = Fexpand(b, varargin)
c = varargin{1};
try % newer MATLAB has fix the bug
    c(b) = c;
end
varargout = cell(1,nargout);
[varargout{:}] = F(c, varargin{2:end}); % <- Call  original model
end

Though I understand it with a little bit difficult, it successfully solves the problems I have encountered.

But I had a new problem.

What does this line codes mean below？

@(varargin) Fexpand(lb < ub, varargin{:})

and if the original model “F” return a function handle，how can I use the function of Fexpand?

I'm so depressed that I don't know how to solve the problem. Hope to get your help,thanks.

email:[e-mail]