non linear regression model with constraints

I have a set of data that I want to fit with a non linear regression model. The regression function is y=k*x.^alpha (I usually use fitnlm). I would like to put a constraint on the area under the curve. I tried computing the integral and making a substitution but when alpha is close to -1 (and it is my case), the integral has a different expression and thus I do not know how to express the area in a way that works for alpha=-1 and for alpha different from -1. Many thanks in advance

[Kaushik Lakshminarasimhan]

Can you share your code where you define the constraint for the case when alpha is far away from -1?

[FastCar]

I was not able to use a constraint, I just make a substitution, given that the area is A=k/(alpha+1)*[x2^(alpha+1)-x1^(alpha+1)] I can compute k as a function of alpha, the problem is that this definition of the integral requires alpha different from -1, so when the cost function is not continuous. I was not able to use fmincon for my purpose because I was not able to write the constraint in a proper way. I tried to use the integral function and let matlab to care about the area computation but it did not work. I also tried to define the different value of the integral when alpha=-1 but it did not work as well. Let me know if I was clear. Cheers

Answer 1

[Matt J]

Using fmincon, solve 3 separate problems and take the best solution of the three (the solution with the least regression error):

Problem 1: Solve subject to the constraints

   alpha<=-1-delta
    k*[x2^(alpha+1)-x1^(alpha+1)] - (alpha+1)*A=0

where delta>0 is as small as possible without running into numerical problems in the evaluation of 1/(alpha+1)*[x2^(alpha+1)-x1^(alpha+1)].

Problem 2: Solve subject to the constraints

    alpha>=-1+delta
    k*[x2^(alpha+1)-x1^(alpha+1)] - (alpha+1)*A=0

Problem 3: Solve subject to -1-delta <= alpha <= -1+delta. I would just approximate the solution here as alpha=-1 and k=A/log(x2/x1).

Answer 2

[Matt J]

If you know alpha is very close to -1 (nearly within numerical precision), replace the regression function with y=k*T(x,alpha) where T(x,alpha) is a Taylor expansion of x^alpha about alpha=-1.

[FastCar]

I think that my problem is that I also would like to get the uncertainty on the parameters of the regression, so I used fitnlm. How can I get the uncertainty on the parameters from fmincon? Is there a way to give a constraint to fitnlm?

[Matt J]

No, but if you follow the approach with the Taylor expansion, there is no need for fmincon. You can use the area constraint to eliminate k, just as you were planning to do originally.

Alternatively, solve first with fmincon. Then plug the alpha you get as the initial point beta0 for fitnlm. Here, you have prior knowledge of whether alpha=-1 or not and so can choose the expression for A accordingly. Since you are initializing fitnlm with the known solution, it should converge in few or zero iterations and give you the goodness of fit metrics.

Answer 3

[Matt J]

If you know a prior that alpha is very close to -1, maybe you should just bound the optimization to a small interval -1-delta <= alpha <= -1+delta around -1. Then you can just approximate the area as constant over that interval,

A=k*log(x2/x1)

and eliminate k from the optimization,

k=A/log(x2/x1)

[FastCar]

And what if the confidence interval of the solution streaches around -1 but streaches quite a lot?

[Matt J]

Well, that's the same as saying you don't really have any a priori knowledge of where alpha lies. That takes you back to my original proposal, which is a generic solution.