Column of ONES in multiple regression

What is the meaning of adding a column of ones in the multiple regression? Thanks a lot, G.

Answer 1

[Shashank Prasanna]

The columns of ones is for the intercept or the constant term as already pointed out above.

Ax = b

[a11 a12 1] [x1]   [b1]
[a21 a22 1] [x2] = [b2]
[a31 a32 1] [x3]   [b3]

It is basically for the x3 term in the model
     b = a1*x1 + a2*x2 + 1*x3
  => b = a1*x1 + a2*x2 + x3

Since all regress does is A\b to solve the system, the ones have to explicitly be added.

In any case I recommend the use of the new LinearModel.fit function instead of regress, since it does this for you and also directly provides the fit statistics.

Answer 2

[Image Analyst]

That means that your equation (model) will have a constant offset, like the "a3" in y = a1 * x1 + a2 * x2 + a3. In the manual method of making the arrays for least squares regression, you make a "tall" array of your measured (observed) values for x1 and x2

x1(1), x2(1), 1
x1(2), x2(2), 1
x1(3), x2(3), 1
x1(4), x2(4), 1
...
x1(n), x2(n), 1

Then you do the matrix math and you get estimates for all the "a" coefficients of your model.

[Nuchto]

Unfortunately I couldn't follow your answer ;( Why is a constant offset necessary? Wouldn't it be there anyway without specifying it (zero)?

[Image Analyst]

No there wouldn't. You'd have an equation like this:

y = a1 * x1 + a2 * x2

instead of this:

y = a1 * x1 + a2 * x2 + a3

If you want, in the first equation you can say that the offset is zero but I don't know how that helps. I hope you can see that the second equation is different than the first equation. The second equation will give you an offset. If, for x = [0 1 2 3 4] you had y = [10 11 12 13 14], then you'd have y=x+10. If you didn't have the 1's you'd get an equation like y=12*x or something - totally different.

Answer 3

[Oleg Komarov]

In regress(), you need to add a column of ones to allow for an intercept in the regression. Otherwise, you constrain the fit to go through the origin.

[Nuchto]

But isn't the column of ones constraining the fit equally?

[Oleg Komarov]

No. The coefficient of the intercept is determined by least squares method and not set to 0 ex-ante.

[Nuchto]

I would need a more basic explanation. I am not a mathematician... What does the "fit to go through the origin" mean?

[Shashank Prasanna]

It means the constant acts as an intercept term I found a random image which might make things clearer to you:

http://www.mbs.edu/home/jgans/mecon/value/media%20%20files/static%20graphics/jpegs/Segment%203/3_2_static_slope_intercept_form.jpg

the 10 is the intercept or in your case the value of the coefficient of ONE. Without the 10, the line would pass through the origin.

If you look at my example below in my plot, it becomes clearer how the ONEs contribute to the intercept 'x3'