Ordinary of Least Squares

As a bit of background information, I have yet to have taken linear Algebra (as it is not a pre req for an intro course) so I'm having a bit of trouble even researching for the solution. I was able to get the first part of the problem where we're supposed to create a vector named 'x' of 100 elements linearly spaced and ranging from -10 to 10. As well as creating a vector called y_original = m*x+b. (where: m = 2, b = -5). We also had to code some white noise and that is included as well.

This is what we are supposed to code

At this point, you now have realistic ‘x’ data and ‘y’ data. With this in mind, how could you re-arrange this data into a format such that you could use linear algebra to solve for the coefficients of the linear equation (m*x+b = y).

Hint: first re-write the above equation into matrix format and try adding more data points to the matrix.

What will be the dimensions of the matrices A, c, y? (Assuming the typical format of A*c = y) (I don't understand where these new matrices came from.)

Also is there a way to adjust this so that we're able to use if on a quadratic instead of just a linear?

I appreciate the time and help, thank you!

Answer 1

[Ameer Hamza]

Solving this problem does require knowledge of linear algebra, or atleast how matrices work. Conside this equation

it can be written in matrix form as

Now consider you have 100 'x' and 'y' datapoints, you will have 100 equations

and in matrix form

You can write this matrix equation like this

Since A is not a square matrix, the least square solution of this eqution is

In MATLAB, you can write find the solution using mldivide (\) operator.

MATLAB Example:

x = linspace(-10, 10, 100).';
y = 2*x-5;

A = [x ones(size(x))];
c = A\y;

Result

>> c
c =
    2.0000
   -5.0000

'c' return same cofficients used in line y = 2*x-5;

[Sam Zavala]

I see why it'd be really hard to solve this without having some experience or knowledge with matrices. I'll try using these with my code :) Thank you, I appreciate it a ton!

[Ameer Hamza]

I am glad to be of help! :)