Another example where fitrlinear performs badly and I don't know where the problem is

I'm testing the three functions fitlm, fitglm, and fitrlinear by trying linear regression using the same dataset. fitrlinear performs very poorly compared to fitlm and fitglm. I was expecting the same models (therefore the same MSE's) for the three different functions.

I had another issue with fitrlinear when ridge regularization is used. Here is the link to that question,wich has not been answered yet.

https://www.mathworks.com/matlabcentral/answers/1682569-something-wrong-in-fitrlinear-with-ridge-regularization?s_tid=prof_contriblnk

Answer 1

[Milan Bansal]

Hi Peter He,

It is my understanding that you are testing "fitlm", "fitglm", and "fitrlinear" function to fit a linear model to your dataset but the "fitrlinear" model is performing poorly as compared to other two models. Also the Mean Square Error (MSE) for "fitlm" and "fitglm" model are same but it significantly high in case of "fitrlinear" model.

The "fitrlinear" model is optimized for large-scale, high-dimensional data. It uses iterative solvers that are more scalable to large datasets, such as stochastic gradient descent (SGD) or dual coordinate descent. The convergence is subjected to tolerance on Beta coefficients.

To achieve the same performance and MSE as "fitlm" and "fitglm" models, set the value "BetaTolerance" parameter in "fitrlinear" as 0. Also ensure that the "lambda" parameter is also set to 0 to avoid regularization. Please refer to the code snippet below: -

mdl = fitrlinear(trainingSet_X,trainingSet_Y,'Learner','leastsquares','Lambda',0,"BetaTolerance",0)

Please refer to the documentation below to learn more about "fitrlinear" model.

https://www.mathworks.com/help/stats/fitrlinear.html

Please refer to the documentation below to learn more about "BetaTolerance" parameter of "fitrlinear" model.

https://in.mathworks.com/help/stats/fitrlinear.html#d126e467297

Hope it helps!

[Peter He]

Hi Milan,

Thank you very much for taking time to answer my question! It's been over one and a half year and I didn't know the answer to the discrepancy among those functions. Now, thanks to you, I know! I tested your suggestion and it worked out perfectly!

Thanks again for your generous help and Happy New Year to you!

Peter