Coefficient Standard Errors and Confidence Intervals

Coefficient Covariance and Standard Errors

Purpose

Estimated coefficient variances and covariances capture the precision of regression coefficient estimates. The coefficient variances and their square root, the standard errors, are useful in testing hypotheses for coefficients.

Definition

The estimated covariance matrix is

$$\sum =MSE{\left(X^{\prime }X\right)}^{-1},$$

where MSE is the mean squared error, and X is a matrix of values for the terms in the model formula. CoefficientCovariance, a property of the fitted model, is a p-by-p covariance matrix of regression coefficient estimates. p is the number of coefficients in the regression model. The diagonal elements are the variances of the individual coefficients.

How To

After obtaining a fitted model, say, mdl, using fitlm or stepwiselm, you can display the coefficient covariances using

mdl.CoefficientCovariance

Compute Coefficient Covariance and Standard Errors

This example shows how to compute the covariance matrix and standard errors of the coefficients.

Load the sample data and define the predictor and response variables.

load hospital
y = hospital.BloodPressure(:,1);
X = double(hospital(:,2:5));

Fit a linear regression model.

mdl = fitlm(X,y);

Display the coefficient covariance matrix.

CM = mdl.CoefficientCovariance

CM = 5×5

   27.5113   11.0027   -0.1542   -0.2444    0.2702
   11.0027    8.6864    0.0021   -0.1547   -0.0838
   -0.1542    0.0021    0.0045   -0.0001   -0.0029
   -0.2444   -0.1547   -0.0001    0.0031   -0.0026
    0.2702   -0.0838   -0.0029   -0.0026    1.0829

Compute the coefficient standard errors.

SE = diag(sqrt(CM))

SE = 5×1

    5.2451
    2.9473
    0.0673
    0.0557
    1.0406

Coefficient Confidence Intervals

Purpose

The coefficient confidence intervals provide a measure of precision for linear regression coefficient estimates. A 100(1–α)% confidence interval gives the range that the corresponding regression coefficient will be in with 100(1–α)% confidence.

Definition

The software finds confidence intervals using the Wald method. The 100(1 – α)% confidence intervals for regression coefficients are

$$b_i\pm t_{\left(1-\alpha /2,n-p\right)}SE\left(b_i\right),$$

where b_i is the coefficient estimate, SE(b_i) is the standard error of the coefficient estimate, and t_{(1–α/2,n–p)} is the 100(1 – α/2) percentile of the t-distribution with n – p degrees of freedom. n is the number of observations and p is the number of regression coefficients.

How To

After obtaining a fitted model, say, mdl, using fitlm or stepwiselm, you can obtain the default 95% confidence intervals for coefficients using

coefCI(mdl)

You can also change the confidence level using

coefCI(mdl,alpha)

For details, see the coefCI function of LinearModel object.

Compute Coefficient Confidence Intervals

This example shows how to compute coefficient confidence intervals.

Load the sample data and fit a linear regression model.

load hald
mdl = fitlm(ingredients,heat);

Display the 95% coefficient confidence intervals.

coefCI(mdl)

ans = 5×2

  -99.1786  223.9893
   -0.1663    3.2685
   -1.1589    2.1792
   -1.6385    1.8423
   -1.7791    1.4910

The values in each row are the lower and upper confidence limits, respectively, for the default 95% confidence intervals for the coefficients. For example, the first row shows the lower and upper limits, -99.1786 and 223.9893, for the intercept, $${\beta }_0$$. Likewise, the second row shows the limits for $${\beta }_1$$ and so on.

Display the 90% confidence intervals for the coefficients ($$\alpha$$ = 0.1).

coefCI(mdl,0.1)

ans = 5×2

  -67.8949  192.7057
    0.1662    2.9360
   -0.8358    1.8561
   -1.3015    1.5053
   -1.4626    1.1745

The confidence interval limits become narrower as the confidence level decreases.