Each regression function has a specific operation. This section
shows how to use these functions to perform specific types of regressions.
To illustrate use of the functions for various regressions, “typical”
usage is shown with optional arguments kept to a minimum. For a typical
regression, you estimate model parameters and residual covariance
matrices with the mle
functions and estimate the
standard errors of model parameters with the std
functions.
The regressions “without missing data” essentially ignore
samples with any missing values, and the regressions “with
missing data” ignore samples with every value missing.
Multivariate normal regression, or MVNR, is the “standard” implementation of the regression functions in Financial Toolbox™ software.
Estimate Parameters
[Parameters, Covariance] = mvnrmle(Data, Design);
Estimate Standard Errors
StdParameters = mvnrstd(Data, Design, Covariance);
Estimate Parameters
[Parameters, Covariance] = ecmmvnrmle(Data, Design);
Estimate Standard Errors
StdParameters = ecmmvnrstd(Data, Design, Covariance);
Least-squares regression, or LSR, sometimes called ordinary least-squares or multiple linear regression, is the simplest linear regression model. It also enjoys the property that, independent of the underlying distribution, it is a best linear unbiased estimator (BLUE).
Given m = NumSamples
observations,
the typical least-squares regression model seeks to minimize the objective
function
which, within the maximum likelihood framework of the multivariate
normal regression routine mvnrmle
, is equivalent
to a single-iteration estimation of just the parameters to obtain Parameters
with
the initial covariance matrix Covariance
held fixed
as the identity matrix. In the case of missing data, however, the
internal algorithm to handle missing data requires a separate routine ecmlsrmle
to
do least-squares instead of multivariate normal regression.
Estimate Parameters
[Parameters, Covariance] = mvnrmle(Data, Design, 1);
Estimate Standard Errors
StdParameters = mvnrstd(Data, Design, Covariance);
Estimate Parameters
[Parameters, Covariance] = ecmlsrmle(Data, Design);
Estimate Standard Errors
StdParameters = ecmmvnrstd(Data, Design, Covariance);
Given m = NUMSAMPLES
observations,
the typical covariance-weighted least squares, or CWLS, regression
model seeks to minimize the objective function
with fixed covariance C0.
In most cases, C0 is a diagonal matrix. The inverse matrix has diagonal elements that can be considered relative “weights” for each series. Thus, CWLS is a form of weighted least squares with the weights applied across series.
Estimate Parameters
[Parameters, Covariance] = mvnrmle(Data, Design, 1, [], [], [], Covar0);
Estimate Standard Errors
StdParameters = mvnrstd(Data, Design, Covariance);
Estimate Parameters
[Parameters, Covariance] = ecmlsrmle(Data, Design, [], [], [], [], Covar0);
Estimate Standard Errors
StdParameters = ecmmvnrstd(Data, Design, Covariance);
An ad hoc form of least squares that has surprisingly good properties for misspecified or nonnormal models is known as feasible generalized least squares, or FGLS. The basic procedure is to do least-squares regression and then to do covariance-weighted least-squares regression with the resultant residual covariance from the first regression.
Estimate Parameters
[Parameters, Covariance] = mvnrmle(Data, Design, 2, 0, 0);
or (to illustrate the FGLS process explicitly)
[Parameters, Covar0] = mvnrmle(Data, Design, 1); [Parameters, Covariance] = mvnrmle(Data, Design, 1, [], [], [], Covar0);
Estimate Standard Errors
StdParameters = mvnrstd(Data, Design, Covariance);
Estimate Parameters
[Parameters, Covar0] = ecmlsrmle(Data, Design); [Parameters, Covariance] = ecmlsrmle(Data, Design, [], [], [], [], Covar0);
Estimate Standard Errors
StdParameters = ecmmvnrstd(Data, Design, Covariance);
Given a multivariate normal regression model in standard form
with a Data
matrix and a Design
array,
it is possible to convert the problem into a seemingly unrelated regression
(SUR) problem by a simple transformation of the Design
array.
The main idea of SUR is that instead of having a common parameter
vector over all data series, you have a separate parameter vector
associated with each separate series or with distinct groups of series
that, nevertheless, share a common residual covariance. It is this
ability to aggregate and disaggregate series and to perform comparative
tests on each design that is the power of SUR.
To make the transformation, use the function convert2sur
, which converts a standard-form
design array into an equivalent design array to do SUR with a specified
mapping of the series into NUMGROUPS
groups. The
regression functions are used in the usual manner, but with the SUR
design array instead of the original design array. Instead of having NUMPARAMS
elements,
the SUR output parameter vector has NUMGROUPS
of
stacked parameter estimates, where the first NUMPARAMS
elements
of Parameters
contain parameter estimates associated
with the first group of series, the next NUMPARAMS
elements
of Parameters
contain parameter estimates associated
with the second group of series, and so on. If the model has only
one series, for example, NUMSERIES
= 1, then the
SUR design array is the same as the original design array since SUR
requires two or more series to generate distinct parameter estimates.
Given NUMPARAMS
parameters and NUMGROUPS
groups
with a parameter vector (Parameters
) with NUMGROUPS
* NUMPARAMS
elements from any of the regression routines,
the following MATLAB® code fragment shows how to print a table
of SUR parameter estimates with rows that correspond to each parameter
and columns that correspond to each group or series:
fprintf(1,'Seemingly Unrelated Regression Parameter Estimates\n'); fprintf(1,' %7s ',' '); fprintf(1,' Group(%3d) ',1:NumGroups); fprintf(1,'\n'); for i = 1:NumParams fprintf(1,' %7d ',i); ii = i; for j = 1:NumGroups fprintf(1,'%12g ',Param(ii)); ii = ii + NumParams; end fprintf(1,'\n'); end fprintf(1,'\n');
Form an SUR Design
DesignSUR = convert2sur(Design, Group);
Estimate Parameters
[Parameters, Covariance] = mvnrmle(Data, DesignSUR);
Estimate Standard Errors
StdParameters = mvnrstd(Data, DesignSUR, Covariance);
Form a SUR Design
DesignSUR = convert2sur(Design, Group);
Estimate Parameters
[Parameters, Covariance] = ecmmvnrmle(Data, DesignSUR);
Estimate Standard Errors
StdParameters = ecmmvnrstd(Data, DesignSUR, Covariance);
Without missing data, you can estimate the mean of your Data
with
the function mean
and the covariance with the function cov
.
Nevertheless, the function ecmnmle
does this for
you if it detects an absence of missing values. Otherwise, it uses
the ECM algorithm to handle missing values.
Estimate Parameters
[Mean, Covariance] = ecmnmle(Data);
Estimate Standard Errors
StdMean = ecmnstd(Data, Mean, Covariance);
convert2sur
| ecmlsrmle
| ecmlsrobj
| ecmmvnrfish
| ecmmvnrfish
| ecmmvnrmle
| ecmmvnrobj
| ecmmvnrstd
| ecmmvnrstd
| ecmnfish
| ecmnhess
| ecmninit
| ecmnmle
| ecmnobj
| ecmnstd
| mvnrfish
| mvnrmle
| mvnrobj
| mvnrstd