Cross-validate regularization of linear discriminant
err = cvshrink(obj)
[err,gamma] = cvshrink(obj)
[err,gamma,delta] = cvshrink(obj)
[err,gamma,delta,numpred] = cvshrink(obj)
[err,...] = cvshrink(obj,Name,Value)
Discriminant analysis classifier, produced using
Specify optional pairs of arguments as
the argument name and
Value is the corresponding value.
Name-value arguments must appear after other arguments, but the order of the
pairs does not matter.
Before R2021a, use commas to separate each name and value, and enclose
Name in quotes.
Vector of Gamma values for cross-validation.
Number of Delta intervals for cross-validation. For every value
Number of Gamma intervals for cross-validation.
Verbosity level, an integer from
Numeric vector or matrix of errors.
Vector of Gamma values used for regularization. See Gamma and Delta.
Vector or matrix of Delta values used for regularization. See Gamma and Delta.
Numeric vector or matrix containing the number of predictors
in the model at various regularizations.
Regularize Data with Many Predictors
Regularize a discriminant analysis classifier, and view the tradeoff between the number of predictors in the model and the classification accuracy.
Create a linear discriminant analysis classifier for the
ovariancancer data. Set the
FillCoeffs options to keep the resulting model reasonably small.
load ovariancancer obj = fitcdiscr(obs,grp,... 'SaveMemory','on','FillCoeffs','off');
Use 10 levels of
Gamma and 10 levels of
Delta to search for good parameters. This search is time-consuming. Set
1 to view the progress.
rng('default') % for reproducibility [err,gamma,delta,numpred] = cvshrink(obj,... 'NumGamma',9,'NumDelta',9,'Verbose',1);
Done building cross-validated model. Processing Gamma step 1 out of 10. Processing Gamma step 2 out of 10. Processing Gamma step 3 out of 10. Processing Gamma step 4 out of 10. Processing Gamma step 5 out of 10. Processing Gamma step 6 out of 10. Processing Gamma step 7 out of 10. Processing Gamma step 8 out of 10. Processing Gamma step 9 out of 10. Processing Gamma step 10 out of 10.
Plot the classification error rate against the number of predictors.
plot(err,numpred,'k.') xlabel('Error rate'); ylabel('Number of predictors');
Gamma and Delta
Regularization is the process of finding a small set of predictors
that yield an effective predictive model. For linear discriminant
analysis, there are two parameters, γ and δ,
that control regularization as follows.
you select appropriate values of the parameters.
Let Σ represent the covariance matrix of the data X, and let be the centered data (the data X minus the mean by class). Define
The regularized covariance matrix is
Whenever γ ≥
MinGamma, is nonsingular.
Let μk be the mean vector for those elements of X in class k, and let μ0 be the global mean vector (the mean of the rows of X). Let C be the correlation matrix of the data X, and let be the regularized correlation matrix:
where I is the identity matrix.
The linear term in the regularized discriminant analysis classifier for a data point x is
The parameter δ enters into this equation as a threshold on the final term in square brackets. Each component of the vector is set to zero if it is smaller in magnitude than the threshold δ. Therefore, for class k, if component j is thresholded to zero, component j of x does not enter into the evaluation of the posterior probability.
DeltaPredictor property is a vector related
to this threshold. When δ ≥
DeltaPredictor(i), all classes k have
Therefore, when δ ≥
DeltaPredictor(i), the regularized
classifier does not use predictor
numpredoutputs to see the tradeoff between cross-validated error and number of predictors. When you find a satisfactory point, set the corresponding
deltaproperties in the model using dot notation. For example, if
(i,j)is the location of the satisfactory point, set
obj.Gamma = gamma(i); obj.Delta = delta(i,j);