Linear Discriminant analysis WITH variable selection

Does the toolbox in MATLAB allow you to do variable selection in a discriminant analysis? I have read the documentation and can not see anywhere where this is stated. i have also checked online to see if it has been asked before. No luck there either. this seems like it should be a standard part of the toolbox.

Am i missing something? i have v 2012b

thanks for any help

Answer 1

[Ilya]

Here is an example in the doc: http://www.mathworks.com/help/stats/discriminant-analysis.html#btaf5dv It shows how to use the DeltaPredictor property for selecting predictors.

Another approach is the sequentialfs function. It can work with any classifier, but it needs a hand-crafted function handle.

[Andy]

i have read about regularization and how you can select variables in this way, but this is not directly related to variable selection. Other stats packages have a stepwise approach built in, like SAS, R, SPSS. This is the classic method. Can not believe MATLAB doesn't have this! This regularization approach by GUO et al. was designed for microarray data and compared to another approach (PAM). The "variable selection" part is not the focus and is not really variable selection in the classical sense.

[Ilya]

The doc piece I pointed at shows how to find the optimal regularization and threshold on the coefficient magnitude by cross-validation. If you don't want to regularize, don't. Use the same workflow to find the optimal threshold on the coefficient magnitude. Makes your life easier since you need to optimize one variable instead of two. This approach to selecting predictors for LDA is well known and was used long before the paper by Guo.

Stepwise selection is available from LinearModel.stepwise and GeneralizedLinearModel.stepwise. There, "stepwise" means that the coefficients are refitted at every step and removed or added after comparing their p-values with some known threshold. I don't know what you mean by "stepwise" for LDA because computing p-values for predictors in a classification model is far less popular (and most certainly not "classic" in any sense). I am hardly an expert on SAS or SPSS, but as far as R goes - there is, to my knowledge, only one package that supports a "stepwise" procedure for LDA. That's SDDA. That package appears to provide the diagonal discriminant (one in which predictor correlations are ignored) and supports forward selection available from sequentialfs.