Does the type of classifier make that much difference in feature selection in sequentialfs function?

Hi,

I am wondering if does it make significant difference in feature selcetion if we are using difference classifiers during feature selection?

For example, I am using the following naive bayes classifier:

fun = @(Xtrain,Ytrain,Xtest,Ytest)...
sum(Ytest~=predict(fitcnb(Xtrain,Ytrain,'Distribution','kernel'),Xtest));
[inmodel,history] = sequentialfs(fun,myData,classes,'cv',c2,'options',opts,'nfeatures',60);

or if I use fitsvm, instead of fitcnb, will make that much difference on the selceted features?

Thanks

Answer 1

[Don Mathis]

I think in general it will make a difference which classifier you use, because different classifiers deterct different kinds of patterns. An SVM could discover that a pair of uncorrelated predictors are both relevant because it can learn nonlinear I/O relationships, while a linear classifier might not be able to detect that nonlinear structure, and then "decide" that both predictors are irrelevant.

[Sara Salimi]

Thank you very much for your help,

So, can I come to this conclusion that for finding the relevant features, it would be better to use SVM?

[Don Mathis]

I don't think there's any right answer here. You could even try both and take the union of the features. The features you get will also depend on the hyperparameters you choose for your SVM or NB models.