Classification Learner App Performance Reporting

Can anyone confirm whether or not the Classification Learner App uses Mean Square Error for performance?

My project has me comparing classification performances using Neural Networks (via patternnet) and Support Vector Machines (via classification learner app).

The documentation for patternnet says the performance accuracies are given in Mean Square Error (MSE). This is great that it is in the documentation.

On the other hand, the performance of the SVMs do not specificy whether they are MSE or Root Mean Square Error or something else. I would like to compare the performance of the models and I cannot do this if I cannot ensure they both are using MSE. I would like to assume that all the classification learner performances are given in MSE (as that might be the default).

Which leads me to my question: can anyone confirm whether or not the Classification Learner App uses Mean Square Error for performance?

[Sahil Jain]

"the performance of the SVMs do not specificy whether they are MSE or Root Mean Square Error or something else". From where in the Classification Learner App are you checking the performance metric?

Answer 1

[Sahil Jain]

Hi Luke. After training in the Classification Learner App, the "Current Model Summary" tab contains a section called "Training Results". This section has two performance metrics - "Accuracy" and "Total cost". Here, "Total Cost" refers to the misclassification cost and not mean squared error. To calculate mean squared error, export the model to the workspace and predict the outputs, then use the "immse" function to calculate the mean squared error. The steps for exporting the model and predicting outputs can be found on the Export Classification Model to Predict New Data documentation page.