Uniform class probabilities vs. Empirical class probabilities

Hi;

I found on one Matlab example of Uniform class probabilities and Empirical class probabilities.

Empirical class probabilities is calculated as follows:

svmStruct = fitcsvm(X,Y); % X is training data and Y are classes
 %%10-fold cross-validation
    cvm = crossval(svmStruct);    

       %%Accuracy on cross-validated data
      [yhatcv,S] = kfoldPredict(cvm);

       % cross-validated error with empirical class probabilities

      empirical_error=mean(Y~=yhatcv)

Uniform class probabilities is calculated as follows:

% cross-validated error with uniform class probabilities
   uniform_error=kfoldLoss(cvm)

Could you pleas give me a formal definition of those 2 errors types?

Answer 1

[Ilya]

If you are still looking for an answer, there is only one definition for error. In each case, you form a confusion matrix and then take a weighted sum of off-diagonal elements. This code snippet should explain it:

load ionosphere
prior = [1 3]'/4;
m = fitcsvm(X,Y,'prior',prior,'kfold',5,'stand',1);
Yhat = m.kfoldPredict;
C = confusionmat(Y,Yhat,'order',m.ClassNames)
Coff = C;
Coff(1:3:end) = 0
sum(sum(Coff,2).*prior./sum(C,2))
m.kfoldLoss

[Diver]

@Ilya Thank you .. I think in this case I better use kfoldLoss error.