How to evaluate the performance metrics Accuracy, precision, recall, f1Score on the tuned FIS?
55 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
In the example contained in the Fuzzy logic user guide documentation by mathworks, Tune Fuzzy Inference System at the Command Line, page 225, is it possible to use the confusion matrix to calculate Accuracy, precision, recall, f1Score? if not possible, how can I evaluate it using accuracy, specificity and sensitivity? I tried so many times to write a function that will calculate those performance metrics. Below is the sample of the Tune Fuzzy Inference System at the Command Line in page 225 of Fuzzy logic user guide documentation by mathworks. I tried something like this:
[data,name] = loadGasData;
X = data(:,1:6);
Y = data(:,7);
trnX = X(1:2:end,:); % Training input data set
trnY = Y(1:2:end,:); % Training output data set
vldX = X(2:2:end,:); % Validation input data set
vldY = Y(2:2:end,:); % Validation output data set
dataRange = [min(data)' max(data)'];
fisin = mamfis;
for i = 1:6
fisin = addInput(fisin,dataRange(i,:),'Name',name(i),'NumMFs',2);
end
fisin = addOutput(fisin,dataRange(7,:),'Name',name(7),'NumMFs',64);
figure
plotfis(fisin)
options = tunefisOptions('Method','particleswarm',...
'OptimizationType','learning', ...
'NumMaxRules',64);
options.MethodOptions.MaxIterations = 20;
rng('default')
runtunefis = false;
%
if runtunefis
fisout1 = tunefis(fisin,[],trnX,trnY,options); %#ok
else
tunedfis = load('tunedfismpgprediction.mat');
fisout1 = tunedfis.fisout1;
fprintf('Training RMSE = %.3f MPG\n',calculateRMSE(fisout1,trnX,trnY));
%% this is where I want to call and print the calculateMetrics function.
end
plotfis(fisout1)
I tried something like this:
function [accuracy, precision, recall, f1Score] = calculateMetrics(confmat)
%%confmat
% Extract TP, TN, FP, FN from the confusion matrix
TP = confmat(2, 2);
TN = confmat(1, 1);
FP = confmat(1, 2);
FN = confmat(2, 1);
% Calculate metrics
accuracy = (TP + TN) / (TP + TN + FP + FN);
precision = TP / (TP + FP);
recall = TP / (TP + FN);
f1Score = 2 * (precision * recall) / (precision + recall);
end
2 comentarios
Respuestas (1)
Sam Chak
el 4 de Dic. de 2023
Hi @Ahmad
Most special performance metrics are user-defined, and only designers understand what they really mean. A simple test below shows that your performance metrics work in the sense that you coded them, as long as you supply a proper Confusion Matrix as the argument of the function.
Since the metric function works, the next question is how exactly you produce the Confusion Matrix (and plot it if you want)? I'm not an expert, but a simple search shows that you can produce the matrix using either the confusionmat() or evaluate() functions.
%% a Matrix
confmat = magic(4)
%% Call calculateMetrics() function to print accuracy, precision, recall, f1Score
[accuracy, precision, recall, f1Score] = calculateMetrics(confmat)
function [accuracy, precision, recall, f1Score] = calculateMetrics(confmat)
%% confmat
% Extract TP, TN, FP, FN from the confusion matrix
TP = confmat(2, 2);
TN = confmat(1, 1);
FP = confmat(1, 2);
FN = confmat(2, 1);
% Calculate metrics
accuracy = (TP + TN) / (TP + TN + FP + FN);
precision = TP / (TP + FP);
recall = TP / (TP + FN);
f1Score = 2*(precision*recall)/(precision + recall);
end
0 comentarios
Ver también
Categorías
Más información sobre Fuzzy Logic Toolbox en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!