How to check the robustness of the Neural network model?
4 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
Le Truong An
el 13 de Jul. de 2019
Comentada: Le Truong An
el 14 de Jul. de 2019
Hi everyone,
I need your help for my project.
- I used Neural Pattern Recognition app, and have already built an NN model for classification with 4 labels. The NN model worked very well. Accuracy classification reaches more than 90%.
- However, when I want to check this model with new data (new data = the original data through an AWGN channel having a 10 dB signal-to-noise ratio (SNR). The classification result is always less than 30% accuracy.
- How to create an augmented dataset and train it?
- How to check the robustness of the Neural network model?
Please help me!!!
Code to create a neural network:
% Solve a Pattern Recognition Problem with a Neural Network
% Script generated by Neural Pattern Recognition app
%% Load data
% X - input data.
% Y - target data.
load('mydata.mat')
x = X;
t = Y;
%% Choose a Training Function
trainFcn = 'trainscg'; % Scaled conjugate gradient backpropagation.
%% Create a Pattern Recognition Network
hiddenLayerSize = 50;
net = patternnet(hiddenLayerSize, trainFcn);
% Choose Input and Output Pre/Post-Processing Functions
net.input.processFcns = {'removeconstantrows','mapminmax'};
net.output.processFcns = {'removeconstantrows','mapminmax'};
% Setup Division of Data for Training, Validation, Testing
net.divideFcn = 'dividerand'; % Divide data randomly
net.divideMode = 'sample'; % Divide up every sample
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;
% Choose a Performance Function
net.performFcn = 'crossentropy'; % Cross-Entropy
% Choose Plot Functions
net.plotFcns = {'plotperform','plottrainstate','ploterrhist', ...
'plotconfusion', 'plotroc'};
%% Train the Network
[net,tr] = train(net,x,t);
%% Test the Network
y = net(x);
e = gsubtract(t,y);
performance = perform(net,t,y)
tind = vec2ind(t);
yind = vec2ind(y);
percentErrors = sum(tind ~= yind)/numel(tind);
%% Recalculate Training, Validation and Test Performance
trainTargets = t .* tr.trainMask{1};
valTargets = t .* tr.valMask{1};
testTargets = t .* tr.testMask{1};
trainPerformance = perform(net,trainTargets,y)
valPerformance = perform(net,valTargets,y)
testPerformance = perform(net,testTargets,y)
%% View the Network and plots confusion matrix
view(net)
figure, plotconfusion(t,y)
save ('mynet');
Code to check neural network with data added white noise:
clc;
clear all;
%load
load('mynet');
%% add Gaussian white noise SNR=10dB
load('mydata.mat')
X10dB = awgn(X,10,'measured');
x = X10dB; % train data
t = Y; % target
%% Test the Network
y = net(x);
e = gsubtract(t,y);
performance = perform(net,t,y)
tind = vec2ind(t);
yind = vec2ind(y);
percentErrors = sum(tind ~= yind)/numel(tind);
% Recalculate Training, Validation and Test Performance
trainTargets = t .* tr.trainMask{1};
valTargets = t .* tr.valMask{1};
testTargets = t .* tr.testMask{1};
trainPerformance = perform(net,trainTargets,y)
valPerformance = perform(net,valTargets,y)
testPerformance = perform(net,testTargets,y)
%% Plots
figure, plotconfusion(t,y)
0 comentarios
Respuesta aceptada
Greg Heath
el 14 de Jul. de 2019
Editada: Greg Heath
el 14 de Jul. de 2019
If you are going to test with white noise, include white noise in your design (i.e., training + validation)
Then, given a fixed input level of white noise for design (i.e., design + noise1) you can obtain individual performance measures of training, validation and test as a function of added noise level.
Hope this helps,
Thank you for formally accepting my answer
Greg
Más respuestas (0)
Ver también
Categorías
Más información sobre Pattern Recognition and Classification en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!