How to get validation test and training errors of a neural network?

I have created and trained a neural network using the following code .I want to know how to get the training testing and validation errors/mis-classifications the way we get using the matlab GUI.

trainFcn = 'trainscg';  % Scaled conjugate gradient backpropagation.

   % Create a Pattern Recognition Network
   hiddenLayerSize = 25;
   net = patternnet(hiddenLayerSize);

% Setup Division of Data for Training, Validation, Testing
net.divideParam.trainRatio = trainper/100;
net.divideParam.valRatio = valper/100;
net.divideParam.testRatio = testper/100; 

% Train the Network
[net,tr] = train(net,x,t);

Answer 1

[Greg Heath]

BOTH documentation commands

 help patternnet
and
 doc patternnet

have the following sample code for CLASSIFICATION & PATTERN-RECOGNITION:

    [x,t] = iris_dataset;
     net  = patternnet(10);
     net  = train(net,x,t);
     view(net)
     y = net(x);
     perf = perform(net,t,y);
     classes = vec2ind(y);

However, the following are missing

 1. Dimensions of x and t 
 2. Plots of x, t, and t vs x
 3. Minimum possible number of hidden nodes
 4. Initial state of the RNG (Needed for duplication)
 5. Training record, tr
 6. Plots of e = y-t vs x
 7. Misclassified cases
 8. trn/val/tst Error rates

For details see my NEWSGROUP posts

 SIZES OF MATLAB CLASSIFICATION EXAMPLE DATA SETS
http://www.mathworks.com/matlabcentral/newsreader/...
       view_thread/339984

 and

 BEYOND THE HELP/DOC DOCUMENTATION : PATTERNNET for 
 NN Classification and PatternRecognition
 http://www.mathworks.com/matlabcentral/newsreader/...
     view_thread/344832

Hope this helps.

Thank you for formally accepting my answer

Greg

[Anton Tichenko]

Hi Greg,

Noticed in a few answers here you are referring to "THE HELP/DOC DOCUMENTATION : PATTERNNET for NN Classification and PatternRecognition" and "SIZES OF MATLAB CLASSIFICATION EXAMPLE DATA SETS" posts

I can't find them anywhere, have they been removed? Any chance you could share again?

Thanks!

[Greg Heath]

help and doc are to be used in the MATLAB command line. For example

>> help patternnet

patternnet Pattern recognition neural network.

...

Hope this helps

Greg

Answer 2

[Greg Heath]

 clear all, clc
  [ x, t ] = simplefit_dataset;
  net      = fitnet;
  rng('default')      % For reproducibility
  [ net tr y e ]   =  train( net, x, t );
  %  y = net(x); e = t - y;
   tr = tr % No semicolon:  LOOK AT ALL OF THE GOODIES!!!

 msetrn = tr.best_perf
 mseval = tr.best_vperf
 msetst = tr.best_tperf

Hope this helps

Thank you for formally accepting my answer

Greg

[Newman]

Dear Sir these are the mean square errors of my 3 sets how to see the percentages of mis classifications, instead of just errors?

msetrn =

   5.7989e-07

mseval =

    0.0021

msetst =

    0.0028

[Greg Heath]

SORRY!!!

I didn't notice that your problem is classification instead of regression.

Will post more after I go upstairs and have a snack.

Greg

[Newman]

OK I basically want to get the percentage of misclassifications and also cross-entropy the way patternet GUI displays it

[Mohamed Nedal]

Dear @Greg,

Could you please elaporate on the difference between msetrn, mseval, and msetst?

If I want to assess the overall performance of the NN and want to find the MSE for the NN as a whole, Which one should I use?

[Mohamed Nedal]

If I wrote this:

rmse = sqrt(mean(e)); 

Does this give me the RMSE of the whole NN?