MATLAB Answers

How to calculate Akaike Information Criterion and BIC from a Neural Network?

104 views (last 30 days)
Javier Bastante
Javier Bastante on 13 Feb 2017
Answered: David Franco on 28 Aug 2017
I know "aic" function exists, but I don't know how to use it with fitting neural networks.
Any help?
Thank you in advance

Answers (2)

David Franco
David Franco on 28 Aug 2017
After training the network and simulating the outputs:
[net,tr] = train(net,inputs,targets);
output = sim(net,inputs);
Get the parameters and calculate de criterions (Sarle, 1995):
% Getting the training targets
trainTargets = gmultiply(targets,tr.trainMask);
SSE = sse(net,trainTargets,output); % Sum of Squared Errors for the training set
n = length(tr.trainInd); % Number of training cases
p = length(getwb(net)); % Number of parameters (weights and biases)
% Schwarz's Bayesian criterion (or BIC) (Schwarz, 1978)
SBC = n * log(SSE/n) + p * log(n)
% Akaike's information criterion (Akaike, 1969)
AIC = n * log(SSE/n) + 2 * p
% Corrected AIC (Hurvich and Tsai, 1989)
AICc = n * log(SSE/n) + (n + p) / (1 - (p + 2) / n)
References:
  • Akaike, H. (1969), "Fitting Autoregressive Models for Prediction". Annals of the Institute of Statistical Mathematics, 21, 243-247.
  • Hurvich, C.M., and Tsai, C.L. (1989), "Regression and time-series model selection in small samples". Biometrika, 76, 297-307.
  • Sarle, W.S. (1995), "Stopped Training and Other Remedies for Overfitting". Proceedings of the 27th Symposium on the Interface of Computing Science and Statistics, 352-360.
  • Schwarz, G. (1978), "Estimating the Dimension of a Model". Annals of Statistics, 6, 461-464.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by