How to choose number of hidden layers

88 views (last 30 days)
Hi, I want to design a neural network with 3 input and 1 output. Samples are 21 Millions. For predictions and I don't know how many hidden layers and also the network parameters to use to get best results

Accepted Answer

Walter Roberson
Walter Roberson on 25 Jun 2017
When you call patternnet, pass it a vector of the sizes of the hidden layers.
  2 Comments
Greg Heath
Greg Heath on 26 Jun 2017
Walter's comment is valid for REGRESSION, CLASSIFICAITON, TIME-SERIES, etc
If you add FITNET to my suggested search you will see how to implement the double loop approach to minimize the number of hidden nodes for REGRESSION
ANSWERS
Hmin:dH:Hmax Ntrials FITNET ==> 37 HITS
Hmin:dH:Hmax Ntrials PATTERNNET ==> 28 HITS
the NEWSGROUP doesn't yield much with that detail
NEWSGROUP
Hmin Hmax Ntrials FITNET ==> 12 HITS
Hmin Hmax Ntrials PATTERNNET ==> 4 HITS
Hmax Ntrials FITNET ==> 12 HITS
Hmax Ntrials PATTERNNET ==> 4 HITs
Hmax FITNET ==> 14 HITS
Hmax PATTERNNET ==> 4 HITS
Ntrials FITNET ==> 27 HITS
Ntrials PATTERNNET ==> 15 HITS
Hope this helps.
Greg

Sign in to comment.

More Answers (1)

Greg Heath
Greg Heath on 25 Jun 2017
For 3 inputs and 1 output you only need 1 hidden layer. Minimize the number of hidden nodes subject to the maximum training error constraint.
mse(errortrn) <= 0.01*mean(var(targettrn',1))
then the training error Rsquare is 99%.
I use a double loop approach over
1. Number of hidden nodes h = Hmin:dH:Hmax
2. j = 1:Ntrials number of random assignments for initial
weights and trn/val/tst data divisions.
3. [ I N ] = size(input)% [ 3 N ]
[ O N ] = size(target)% [ 1 N ]
Ntrn ~ 0.7*N % default
Ntrneq = Ntrn*O % No of training equations
Nw = (I+1)*H+(H+1)*O = 5*H + 1 % No. of unknown weights
4. A reasonable choice to prevent overtraining an overfit net is
Ntrneq >= 10*Nw ==> Ntrn >~ 50*H
5. Obviously, a major task is to find a small ( << 21 millions!)
subset of data that can adequately represent the 21 million 3-dimensional
inputs. My experience is that for d-dimensional Gaussian distributions
Ntst >= 30*d random picks is usually sufficient.
Taking Nval = Ntst ~ 30*d and
Ntrn ~ (7/3)*(Nval+Ntst) ~ 140*d
Then, N ~ Ntrn + Nval + Ntst ~ 200*d = 600
6. There are zillions of examples of my double loop minimum H approach in
both the NEWSGROUP and ANSWERS. Just search on
Hmin:dH:Hmax Ntrials
Hope this helps.
Thank you for formally accepting my answer
Greg
  2 Comments
Walter Roberson
Walter Roberson on 26 Jun 2017
The GUI tool is not always flexible enough. You can have the GUI tool create a network with the default number of hidden layers, and then you can tell it to generate the code for the network. You can then edit the code for the network so that it initializes the sizes of the hidden layers the way you want.

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by