How to add more than one hidden layer?

I need to use feedforwardnet to classify the images and also have train the NN in 3 levels.

Is it possible to add 3 hidden layers to feedforwardnet?

Answer 1

[Vinod Sudheesh]

Yes, it is possible to create a "feedforward neural network" with three hidden layers using the "feedforwardnet" function. This can be achieved by passing a vector of hidden layer sizes as the argument to the "feedforwardnet" function.

>> net=feedforwardnet([10 11 12]);

>> view(net);

[Greg Heath]

Since 1 hidden layer can provide a universal approximator, I question your desire to use more than one:

Why?

[Han Xia]

Hey Greg,

The reason to have multiple hidden layers is to allow the neural network to emulate non-linear systems (n-layers correspond to nth power of X). I don't see why this can be approximated by 1 signle hidden layer, could you please elaberate a little bit more on this approximation?

Best regards,

Han

[Greg Heath]

There is nothing to elaborate. The following is basic ( See any elementary book on NNs ):

Given piecewise continuous inputs and targets, the MSE between output and target can be minimized using a SINGLE layer of sigmoids or Gaussians.

Obviously, the trick is to determine 1. how many nodes are needed and 2. the weights and biases to mimimize the mean squared error.

P.S. Sometimes multiple hidden layers are used to help explain the physical relationships between input and output.

INTERESTING OBSERVATION:

I have been designing NNs since ~1980 . However, it is only recently that I discovered that the minimum number of necessary hidden nodes is approximately the same as the number of local maxima in the target vs input plot.

Hope this helps

Greg

[ismail tepedag]

Hi Greg, I'm desgining a neural network to solve a quality control issue related to power folding mechanism production ( my thesis study of Industrial engineering) . Classification neural network I use. I tried many versions of neural network. AS far as I can see, small numbers and big numbers for hidden layers produces bad results but there is no math here. IF you acheived some experience based approximation, can you share with me an example ? I'm interested in. I was really wondering why not 2nd layer ? I think my problem is non-linear and need to use more than 1 layer. I acheived 95,9% success ratio with 1 layer but want to reach 99,5%.

Thanks.

[Greg Heath]

A linear function does not need a hidden layer.

A nonlinear function needs no more than one.

However some nonlinear functions are more conveniently represented by two or more hidden layers.

There is an inherent degree of approximation for bounded piecewise continuous functions. Trying to force a closer fit by adding higher order terms (e.g., adding additional hidden nodes )often leads to instability.

You can test the stability of different designs with a different no. of hidden nodes. by comparing their performance as increasing levels of noise are added to the input.

Hope this helps.

Thank you for formally accepting my answer

Greg

[Marco Pizzoli]

Hi Greg,

I am very curious about your observation on the minimum number of necessary hidden nodes. In this regard I have a question: what do you mean by target vs input plot? Because, I can imagine finding the local maxima of the time series of the target or input (taken separately), but not on the graph that considers them together. I apologize in advance for my stupid question.

Best regards,

Marco