MATLAB Neural Network Training: Crazy Validation Output

In order to prevent Neural Network from overfitting, I have divided the training data into two parts: Training and Validation Set (see code below):

net.divideFcn= 'divideind'; % divide the data manually
net.divideParam.trainInd= 1:99; % training data indices 
net.divideParam.valInd= 2:100; % validation data indices 
net.divideParam.testInd= 1:1;  % testing data indices

So, the training and validation set have 99 data values (among them 98 data values are common).

However, the MATLAB Neural Network training algorithm is showing a huge performance difference in training and validation set (see image below):

In addition, this is happening always after a very few epoch (no matter how the training and validation set data are divided).

Can anybody explain why is this happening and how to solve/fix this problem?

The complete code is given below:

% Neural Network Design
net = feedforwardnet([20 40 20]);

% hidden layer transfer function (data distributed within: -1,+1) 
net.layers{1}.transferFcn = 'tansig';
net.layers{2}.transferFcn = 'tansig';
net.layers{3}.transferFcn = 'tansig';
net.layers{4}.transferFcn = 'purelin';
net.biasConnect(1) = 0; // no bias connection needed
net.biasConnect(2) = 0;
net.biasConnect(3) = 0;
net.biasConnect(4) = 0;

% network training parameters
net.trainFcn = 'trainlm';
net.performFcn = 'mse';
net.trainParam.epochs = 300;
net.trainParam.max_fail = 10;
net.trainParam.min_grad = 1e-5;
net.trainParam.mu = .001;
net.trainParam.mu_dec = 0.1;
net.trainParam.mu_inc = 10;

%training and test set data
net.divideFcn= 'divideind'; % divide the data manually
net.divideParam.trainInd= 1:99; % training data indices 
net.divideParam.valInd= 2:100; % validation data indices 
net.divideParam.testInd= 1:1;  % testing data indices 

% configure and initialize Neural Network
net = configure(net, Input, Output);
net = init(net);
view(net);

% Neural Network training
[net, tr] = train(net, Input, Output);
plotperf(tr)

Answer 1

[Greg Heath]

I'm only using upper case for highlighting; always use lower case for coding.

You are making an easy problem hard. To prevent overfitting you do not have to do anything beyond what is in the documentation examples. In particular, it makes no sense to use to the same data for training and validation. The validation results are used to stop training when the net stops generalizing. Using the same data for both training and validation defeats the purpose.

Before attacking your problem directly, understand how the default approaches work. For example

 1. Use FITNET for regression and curve-fitting. It automatically calls FEEDFORWARDNET.

 2. Use PATTERNNET for classification and pattern-recognition. It also automatically 
    calls FEEDFORWARDNET.

3. Examples using default parameters are given by the HELP and DOC commands. Sometimes the examples are different so check out both. For example:

 help fitnet  (also doc fitnet)

 doc patternnet (also help patternnet)

4. After running the help and doc examples, substitute your own data into the simple code. I like to run it 10 or 20 times using different random number seeds so that initial weights are different for each run.

5. After trying to understand the results, change some of the default parameters (I tend to change them one at a time so I can understand the effect of each).

6. I have posted zillions of examples in both the NEWSGROUP and ANSWERS. Just search using, for example,

 greg fitnet tutorial

Hope this helps.

Thank you for formally accepting my answer

Greg

[Greg Heath]

Sorry, my computer crashed and I lost MATLAB. Haven't had time to get a replacement.

Greg