Predictions Validity using FitNet

Similar to what I am trying to do with a NarxNet, I am also using a Fitnet to see how far I can have a prediction with a certain maximum error. The script I wrote loads a file with data with over 3500 points, from here I start training the network using only the corresponding to about 4 years of data and do about 10 trials. Then I test the network using values outside the training sample to compare its output with the targets I have.

After running the 10 trials, the training sample is increased by 1 year and trained again. This is done until near the end of my data (about 42 years so 42 cycles).

From my experiments with the sinus function, I don't expect this model to give very good results, but it is required of me to do this.

My question is if there are any suggestions on how I can improve my code in order to reduce overfit and overtraining with too many neurons or layers, etc.

NOTE: My code saves every trial data in one file and in the end of the X trials it saves the prediction errors in another file. This is done in case I require access to any of the random networks later for testing. The ERROR file allows easy verification of the network results for each loop and all trials.

Thanks in advance!

My code is as follows:

%%Creation of the network
% One year = 73 points (data has a step of 5 days)

trials = 10;            % Number of trials
hiddenLayerSize = 6;    % Number of neurons

endyear = 42;         % FOR Loop counter - each loop will add 1 year for training
pred_period = 5;      % Number of years to predict (must be a multiple of 5 due to the step of the data)

data_start = 38034;                  % first MJD of the list
data_finish = 39489;                 % target MJD of the list - 4 years

S = 1;                               % index of the first date  in the file
N = (data_finish-data_start)/5+1;    % index of the target date in the file

prediction_size = pred_period*73;       % input size 

%%The Network
%net = fitnet(hiddenLayerSize);                     % 1 Layer
net = fitnet([hiddenLayerSize,hiddenLayerSize]);    % 2 Layers

%%Begin Cycle for Training and Prediction of the Network

for j = 1:1:endyear

      table_errors = zeros(trials,prediction_size);
      for i = 1:1:trials
          %%Identifiers for control purposes only in the saved files
          identifier_trial = i;   % identifies the trial 
          identifier_cycle = j;   % identifies the cycle

          %%Network Parameters

          %net.trainParam.epochs = 10000;  -------> this doesnt work, why?
          %net.trainParam.goal = 1e-5;
          %net.trainParam.lr = 0.75;
   %----------BEGIN SNIP ---------       

        %Default values( dividerand, mse, trainlm)

   %----------END SNIP------------

          %%Training the Network

          % Calls the input and targets for training    
          train_inputs = x(S:N)';
          train_targets = y(S:N)';

          % Train the Network
          [net,tr] = train(net,train_inputs,train_targets);
          train_outputs = net(train_inputs);
          train_errors = gsubtract(train_targets,train_outputs);

          %%Test the Network - Prediction of the next points
          % Input and targets
          pred_inputs = x(N+1:N+prediction_size)';
          pred_targets = y(N+1:N+prediction_size)';

          % Results
          pred_outputs = net(pred_inputs);
          %pred_errors = gsubtract(pred_targets,pred_outputs);

          table_errors(i,:) = gsubtract(pred_targets,pred_outputs);

          %%Saves  all the network data from the workspace
          % This allows us to load any particular network that had good
          % results
          save(['cycle' int2str(j) int2str(i)]);     

      end
      %%Saves all the errors of a a cycle 
      save(['errors' int2str(j)], 'table_errors');
      %save(['errors' int2str(j)], 'table_errors','table_rms');

      %%Adding data to training
      N = N+73;              % adds one more year of training data

end

[Greg Heath]

What are the dimensions of x and y?

What is the relationship between x and y?