neural network performance analysis

Hello I have trained a neural network using house_dataset in matlab. When I trained the network through the following code the tr.best_tperf was reasonable: [inputs,targets] = house_dataset; hiddenLayerSize = 10; net = fitnet(hiddenLayerSize); net.divideParam.trainRatio = 70/100; net.divideParam.valRatio = 15/100; net.divideParam.testRatio = 15/100; [net,tr] = train(net,inputs,targets); outputs = net(inputs); performance = perform(net,targets,outputs)

however, when we extract the test dataset and consider the performance using these dataset the performance was unacceptable: tInd = tr.testInd; tstOutputs = net(inputs(tInd)); tstPerform = perform(net,targets(tInd),tstOutputs)

Could you please kindly explain the reason for this. Should we use another data division algorithm? best

[Greg Heath]

As you can see from my second answer, you didn't use (:,tInd) for the input indices.

Please accept the 2nd answer.

Greg

Answer 1

[Greg Heath]

result =

    ntrial    R2trn     R2val     R2tst
    1.0000    0.8610    0.8746    0.8218
    2.0000    0.8532    0.8778    0.8648
    3.0000    0.8607    0.8503    0.8547
    4.0000    0.8464    0.8942    0.8585
    5.0000    0.8944    0.8883    0.8843
    6.0000    0.8958    0.8423    0.9184
    7.0000    0.9085    0.7988    0.9078
    8.0000    0.8906    0.8690    0.9186
    9.0000    0.8978    0.8637    0.9001
   10.0000    0.8810    0.9125    0.9267

 >> ytst = net(inputs(:,tr.testInd));

 R2tstx = 1 - mse(ttst-ytst)/MSEtst00

 R2tstx = 
           0.9267

 Rtstx = sqrt(R2tstx) 

 Rtstx =

           0.9627

How did you get 0.999?

Answer 2

[Greg Heath]

Initialize the random number generator at the beginning. So we can compare, use

rng(0).

Also make 10 designs in a loop over random weight initializations obtained by using

net = configure(net,x,t);

[net tr ] = ...

Hope this helps,

Thank you for formally accepting my answer

Greg

Answer 3

[Mohamad]

Hello Greg Many thanks for your kind help.