How can I trust my result which is changing according to train,test and validation set choice in NeuralNetwork?

Hi, Now I'm using the NeuralNetwork Pattern recognition.

In my case, I have a 60 subjects x 700 features data, and it has 2 class labels.

Using dividerand Function, I separated the subjects with train 70%, test 15%, validation 15%

As the result, The accuracy always changes according to subject select in train, test and validation set.

How can I trust my result which is changing according to train,test and validation set choice in NeuralNetwork ?

As I know, model performance will be trust throught the divided train, test and validation set.

Then, I can used the max accuracy ?

of After the repeat, and using the average of results?

I need the help, please..!

thank you !

Answer 1

[Prince Kumar]

Hi,

This generally happens when you do not have a pre-defined train, validation and test set for training and testing the model. As you keep changing the data distribution between these sets, the result will vary accordingly. In such cases try to do a k-fold cross validation.

Please refer to the following documentations for more clarity

• cvpartition
• crossvalind

Hope this helps!