Is there any automatic code/way to choose/optimize the neural networks parameters on MATLAB?

Dear experts,

I am an undergraduate student and i am doing my bachelor thesis in classification using neural networks.

I am struggling in optimizing neural networks hyper parameters (example: learning rate, regularization parameter, number of hidden neurons, mini-batch sizes, which training algorithm, weights/biases, mu(for Jacobin methods)....etc.). I read different books, where they provide some rules of thumb to choose the parameters.

However, my supervisor mentioned that there is an automatic way/code to choose the hyper parameters on matlab and then matlab runs with the parameters which fits the most and then it trains the data.

Is there any possibility to mention where can i find such an automated way or mention it in the answer? (i have looked everywhere in the Q&A asked about neural networks, and the documentation it is really hard to find such method so i decided to ask the experts themselves) Thanks in advance

Answer 1

[David Ding]

Hi Yehia,

Perhaps the following on model building and assessment might be of use to you:

https://www.mathworks.com/help/stats/model-building-and-assessment.html

Thanks,

David

[Yehia Elbahrawy]

Thank you for your answer, appreciate it.

Answer 2

[Greg Heath]

Search the NEWSGROUP and ANSWERS with

 greg patternnet

Most of my designs accept all defaults except

 1. No of hidden nodes

 2. Initial random weights

and use a double for loop to obtain tens of candidate designs that yield

 mse(target-output) <= 0.01*mean(var(target,1))

The resulting classification error rates are usually very acceptable. In rare cases, I have to reduce 0.01 to 0.005 or lower.

Hope this helps.

Thank you for formally accepting my answer

Greg

Answer 3

[Alan Weiss]

In addition to Greg's comment (and he is the source of most wisdom on this matter), perhaps this example will show you the technique your advisor is referring to (requires Statistics and Machine Learning Toolbox).

Alan Weiss

MATLAB mathematical toolbox documentation