How can I train a neural network manually choosing a specific set of initial weights and biases?
6 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
Roberto
el 3 de Feb. de 2016
Comentada: Nga Dao Thi
el 4 de En. de 2017
My goal is to reproduce many times exactly the same training in order to do some tests about some training parameters variations. So what i tried first was to define a neural network, configure it, initialize it. Then I "switched off" the init function in order to keep fixed the initial values of weights and biases and use them again later in another training. I succeeded in this by setting
net.initFcn = 'initlay';
net.layers{1,1}.initFcn = 'initwb';
net.layers{2,1}.initFcn = 'initwb';
net.inputWeights{1,1}.initFcn = '';
net.layerWeights{2,1}.initFcn = '';
net.biases{1,1}.initFcn = '';
net.biases{2,1}.initFcn = '';
and it actually worked. In this way even calling the function "init" or "configure" does not affect in any way the weights and biases of the net. So I thought that if I manually set two nets in this way and with the same initial values, training them would result in exactly the same final weights/biases...but it doesn't... My question is why?
Does "trainlm" re-initialize the net in some way different than calling "init"? Has it some randomness other than initialization?
0 comentarios
Respuesta aceptada
Greg Heath
el 3 de Feb. de 2016
If you want to reproduce initial weights, reinitialize the rng to the same state before using configure.
If any non-zero weights are present, TRAIN will treat all weights as initial weights. Otherwise it will self-initialize using random weights.
Hope this helps.
Thank you for formally accepting my answer
Greg
Más respuestas (1)
Greg Heath
el 3 de Feb. de 2016
After net is trained the state of the rng is changed. Therefore the weight initialization of net2 is different.
% help fitnet
close all, clear all, clc
[ x, t ] = simplefit_dataset;
net1 = fitnet; net2 = fitnet; net3 = fitnet;
isequal(net1,net2), isequal(net1,net3)% 1,1
WB0 = [getwb(net1) getwb(net2) getwb(net3) ] % zeros(10,3)
rng(0), net1 = train(net1,x,t); net2 = train(net2,x,t);
rng(0), net3 = train(net3,x,t);
WB = [getwb(net1) getwb(net2) getwb(net3)]
WB =
-16.568 -11.025 -16.568
-5.5931 -7.8365 -5.5931
3.6185 -3.8022 3.6185
-4.0666 4.3355 -4.0666
-1.85 1.6122 -1.85
0.07357 -0.094195 0.07357
-2.834 3.1095 -2.834
4.3672 -5.1322 4.3672
8.4144 7.4463 8.4144
12.07 13.83 12.07
12.118 12.197 12.118
6.6852 9.9585 6.6852
-6.9822 7.4495 -6.9822
10.482 -11.442 10.482
12.836 -11.914 12.836
-12.557 -14.228 -12.557
-7.3149 7.7502 -7.3149
7.6948 -8.879 7.6948
9.7115 8.1667 9.7115
12.196 12.392 12.196
-0.84428 -1.9359 -0.84428
-0.71237 0.25416 -0.71237
0.72471 0.36081 0.72471
0.81622 -0.766 0.81622
-0.18894 0.17421 -0.18894
0.090501 -0.095302 0.090501
-0.051973 -0.040525 -0.051973
0.11345 -0.13442 0.11345
-0.21845 0.19223 -0.21845
0.31931 0.53145 0.31931
0.41655 1.956 0.41655
2 comentarios
Nga Dao Thi
el 4 de En. de 2017
Thank you! I experienced the similar situation. I define a neural network (NN) with initial weights are 0s. However, when this NN trains the same dataset differently, i.e., the results are different when I run program many times. I also don't know why it happens. Later, I find that the training function trainscg will update weight and bias values according to the scaled conjugate gradient method. If I set biases are 0s, my NN has the same result. In contrast, If I set biases are not 0s, trainscg will update biases and weights. So that, the NN trains differently.
Ver también
Categorías
Más información sobre Function Approximation and Nonlinear Regression en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!