Issue with batch normalization layer of saved CNN
17 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
Kjetil Kveim
el 21 de Mzo. de 2019
Respondida: Sam Leeney
el 15 de Dic. de 2022
When loading a previously trained CNN, I got an issue with the batch normalization layers. When looking into the loaded CNN model the trainable mean and variance are empty.
Name: 'batchnorm_1'
TrainedMean: []
TrainedVariance: []
So the checkpoint doesn't seem to save these parameters. Are there any workarounds for this issue? I am using Matlab R2018b.
1 comentario
Wes Baldwin
el 29 de Jul. de 2020
Doesn't this mean using checkpoints on networks with a batchnorm layer is useless??? Kinda a big deal for long training!!! You could potentially lose days or weeks of training with no option but to start from the beginning.
Respuesta aceptada
Vivek Singh
el 22 de Mzo. de 2019
We were able to reproduce the issue. We will inform you once the issue is fixed.
Since TrainedMean and TrainedVariance are calculated after the training is finished, therefore as a workaround you can use the below mentioned codes to explicitly save and load the Model.
%To save model with name "demoModel", assuming your network is in "net"
save('demoModel','net')
%To load model to variable net1
net1=load('demoModel.mat','net');
net1.net.Layers(n).TrainedMean %where n is the batch normalization layer
4 comentarios
ramin nateghi
el 5 de Nov. de 2020
Editada: ramin nateghi
el 5 de Nov. de 2020
Hello,
I also faced this problem. When a model is saved by "save" function, it is ok and the model contains all information (TrainedMean and TrainedVariance) of the trained batch normalization layers. But, when the model is saved by the"checkpoint" during training, both of the TrainedMean and TrainedVariance params became empty. This is a bug for "checkpoint".
Yi Wei
el 30 de Nov. de 2020
Does it mean the saving process will cost too much time if a network contains normalization layers and the training data volume is large(e.g.8T)?
Más respuestas (1)
Sam Leeney
el 15 de Dic. de 2022
For anyone else stuck, there is a fix here; https://uk.mathworks.com/matlabcentral/answers/423588-how-to-classify-with-dag-network-from-checkpoint
0 comentarios
Ver también
Categorías
Más información sobre Image Data Workflows en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!