How can I calculate BatchNorma​lisationLa​yer's parameters after a checkpoint saving

Hi, I use matlab R2017b with the neural network toolbox. I trained the following neural-network with the following training options :

     layers = [
    imageInputLayer([60 60 1])    

    convolution2dLayer(3,32,'Padding',2)
    batchNormalizationLayer             % My problem comes from the batch Normalisation Layers...
                                        % whose parameters are not saved at it checkPoint saving    
    reluLayer                              
    convolution2dLayer(3,32,'Padding',2)    
    batchNormalizationLayer                 
    reluLayer                               
    maxPooling2dLayer(2,'Stride',2)         

    convolution2dLayer(3,128,'Padding',2)    
    batchNormalizationLayer                 
    reluLayer 
    convolution2dLayer(3,128,'Padding',2)    
    batchNormalizationLayer                 
    reluLayer 
    maxPooling2dLayer(2,'Stride',2)         

    convolution2dLayer(3,256,'Padding',2)    
    batchNormalizationLayer                
    reluLayer 
    convolution2dLayer(3,256,'Padding',2)    
    batchNormalizationLayer                 
    reluLayer 
    maxPooling2dLayer(2,'Padding', 'same','Stride',2)         

    fullyConnectedLayer(1024)
    reluLayer
    fullyConnectedLayer(128)
    reluLayer
    fullyConnectedLayer(16)
    reluLayer
    fullyConnectedLayer(2)

    softmaxLayer

    classificationLayer()];

    options = trainingOptions('sgdm',...
      'MaxEpochs',50, ...
      'ValidationData',testDigitData,...
      'ValidationFrequency',10,...
      'ValidationPatience',Inf,...
      'Verbose',false,...
      'Plots','training-progress',...
      'MiniBatchSize',10,...
      'CheckpointPath','savedNetworkAtEachEpoch',... % I saved My network at each epoch but the parameters of the 
                                                     % batchNormalisationLayer are not saved
      'LearnRateSchedule','piecewise',...
      'LearnRateDropFactor',.1,...
      'LearnRateDropPeriod',2,...
      'Shuffle','every-epoch');

PierreNet = trainNetwork(trainDigitData,layers,options);

When I load a checkpoint network saved at the end of one epoch, the BatchNormalisationLayer parameters 'TrainedMean' and 'TrainedVariance' are empty. Therefore, it is not possible to do any classification. I got this error : "Unable to use networks with batch normalization layers before training is complete. UsetrainNetwork to complete network training."

Am I oblige to run the training again to compute the missing parameters ? If not, how can I easily compute those parameters ?

Thanks

[Korhan]

I'm having the same problem. Have you figured out a solution?

[pierre ghesquiere]

No I don't have a clear answer on the way to calculate those batch parameters. As you can read here : https://www.mathworks.com/matlabcentral/answers/391509-r2018a-neural-network-toolbox-bugs "If your network contains batch normalization layers, then the final validation metrics are often different from the validation metrics evaluated during training. This is because batch normalization layers in the final network perform different operations than during training.

In particular, batch normalization layers use means and variances evaluated on each mini-batch during training, but use the whole training set for the final network."

But it doesn't explain how to calculate those parameters.