How do I train only one layer of a neural network?

I have a pre trained neural network, and I want to apply a temperature scaling calibration to it.

This means that I want to add one layer in front of the softmax layer, and train only the weights of this layer on the validation set.

However, I cannot seem to edit the network such that only this layer 'trains'. I have tried setting all the learning rates to zero, but even then some things change. Like; Batch Normalization layers and the input layers still update.

I even tried setting the overall learnrate to some value very small (1e-12) and the learn rate of my layer to something very large (1e8) but this does not work either. Other parts of the model still change.

This should be a relatively easy task, but I cannot seem to find a way to solve it. Any tips/ideas?

Answer 1

[Shreyansh Mehra]

Hello,

The MATLAB Answers link given below explains that to freeze some weights of a neural network, learning parameters 'BiasLearnRateFactor' and/or 'WeightLearnRateFactor' can be set to zero, and might be of help. Please have a look.

https://www.mathworks.com/matlabcentral/answers/548100-how-can-i-freeze-specific-weights-of-neural-network-model