Transfer Learning with GoogLeNet Error when Freezing Weights

Hello,

Recently I have been exploring the machine learning and neural network capabilities of Matlab and I have been trying to use transfer learning on the GoogLeNet pretrained network to see how it performs on some text recognition datasets.

I have been following the syntax closely in the documentation here but I'm encountering some errors when I try to freeze the initial layers. The syntax in the example is as follows:

layers = lgraph.Layers;
connections = lgraph.Connections;

layers(1:110) = freezeWeights(layers(1:110));
lgraph = createLgraphUsingConnections(layers,connections);

For some reason when trying to set the layers and connections variables as is shown above, I get the error message "The expression to the left of the equals sign is not a valid target for an assignment". This happens both when I try to assign the 'layers' and 'connections' variables. Strangely enough, I can set both of these variables if I first type 'lgraph.Layers' or 'lgraph.Connections' into the workspace and then following those workspace, allowing the '.Layers' or '.Connections' properties be stored in the default temporary variable 'ans' and then setting the desired variables with "layers = ans" or "connections = ans".

The second problem I am encountering is with the 'freezeWeights()' command. When I try to run that I get the error message, "Undefined function or variable 'freezeWeights'" which is strange considering I have downloaded the googleNet support package. Is there another support package that has the freezeWeights command? I can't seem to find anything on that command.

Thank you for any assistance.

Answer 1

[Oleksii Sidorov]

Dear Andrew,

the first version of MatLab where function freezeWeights() was realized is 2018a. I'm quite sure that you face problems because you are using older version of MatLab.

[Andrew Chen]

This is in fact the case. I was using 2017b. There is a transfer learning example there with the googlenet but it does not use the freeze weights function.

Answer 2

[pefage pefage]

Got the same issue on MATLAB 2018a

[pefage pefage]

Not sure but I've written the code for freezeWeights() myself:

    for i = 1:numel(layers)
        if isprop(layers(i),'WeightLearnRateFactor')
            layers(i).WeightLearnRateFactor = 0;
        end
        if isprop(layers(i),'WeightL2Factor')
            layers(i).WeightL2Factor = 0;
        end
        if isprop(layers(i),'BiasLearnRateFactor')
            layers(i).BiasLearnRateFactor = 0;
        end
        if isprop(layers(i),'BiasL2Factor')
            layers(i).BiasL2Factor = 0;
        end
    end

Additional not working functions I've found here: https://blogs.mathworks.com/deep-learning/2018/03/19/creating-a-dag-network-from-dag-parts/

[Runcong Kuang]

I can see the "freezeWeights.m" file integrated in matlab now:

```

% layers = freezeWeights(layers) sets the learning rates of all the

% parameters of the layers in the layer array |layers| to zero.

function layers = freezeWeights(layers)

for ii = 1:size(layers,1)

props = properties(layers(ii));

for p = 1:numel(props)

propName = props{p};

if ~isempty(regexp(propName, 'LearnRateFactor$', 'once'))

layers(ii).(propName) = 0;

end

end

end

end

```

So there is no need to make '2Factor' 0.

Answer 3

[Michael Markzon]

I am using 2018a and got the error

[Christopher Bokros]

Same here, but since the code is as easy to write as @pefage pefage demonstrated it's not a big deal.

Answer 4

[Aqib Mumtaz]

I am using 2018a on windows10.

I solved freezeWeights issue as @pefage pefage demonstrated

but now I am getting undefined error on createLgraphUsingConnections(layers,connections)

Undefined function or variable 'createLgraphUsingConnections'.

How to proceed now?

[Deepali Mishra]

Don't use 'createLgraphUsingConnections' and train your model after using freezeWeights , that issue won't come again.

[Runcong Kuang]

Hi, I am curious because I am only using my own defined layers as neural network.

If I directly use layers (layer array), do I need to do 'createLgraphUsingConnections'?

It's only for layergraph object, right?

I think the example here is fancy in using lgraph for replacement and connection.

If you have some insight, please tell me.

Thanks.