Conducting automatic differentiation on simple neural network (.net) file

Hi,

I would like to conduct an automatic differentiation on a simple neural network. I know this can be done on deep neural networks. I already tried this but I receive an error message saying 'value to differentiate must be a traced dlarray scalar'.

Here is my code

function [Y, dydA,dydB, dydC, dydD] = Y_partial(A,B,C, D)

Y = load('i_Y.mat','net'); %Loading the saved network

[dydA,dydB, dydC, dydD] = dlgradient(Y, A, B,C, D);

end

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

A = dlarray(A);

B = dlarray(B);

C= dlarray(C);

D = dlarray(D);

[Y,dydA,dydB, dydC, dydD] = dlfeval(@Y,A,B,C, D);

Answer 1

[Abhijeet]

Hi,

The error message indicates that the input to the dlgradient must be a scalar value, but Y in your code seems to be a loaded neural network. To use dlgradient, you need to perform a forward pass through the neural network and obtain a scalar output that you want to differentiate with respect to the inputs.

function [Y, dydA, dydB, dydC, dydD] = Y_partial(A, B, C, D)
    net = load('i_Y.mat','net'); % Loading the saved network
    Y = predict(net.net, A, B, C, D); % Performing a forward pass
    [dydA, dydB, dydC, dydD] = dlgradient(Y, A, B, C, D);
end

A = dlarray(A);
B = dlarray(B);
C = dlarray(C);
D = dlarray(D);
[Y, dydA, dydB, dydC, dydD] = dlfeval(@Y_partial, A, B, C, D);

Here, predict performs a forward pass through the neural network to obtain the scalar output Y, which can then be used as the first argument to dlgradient. Y_partial is modified to take the input values as arguments and perform the necessary computation to produce Y and the gradients. After this, dlfeval is used to evaluate Y_partial with the input arguments specified as dlarray objects.

Thanks

[Tomey]

Thank you Abhijeet, however the predict function does not work on the neural network rather the sim function works. Is this still feasible?

[Abhijeet]

Welcome Tomey !!

Yes, Sim should work and resolve the issue that you're facing. You can still dig into the documentation of sim and predict to explore more using the commands.

help sim, doc sim, help predict, doc predict

Thanks Again !!

[Tomey]

Thank you. Abjijeet. It seems that the sim function does not take dlarrays number which is applicable to only the automatic differentiation (dlgradient). I tried to create a new variables that can be fed into the sim function and then use the dlarray variables on a the dlgradient. But it doesnt seem to work. What do you suggest?