Maximixe output of Neural Network After training

Suppose that I've successfully trained a neural network. Given that the weights are now fixed, is there a way to find the input (x) of the neural network that leads to the maximum output (y)? Such an input-based optimization should be possibile, since neural networks inherently support gradient backpropagation. I wonder if this operation is properly supported in Matlab (with a dedicated function or even a guideline), or of it has to be built from scratch

Answer 1

[Shreeya]

Hello @Federico Toso

According to my understanding, you want to maximize the output of a neural network across all predictions i.e. the output values of the last layer.

If I understand the problem statement correctly, you can apply a linear activation function to the last layer. You can use the "leakyRelu" function if the prediction values are supposed to be positive values. In the other case, define a custom activation function of the form .

Refer to the documentation for further details:

https://www.mathworks.com/help/deeplearning/ug/list-of-deep-learning-layers.html#mw_f62fbe0d-a138-49cc-a404-19e9bda934fd

This will set the last layer output valuyes as required.

Hope this helps!

[Federico Toso]

Hello @Shreeya thank you for your answer. I understand your suggestion, but I cannot see how it is supposed to help with my maximization goal. After the application of the "leakyRelu" function, I still have to find a way to programmatically find the NN input (x) that leads to maximum NN ouptut (y), assuming the network weights are fixed. I'm editing my question above for better clarity. Please let me know if you have further suggestions

Answer 2

[Harsha Vardhan]

Hi,

I understand that you want to find the maximum output of a trained neural network in MATLAB by optimizing over the input space.

There are 2 possible ways of achieving this.

Optimization Toolboxes: You can use optimization functions from the MATLAB's Global Optimization Toolbox and Optimization Toolbox. Here, the output of your trained neural network becomes the objective function, and the inputs are treated as the variables to be optimized. The objective function may look like this:

function y = objectiveFunction(x)
    % net is your trained neural network
    % Minimizing -net(x) results in maximizing net(x)
    y = -net(x); 
end

Please find the documentation of those toolbox functions here.

• Optimization Toolbox Functions: https://www.mathworks.com/help/optim/referencelist.html?type=function&listtype=cat&category=index&blocktype=all&capability=
• Global Optimization Toolbox Functions: https://www.mathworks.com/help/gads/referencelist.html?type=function&listtype=cat&category=index&blocktype=all&capability=

Custom Training Loop: In general neural networks, loss function is minimized and gradients are calculated over the weights. Similarly, in your case, we can minimize the -net(x) (results in maximization of net(x) output) and calculate the gradients over the inputs, where 'net' is the trained network. This can be done using a custom training loop in the Deep Learning Toolbox. Please find the documentation of custom training loop here - https://www.mathworks.com/help/deeplearning/deep-learning-custom-training-loops.html

Hope this helps in resolving your query!