Borrar filtros
Borrar filtros

Custom Neural Network with learnable weights in activation function

2 visualizaciones (últimos 30 días)
Ahmed Rehan
Ahmed Rehan el 16 de Sept. de 2022
Respondida: Kausthub el 7 de Sept. de 2023
Hi,
I want to create a Custom Neural Network with learnable parameters in the activation function. The learnable parameters are the weights of layers, but the weights are also used inside the activation function in a certain nonlinear relation. Normally, the activation function takes the inputs multiplied with the weights ( e,g, y1=f1(X*W1 + b1)) but here I want to use weights directly inside in a certain nonlinear relation i.e. y1=g1(X*W1 + b1,W1).
Custom layers can be created as metioned here:
But the way I want to create a custom layer with a specific activation function is what i need to know. Any help will be highly appreciated.
AR

Respuestas (1)

Kausthub
Kausthub el 7 de Sept. de 2023
Hi Ahmed Rehan,
I understand that you would like to have a custom activation function that has weights as an additional input i.e., instead of y1= f1(X*W1 + b1) you would like to have y1=g1(X*W1 + b1, W1).
You could leverage the “Define Custom Deep Learning Layer with Formatted Inputs” article which explains how to provide inputs to a custom layer.
I have attached a simple working snippet regarding the same for your reference. Major changes from the article you mentioned (https://www.mathworks.com/help/deeplearning/ug/define-custom-deep-learning-layer.html) include:
  • Adding weights as a parameter in the layer definition
function layer = customActivationLayer(weights, args)
  • Passing weights when using the layer
customActivationLayer(Weights);
Please have a look at the attached code sample for better understanding.
Hope this helps and clarifies your queries regarding how to provide weights to activation functions using custom layers!

Categorías

Más información sobre Image Data Workflows en Help Center y File Exchange.

Productos


Versión

R2022b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by