Regarding Default Terms in DNN

Hello,

for my project on application of Deep reinforcement learning, I have used Reinforcement Learning and Deep Learning Toolboxes. It does not include classification or regression. I have also not created any user defined function or agent in my model. As part of report I want to mention the kind of loss function used in my model, however it seems that any information is not mentioned in the help menu, for instance the kind of loss function used by default with the fully connected layers. Specifically, my NN is series of feature input layer, ReLU and fully connected layers. If the query is still not cleared, I can describe it further.

Answer 1

[Emmanouil Tzorakoleftherakis]

Which algorithm are you using? You can log loss data by following the guidelines here.

[Parth Patel]

The algorithm used is TD3. I am preferring to use the Matlab version 2021b, and could not access this function 'rlDataLogger' here. Is there any alternate way to check the loss data too, in addition to also check what is the default type of function used to calculate the loss here?

[Emmanouil Tzorakoleftherakis]

The choice of loss function depends on the agent. For TD3 you can find it here. Unfortunately there is no easy way to check the loss data before 22b. This is why we introduced the data logger.

The networks in Reinforcement Learning Toolbox are different from traditional supervised learning in the sense that they do not have explicit loss layers. We output the critic prediction and compute the loss internally w.r.t. the target critic network. For actor networks the loss is a bit more complicated and varies with the algorithm. For example, for DDPG the actor loss is -Q(s,a) computed across the mini batch.

Hope this helps