Reinforcement learning - DDPG - minibatch - Continuos action saturation

what is the influence of the minibatch? since in this case its value determines the moment in which the continuous action signal begins to saturate (it only adopts the values ​​of the limits-blue line). In the case in which the value of the minibatch is very large, the weights of the nets do not change over time. Action[-0.15 0.15],

I don't know why before the minibatch which is when it starts from random weights it does not saturate but it is the value of the minibatch that determines where the saturation begins

data:

GradientThreesHold: 1

Learnrate: 0.03

agentOpts.NoiseOptions.StandardDeviation = 0.001;

agentOpts.NoiseOptions.StandardDeviationDecayRate =0.00001;

minibatch: 100 --- x axis=500

Answer 1

[Yash]

The minibatch size in a neural network training process refers to the number of samples used in each iteration to update the weights of the network. A larger minibatch size can lead to slower weight updates and potentially slower convergence, while a smaller minibatch size can result in faster weight updates but with more noisy updates.

In your case, it seems that a larger minibatch size is causing the action signal to saturate earlier. This could be because with a larger minibatch size, the weight updates are slower, and therefore the network takes longer to adapt to the changing environment. As a result, the action signal reaches its limits ([-0.15, 0.15]) earlier. It may be worth experimenting with different minibatch sizes to find the optimal value for your specific case.