Save training result of reinforcement learning training in simulink

Hello,

I am training a DDPG agent in Simulink. After the training, I save the training result and the agent.

Several signals for logging are activated in the model.

For long training sessions, I now have the problem that the variable containing the training result is very large, as it contains all the logged signals of each episode.

Is it possible to log the signals without saving them in the training result variable?

Does logging the signals have an effect on the training duration?

Best regards,

allmo

[Pei Seng Tan]

Hi, you are not alone as I am facing this issue as well.

I am getting the error of "out of memory" in the middle of training (around 800 episodes) due to insufficient RAM (16GB for me). I suspected that a lot of memory is used to store the training result leading to this "out of memory" error. Before the training starts, the RAM is consumed only 5.8GB.

Awaiting for the answer from the experts.

Answer 1

[Avadhoot]

Hi Allmo,

I understand that due to the signal logs your training variable is getting too large. Instead of piling on the logged signals for each session in the training variable, you can save the signal logs in a file.

While Simulink does not provide a direct way to save the signal logs in a file, you can do so by adding a “To File” block in your Simulink model. To give multiple signals as an input to the ”To File” block, you can create a bus object comprising of all the signals you want to log using the bus creator. Specify the filename in which you want to save the logs in the block parameters for the “To File” block. This way your signal logs will be saved in the file. Remember to stop logging the signals so that the signal logs won’t be generated in the training variable. To do that, uncheck the “Signal logging” option in the Data Import/Export section of the model configuration settings of your Simulink Model.

Regarding your second query, logging the signals does affect the training duration. Saving the signal logs to a file can also affect the duration as well as disk space. Ensure that you have sufficient disk space if you are logging a large amount of data to a file.

For more information about the “To File” block and the bus creator block, refer to the following documentation links:

1. https://www.mathworks.com/help/simulink/slref/tofile.html
2. https://www.mathworks.com/help/simulink/slref/buscreator.html

I hope this helps.