Error using GPU in Matlab 2020a for Reinforcement Learning

12 visualizaciones (últimos 30 días)
I keep running into this error when using 'UseDevice',"gpu" in rlRepresentationOptions. The issue seems to appear after the the simulation happens for random period of time. I have tried this with multiple built-in examples and with both DDPG and TD3 agent. Could someone direct me if I am doing something wrong or is this a bug?
Error using rl.env.AbstractEnv/simWithPolicy (line 70)
An error occurred while simulating "IntegratedFlyingRobot" with the agent "agent".
Error in rl.task.SeriesTrainTask/runImpl (line 33)
[varargout{1},varargout{2}] = simWithPolicy(this.Env,this.Agent,simOpts);
Error in rl.task.Task/run (line 21)
[varargout{1:nargout}] = runImpl(this);
Error in rl.task.TaskSpec/internal_run (line 159)
[varargout{1:nargout}] = run(task);
Error in rl.task.TaskSpec/runDirect (line 163)
[this.Outputs{1:getNumOutputs(this)}] = internal_run(this);
Error in rl.task.TaskSpec/runScalarTask (line 187)
runDirect(this);
Error in rl.task.TaskSpec/run (line 69)
runScalarTask(task);
Error in rl.train.SeriesTrainer/run (line 24)
run(seriestaskspec);
Error in rl.train.TrainingManager/train (line 291)
run(trainer);
Error in rl.train.TrainingManager/run (line 160)
train(this);
Error in rl.agent.AbstractAgent/train (line 54)
TrainingStatistics = run(trainMgr);
Caused by:
Error using rl.env.SimulinkEnvWithAgent>localHandleSimoutErrors (line 689)
Invalid input argument type or size such as observation, reward, isdone or loggedSignals.
Error using rl.env.SimulinkEnvWithAgent>localHandleSimoutErrors (line 689)
Unable to compute gradient from representation.
Error using rl.env.SimulinkEnvWithAgent>localHandleSimoutErrors (line 689)
Unable to evaluate the loss function. Check the loss function and ensure it runs successfully.
Error using rl.env.SimulinkEnvWithAgent>localHandleSimoutErrors (line 689)
Input data dimensions must match the dimensions specified in the corresponding observation and action info specifications.
  5 comentarios
Stav Bar-Sheshet
Stav Bar-Sheshet el 21 de Mayo de 2020
Found a workaround to use until this bug will get solved.
You can still train the Actor with GPU and the Critic with CPU.
With this configuration you can also still use the parallel pool for gathering multiple experiences faster.
This reduced my training time in comparison to just train with CPU for both A & C.
Jing Chen
Jing Chen el 4 de Sept. de 2020
Editada: Walter Roberson el 4 de Sept. de 2020
I have a similar error using DDPG and cpu, but I can not understand what the bug is in the last line means. My obeservation and action are like the example in Train DDPG Agent to Swing Up and Balance Pendulum with Image Observation ( https://www.mathworks.com/help/deeplearning/ug/train-ddpg-agent-to-swing-up-and-balance-pendulum-with-image-observation.html?s_tid=srchtitle ). The environment is built in Simulink and the RLAgent in Simulink is used . While this example does not explain how to build an environment by ourselves.
Error in rl.task.SeriesTrainTask/runImpl (line 33)
[varargout{1},varargout{2}] = simWithPolicy(this.Env,this.Agent,simOpts);
Error in rl.task.Task/run (line 21)
[varargout{1:nargout}] = runImpl(this);
Error in rl.task.TaskSpec/internal_run (line 159)
[varargout{1:nargout}] = run(task);
Error in rl.task.TaskSpec/runDirect (line 163)
[this.Outputs{1:getNumOutputs(this)}] = internal_run(this);
Error in rl.task.TaskSpec/runScalarTask (line 187)
runDirect(this);
Error in rl.task.TaskSpec/run (line 69)
runScalarTask(task);
Error in rl.train.SeriesTrainer/run (line 24)
run(seriestaskspec)
Error in rl.train.TrainingManager/train (line 291)
run(trainer);
Error in rl.train.TrainingManager/run (line 160)
train(this);
Error in rl.agent.AbstractAgent/train (line 54)
TrainingStatistics = run(trainMgr);
Caused by:
Error using rl.env.SimulinkEnvWithAgent>localHandleSimoutErrors (line 689)
Invalid input argument type or size such as observation, reward, isdone or loggedSignals.
Error using rl.env.SimulinkEnvWithAgent>localHandleSimoutErrors (line 689)
Unable to compute gradient from representation.
Error using rl.env.SimulinkEnvWithAgent>localHandleSimoutErrors (line 689)
Number of elements must not change. Use [] as one of the size inputs to automatically calculate the appropriate size for that dimension.

Iniciar sesión para comentar.

Respuesta aceptada

Stephan
Stephan el 22 de Mayo de 2020
  2 comentarios
Adriano Mele
Adriano Mele el 13 de Abr. de 2021
Editada: Adriano Mele el 13 de Abr. de 2021
Hi Stephan,
I have a similar error, but my matlab version is 2021 and up-to-date (?)
I tried to build a simulink environment for RL DDPG training. If I simulate the environment alone (with zero actions) it seems to work just fine, but whenever I try to create an integrated environment and train the agent the code breaks with the same error reported above. I tried with both cpu and gpu.
Any hint?
kloner
kloner el 2 de Jul. de 2021
I have a similar error on 2021a when considering tabular Q-Learning and observations larger than [1 x 1]. Guidance for this particular issue or more precise error statements would be greatly appreciated!

Iniciar sesión para comentar.

Más respuestas (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by