Must centralized trained groups of agents have the same action specification?

Hello:

numObservations = 4;
observationInfo = rlNumericSpec([numObservations 1],...
    'LowerLimit',[-inf -inf -inf -inf]',...
    'UpperLimit',[ inf  inf inf inf]');
observationInfo.Name = 'observations';
observationInfo.Description = 'force bias, position';
numActions1 = 1;
actionInfo1 = rlNumericSpec([1 1],...
    'LowerLimit',[0]',...
    'UpperLimit',[66]');
actionInfo1.Name = 'onlyb';
numActions2 = 1;
actionInfo2 = rlNumericSpec([1 1],...
    'LowerLimit',[0]',...
    'UpperLimit',[13]');
actionInfo2.Name = 'onlyK';
mdl = "ER3AC60A";
blks = ["ER3AC60A/RL AgentA", "ER3AC60A/RL AgentB"];
obsInfos = {observationInfo,observationInfo};
actInfos = {actionInfo1,actionInfo2};
env.ResetFcn = @(in)localResetFcn(in);
env = rlSimulinkEnv(mdl,blks, obsInfos,actInfos);
%行为克隆与探索平衡2.5是平衡
Ts = 0.001;
Tf = 1;
rng(0)

Must groups of agents using centralized training have the same output specification?

错误使用 rl.train.marl.util.validateMultiAgentTrainingOptions

Agents in group [1 2] must have the same observation and action specifications.

Answer 1

[Ayush Aniket]

Hi Lin,

From the error statement: 'Agents in group [1 2] must have the same observation and action specifications.', it seems that you have used the same group for agent1 and agent2 with the 'centralized' LearningStrategy option in the rlMultiAgentTrainingOptions function.

In centralized training, agents share their collected experiences and learn collaboratively, necessitating that the observation and action information for all agents in the group have the same structure.

Refer to the following MATLAB answer for a detailed distinction between centralized and decentralized learning strategies: https://www.mathworks.com/matlabcentral/answers/2002007-centralized-vs-decentralized-training-for-multi-agent-reinforcement-learning