Reinforcement Learning Error with two scalar inputs
1 visualización (últimos 30 días)
Mostrar comentarios más antiguos
ali farid
el 31 de En. de 2024
Comentada: Emmanouil Tzorakoleftherakis
el 12 de Feb. de 2024
I have a strange error from a critic network that has 3 inputs, image, and two scalars. But I see the following error:
Error using rl.internal.validate.mapFunctionObservationInput
Unable to automatically specify deep neural network observation input layer names because some specifications have similar dimension. Specify "ObservationInputNames" name-value pair when
creating function object.
Error in rlContinuousGaussianActor (line 95)
modelInputMap = rl.internal.validate.mapFunctionObservationInput(model,observationInfo,nameValueArgs.ObservationInputNames);
Do you have any idea? It is really long time I am working on this error. My network is as follows:
type=3;
w1=2;
w2=1;
% obsMat = [1 1];
obsMat = [4 3; 5 3; 6 3; 7 3; 8 3; 9 3; 5 11; 6 11; 7 11; 8 11; 6 12; 7 12; 10 12; ];
sA0 = [2 5];
sB0 = [11 5];
sC0 = [3 2];
sD0 = [6 5];
sE0 = [12 5];
sF0 = [6 12];
sG0 = [11 5];
sH0 = [3 11];
sI0 = [6 3];
sJ0 = [1 11];
s0 = [sA0; sB0; sC0];
% s0 = [sA0; sB0; sC0; sD0];
% s0 = [sA0; sB0; sC0];
Ts = 0.1;
Tf = 100;
maxsteps = ceil(Tf/Ts);
mdl = "rlAreaCoverage32024";
open_system(mdl)
% Define observation specifications.
scalarObs1Info = rlNumericSpec([1 1]);
scalarObs1Info.Name ="scalarObservation1";
scalarObs2Info = rlNumericSpec([1 1]);
scalarObs2Info.Name ="scalarObservation2";
obsSize = [12 12 4];
oinfo = rlNumericSpec(obsSize);
oinfo.Name ="image";
% oinfo.Name = "observations";
allObsInfo = [ oinfo, scalarObs1Info, scalarObs2Info];%, scalarObs3Info];
actionSpace = {1,2,3,4};
ActionInfo = rlNumericSpec([1, 2], 'Lowerlimit', -1, 'Upperlimit', 1); ainfo = ActionInfo;
ainfo.Name = "actions";
actInfo.UpperLimit=1;
actInfo.Lowerlimit=-1;
blks = mdl + ["/Agent A (Red)","/Agent B (Green)","/Agent C (Blue)"];
env = rlSimulinkEnv(mdl,blks,{allObsInfo,allObsInfo,allObsInfo},{ainfo,ainfo,ainfo});
env.ResetFcn = @(in) resetMap(in, obsMat);
rng(0)
for idx = 1:type
lgraph = layerGraph();
tempLayers = [
featureInputLayer(w1,"Name","scalarInput1")
reluLayer("Name","relu_3")
fullyConnectedLayer(1,"Name","fc_4")];
lgraph = addLayers(lgraph,tempLayers);
tempLayers = [
featureInputLayer(w3,"Name","scalarInput3")
reluLayer("Name","relu_1")
fullyConnectedLayer(1,"Name","fc_2")];
lgraph = addLayers(lgraph,tempLayers);
tempLayers = [
imageInputLayer(obsSize,Normalization="none")
convolution2dLayer(8,16, ...
Stride=1,Padding=1,WeightsInitializer="he")
reluLayer
convolution2dLayer(4,8, ...
Stride=1,Padding="same",WeightsInitializer="he")
reluLayer
fullyConnectedLayer(256,WeightsInitializer="he")
reluLayer
fullyConnectedLayer(128,WeightsInitializer="he")
% Hidden units (default = 128) are the number of units in the hidden layer of the neural network. Its size depends on the complexity of the problem, and should be set larger when there is a complex relationship between agent actions and observed variables.
reluLayer
fullyConnectedLayer(1,"Name","fc_1")];
lgraph = addLayers(lgraph,tempLayers);
tempLayers = [
concatenationLayer(1,3,"Name","concat")
softmaxLayer("Name","softmax")];
lgraph = addLayers(lgraph,tempLayers);
% clean up helper variable
clear tempLayers;
lgraph = connectLayers(lgraph,"fc_2","concat/in3");
% lgraph = connectLayers(lgraph,"fc_3","concat/in2");
lgraph = connectLayers(lgraph,"fc_1","concat/in4");
lgraph = connectLayers(lgraph,"fc_4","concat/in1");
plot(lgraph);
actorNetwork=lgraph;
% dlnetwork(actorNetwork);
actorOptions = rlOptimizerOptions('LearnRate',0.1,'GradientThreshold',inf);
actor(idx) = rlContinuousGaussianActor(actorNetwork,allObsInfo,ainfo);
1 comentario
Emmanouil Tzorakoleftherakis
el 12 de Feb. de 2024
The code you sent cannot be executed as a standalone. Also, I am getting different errors that what you sent. Please modify the code to reproduce the error you mentioned above.
Respuestas (0)
Ver también
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!