Error in "Soft-Actor critic" agent with "rlNumericSpec" Observations and "rlFiniteSpec" Actions.

Hello, I am trying to train SAC agent with "rlNumericSpec" Observations and "rlFiniteSpec" Actions.

This is my code:

ObservationInfo = rlNumericSpec([2 1],...
    'LowerLimit',[-inf -inf]',...
    'UpperLimit',[inf inf]');
ObservationInfo.Name = "observations";
ObservationInfo.Description = 'aa, bb';
numObservations = ObservationInfo.Dimension(1);

ActionInfo = rlFiniteSetSpec([1 2 3 4 5 6 7 8 9]);
ActionInfo.Name = 'Action';
ActionInfo.Description = 'test';
numActions = ActionInfo.Dimension(1);

net = [  
    featureInputLayer(ObservationInfo.Dimension(1))
    fullyConnectedLayer(16)
    reluLayer
    fullyConnectedLayer(16)
    reluLayer
    fullyConnectedLayer(9) 
    ];

net = dlnetwork(net);

critic = rlVectorQValueFunction(net,ObservationInfo,ActionInfo);

net1 = [  
    featureInputLayer(ObservationInfo.Dimension(1))
    fullyConnectedLayer(16)
    reluLayer
    fullyConnectedLayer(16)
    reluLayer
    fullyConnectedLayer(9) 
    ];

net1 = dlnetwork(net1);


actor = rlDiscreteCategoricalActor(net1,ObservationInfo,ActionInfo);

agentOptions = rlSACAgentOptions("SampleTime",Ts, ...
    "DiscountFactor", 0.001, ...
    "ExperienceBufferLength",1e6, ...
    "MiniBatchSize",64, ...
    "TargetSmoothFactor",0.005 ...
    );

agent = rlSACAgent(actor,critic,agentOptions);

However, when I trying to train the model, the error occurs:

Error using rlSACAgent (line 96)
First argument must be an rlDiscreteCategoricalActor object, an rlContinuousGaussianActor object or an observation specification created
using 'rlNumericSpec' or 'rlFiniteSetSpec' objects.

Error in mdl_RF (line 60)
agent = rlSACAgent(actor,critic,agentOptions);

Please help me if you know the solutions.

[Hitesh]

Hi @minhyuk jeung

I am not able to reproduce this error using the above snippet of code, Could you share the complete code where you are creating the "env" and passing it during training ?

Answer 1

[Gayathri]

Hi @minhyuk jeung,

I faced a similar issue in MATLAB R2024a version. But the same code runs fine in MATLAB R2024b version. The solution would be to upgrade to MATLAB R2024b version.

Hope you find this information helpful.