A mix of rlNumericSpec and rlFiniteSetSpec objects - observation for a RL environment

I am looking for an example of which implements a mix of rlNumericSpec and rlFiniteSetSpec object in an RL environment (as mentioned here). Some of my observations are numerical/continuous whereas others are finite/discrete.

Thanks.

Answer 1

[Emmanouil Tzorakoleftherakis]

Hi Krupa,

I don't think there is an example that shows how to do that in the documentation right now - I will let the doc team know.

I think something along the following lines should work:

obsInfoCont = rlNumericSpec([2 1])
obsInfoDisc = rlFiniteSetSpec([1 2 3])
obsInfo = [obsInfoCont obsInfoDisc]

[Krupa Prag]

Hi Emmanouil,

Thank you for your response.

I have a follow up question. I am feeding this information to an actor critic network.

Could you point me to documnetation that could guide me on constructing the image input layers using data as structured in your answer.

Thanks.

[Emmanouil Tzorakoleftherakis]

Hello,

As I mentioned above, we don't have an example that shows how to do that yet. You may be able to follow the structure of this one though and concatenate two different input paths. If you are using R2020b, you don't need to create the neural network structure yourself, you can create an agent using observation and action info and let Reinforcement Learning Toolbox create the neural networks for you. See for example here.

[Jonas Woeste]

Hi,

search the documentation for

rlQValueFunction

there you will find an example for a mixed observation space.

[Sania Gul]

Does the same holds for mixed action space too?

[Mohammed Mohammed]

what should i use if i have this 2 by 2 matrix

R0 = [cos(psi0)*cos(theta0) -sin(psi0)*cos(theta0)+cos(psi0)*sin(theta0)*sin(phi0);

sin(psi0)*cos(theta0) cos(psi0)*cos(theta0)+sin(psi0)*sin(theta0)*sin(phi0)];