Implementing NEAT algorithm in Simulink

I am trying to modify the walking robot example to use the NEAT algorithm, but I don't see any way of building an evolving neural network using the reinforcement learning toolbox. I was thinking that maybe somehow I could incorporate Python-NEAT? I am new so I'm not really sure what's possible.

Answer 1

[Tushar Sharma]

Hi Allegra,

I assume that you are referring to 'msra-walking-robot' - https://github.com/mathworks/msra-walking-robot.

The algorithm used in this case is Deep Deterministic Policy Gradient (DDPG), chosen for the following reasons:

• It learns with environments with continuous action spaces
• Deterministic policies are faster to learn

If you want to write your own environment and training algorithm, the documents listed below servers as a good starting point:

• https://www.mathworks.com/help/reinforcement-learning/matlab-environments.html
• https://www.mathworks.com/help/reinforcement-learning/training-and-validation.html

Also, If you want to incorporate NEAT-Python, then you may leverage MATLAB's ability to call Python functions to directly interact with the NEAT-Python implementation. This means you'll be running the simulation in MATLAB, and each time you need to evaluate a genome's fitness, you'll call the simulation from your Python code through MATLAB.

Here are the steps to follow:

1. Ensure Python is installed.
2. Next, install NEAT-Python by running `pip install neat-python`.
3. Finally, set up MATLAB to utilize Python by configuring the correct Python environment with the `pyenv` function.

The following resources would help-

https://www.mathworks.com/help/matlab/matlab_external/create-object-from-python-class.html#mw_c224a09a-f56b-48e5-b9ec-145388506204