Design and Simulate Scenarios for Automated Driving Applications - MATLAB & Simulink
Video Player is loading.
Current Time 0:00
Duration 26:05
Loaded: 0.63%
Stream Type LIVE
Remaining Time 26:05
 
1x
  • descriptions off, selected
  • en (Main), selected
    Video length is 26:05

    Design and Simulate Scenarios for Automated Driving Applications

    Overview

    In this session, you will learn how to author scenarios for simulation on realistic road networks designed in RoadRunner. You can use this workflow to simulate autonomous driving with built-in agents as well as author and integrate custom agents designed in MATLAB, Simulink, or CARLA. The scenarios can be exported to OpenSCENARIO for simulation and analysis in external tools if desired.

    Highlights

    • Place vehicles and paths, define logic, and parameterize scenarios
    • Simulate and visualize scenarios in the editor
    • Simulate using built-in vehicle actors or design your actor behavior using MATLAB, Simulink or CARLA
    • Export scenarios to ASAM OpenSCENARIO for use in compliant OpenSCENARIO simulators and players, including esmini
    • Programmatically vary scenarios and automate workflows in MATLAB, C++, and Python

    About the Presenter

    Div Tiwari is a Senior Product Manager for Automated Driving. He has supported MathWorks customers establish and evolve their workflows in domains such as autonomous systems, artificial intelligence, and high-performance computing. Div holds B.S. and M.S. degrees in Electrical & Computer Engineering and Computer Science from Cornell University.

    Recorded: 15 Dec 2022

    Welcome. So you want to make a self-driving car. You will likely need to run simulations to test and validate your algorithms. To ensure performance across a variety of real world locations and situations, it is important to be able to design equivalent scenarios for your simulations.

    In this webinar, we will learn how to offer scenarios for testing automated driving applications with RoadRunner scenario. We will run simulations with built in driving agents, as well as integrate custom agents designed in MATLAB, Simulink, or CARLA. We'll explore ways to programmatically vary scenarios and to automate workflows for scene and scenario Generation finally, we'll export these scenarios to the OpenSCENARIO standard for use in external tools.

    I am Dev, a senior product manager for automated driving at MathWorks. This is RoadRunner. It is an interactive editor that lets you create rich 3D road scenes easily and with precise detail, down to varied foliage, signage, intersections, custom traffic islands, high resolution road textures, and more. You can use these scenes for simulating and testing automated driving systems.

    Here is another example of a scene. Now scene is static. Let's switch to RoadRunner Scenario to add dynamic elements, such as vehicles and people. Let's-- I've loaded up one of the sample scenarios that comes with RoadRunner Scenario.

    We are in the Scenario Edit tool to begin. We'll come back to this. For now, let's switch to the simulation tool and hit Play. I can change the simulation speed and the camera view, even as the simulation runs.

    Now let's create our own scenario. Switch back to the scenario edit tool. I'll delete the existing cars so we start from scratch. Right click to add an actor. On the right, we can see its attributes. I can change its name, change its color, or replace it with another 3D asset, using the library browser.

    Notice that each vehicle has its own attributes, like mass, maximum speed, and acceleration. Here, I'm using prebuilt 3D models from RoadRunner asset library. You can also create or import custom vehicle and character meshes.

    The 2D Editor window shows the logic of our actors, using blocks called action phases. This car is initialized to follow its lane at a constant speed for the entire duration of the simulation. Let's add another actor for it to interact with. I'll use a school bus, and I'll add it here.

    Let's modify its action phase so that the bus is initially stopped. I can give this phase a name for my convenience. Now with the 2D Editor, I'll add an action phase after for the bus to start moving at a speed of, let's say 12 meters per second. We can choose its acceleration or specify a particular time duration or distance to reach that speed. By the way, you can use speed and acceleration units of your choice.

    But there's something still missing. The bus needs to know when to start accelerating. In other words, when should it transition from its first action phase to the next one?

    Clicking on the circle to the right of an action phase lets us set conditions for transitioning between action phases. Let's go with a duration of eight seconds. Now run the simulation.

    A crash. The car needs to respond to the stopped school bus. In other words, its actions need to be reactive to the other actors and events in the scenario. I can quickly switch back to the scenario edit tool to modify its logic. Remember, I want the car to come to a stop if it sees the school bus stopped in front.

    Add an action phase after to slow down to zero. Set the condition to when the speed of the school bus is zero. But the car also has to be able to see the bus stopped ahead. We can specify such Boolean conditions in RoadRunner Scenario for greater control over our actor's logic.

    In this case, let's add an and condition for the school bus being within 40 meters of the car. So when both of these conditions are true, the car will slow to a halt. To be safe, the car can simultaneously change lanes slowing down.

    By adding an action phase above or below an existing one, both actions happen in parallel. Let's change lane by one lane to the left. We can do this over second.

    Now once it is stopped, the car should wait until the bus starts moving again. So add a condition to wait until the speed of the school bus is, say five meters per second, relative to the car. We'll set speed sampling to continuous. This means that the condition is checked continuously, rather than just once.

    Then add an action to accelerate to a constant speed. Let's run the simulation once more.

    Notice that the 2D Editor highlights the runtime state of the action phases. And there you go. We've seen how a variety of dynamic logic behavior can be easily specified to build out scenarios.

    Now here, both the actors are following their pre-determined lanes through the scene. How do we specify a custom path? Let's go back to Scenario Edit Tool and start by adding another actor. I'll use a truck and a trailer.

    We can combine these two into a single actor group in order to model a trailer hitch to a truck or a car being towed. With the actor selected, right click to add waypoints along its custom path.

    Optionally, the route timing tool gives us control over the actor's speed and timing over the chosen trajectory. For example, I can slow it down when it's taking a turn.

    Be sure to change the actor's action phase to follow the waypoint time data. If you would rather use recorded or previously generated trajectory data, import it from OpenSCENARIO or CSV files. We will learn about generating scenarios from recorded sensor data soon. Consider this a teaser.

    Now that I have a scenario that I like, I would like to simulate it in a different location within this scene. Doing that is as simple as dragging this anchor icon. RoadRunner Scenarios specifies the position of objects as being relative to points called anchors. An anchor can be a point on a road or junction, a path waypoint, or the location of an actor.

    When we move an anchor, the objects anchored to it move with it. This makes it possible to reuse our scenario by relocating it to a different spot in the scene or to a different scene altogether.

    You can have a hierarchy of multiple anchors in a scenario. For example, a series of vehicles anchored to the one ahead can simulate stop and go traffic.

    The point offsets attributes of an actor help identify or change its anchor, as well as set its location relative to the anchor and the lanes. Thus the attributes specify the scenario across actions, conditions, and actors.

    Now it gets even more interesting. Almost all these attributes can be a variable parameter. Let's create a few. We can do this for our actions and conditions as well.

    Now this enables us to programmatically control these variables, and in turn to rapidly generate variations of our scenarios.

    For example, here, I changed the location of the school bus with a single MATLAB function call. This uses the Automated Driving Toolbox.

    This MATLAB live script opens a different scenario. Then it uses function calls to tweak the scenario to my liking. I'm using interactive controls for a quick demonstration here.

    I can then run the simulation and save it in RoadRunner Scenario or export it to OpenSCENARIO for use in another simulator. You can similarly change scenario attributes, using Python, C++, command line, or any programming language supported by the GRPC framework. Visit the RoadRunner Scenario documentation to learn more.

    Automated Driving Toolbox also provides a co simulation framework for simulating scenarios in RoadRunner, with actors modeled in MATLAB and Simulink. We can define custom behaviors for our actors in RoadRunner using Simulink or a MATLAB system object.

    As a result, you can design and test applications such as a highway lane change planner or autonomous emergency braking in a euro NCAP scenario. Let's see how this works.

    Take, for example, this Simulink model. The RoadRunner scenario block establishes the model interface with a scenario. The reader block reads in the world state, such as information about the actors and actions, while the writer block writes back to the scenario and reports errors and warnings.

    This model reads an actor ID, pose, velocity, and angular velocity values. Then a constant value gain arg is added to the pose value. The values from the arc parameter will govern the motion of the vehicle in RoadRunner.

    To use this model as an actor's behavior, create a new behavior in RoadRunner's library browser. Choose MATLAB/Simulink as the platform, and then specify the model to be used. A similar approach can be used for co simulation with CARLA by selecting the External Platform option.

    Drag the behavior file into the behavior attribute of an actor. Now run the simulation from MATLAB or from within RoadRunner Scenario.

    The RoadRunner Scenario reader block can also read in data from actions, such as speed change or lane change, receive and process user defined actions with custom parameters, for even greater extensibility. You can publish actor behavior as a proto file to combine your behavior model and its parameters to share with RoadRunner. You can then tune these parameters within the RoadRunner environment. Refer to the documentation to learn more about co-simulation, as well as about using actor behavior specified in MATLAB or CARLA.

    So far, we have run scenario simulations with actor behavior defined within RoadRunner Scenario or externally, reused scenarios using anchors, and programmatically generated variations of scenarios. Now we will use real world map and sensor data to generate scenes and scenarios, respectively, with Automated Driving Toolbox in MATLAB. This can save you time recreating real world driving scenarios for simulation.

    Say you want to create a 3D scene of Pikes Peak, Colorado in RoadRunner. Start by importing the geographical coordinates of the road and the elevation data for its terrain.

    We then specify the geographical projection information and up sample the data points. This is essential for accurate scenes, especially when they are bigger or when the data is sparse.

    Notice that the projection information is reflected in RoadRunner. RoadRunner HD map is a road data model for representing high definition map data in a RoadRunner scene. We use the RoadRunner HD map function to create an empty RoadRunner HD map, then programmatically specify the lane boundaries and markings along the road.

    We also specify the latitude and longitude bounds in MATLAB, which, too, is seen in RoadRunner. I could have also specified this within RoadRunner with the World Settings tool.

    Finally, we export the RoadRunner HD map to a file. Dragging the file into a RoadRunner scene builds it out using the scene builder tool.

    Let's also drag in the elevation map. We can continue to build out the scene by adding materials and props, if we so choose.

    You may also have real world vehicle data recorded from sensors like GPS, camera, and LiDAR. Creating a digital twin of such real world scenarios allows you to test your automated driving algorithms against real world data. Let's now see how to do that with the scenario builder add-on from Automated Driving Toolbox.

    We begin by loading in sensor data. In this example, we have GPS and camera data from our Eagle vehicles' perspective.

    Next, crop and offset the data so they share the same time stamps. Raw GPS data can be noisy. So we smooth it out to define the ego trajectory. The non-ego actors are seen from the perspective of the ego actor. So recalculate their trajectories in the world frame.

    You can generate an actor track list by processing raw camera or light or sensor data with Scenario Builder. The documentation includes examples that demonstrate both in more detail.

    With that done, write the trajectories for all actors into a CSV file, which can be imported into RoadRunner Scenario. Finally, we can launch RoadRunner Scenario and simulate the generated scenario. Of course, the scenario can be modified and reused, as we've seen before.

    RoadRunner Scenario supports export to ASAM OpenSCENARIO 1.x and 2.0 standards. This lets you work with external tools, such ES Mini, CARLA, and other simulators that support OpenSCENARIO. MathWorks is an ASAM member and very actively participates in the newly introduced OpenSCENARIO 2.0 standard.

    In summary, with RoadRunner Scenario and Automated Driving Toolbox, you can author and simulate scenarios used for testing automated driving systems, define actor behavior and trajectories within RoadRunner Scenario and by co-simulation with MATLAB, Simulink, or CARLA, programmatically generate variations of scenarios by modifying attributes of actors and their actions, generate scenes in scenarios from real world map and sensor data, respectively, and import from and export to simulators like CARLA, with the OpenSCENARIO standard.

    Here are some suggestions to get started. Browse through documentation, examples, and more videos on the RoadRunner Scenario product page. Most of the demonstrations shown in this video are available as documented examples.

    I highly recommend the application examples with Automated Driving Toolbox, which demonstrates systems like highway lane following and autonomous emergency braking.

    Last, but not least, now that you are familiar with the basics, try out RoadRunner Scenario. If you already use RoadRunner, you just need a license activation to start using RoadRunner Scenario.

    Thank you for your time, and good luck in your driving simulation efforts.

    View more related videos