Programming Robots with ROS and ROS2 using MATLAB - MATLAB
Video Player is loading.
Current Time 0:00
Duration 36:00
Loaded: 0.46%
Stream Type LIVE
Remaining Time 36:00
 
1x
  • Chapters
  • descriptions off, selected
  • captions off, selected
  • en (Main), selected
    Video length is 36:00

    Programming Robots with ROS and ROS2 using MATLAB

    Overview

    Robot Operating System (ROS) has become increasingly popular in industry, especially in the development of robotics. ROS, however, is a large robot programming framework, encompassing robot development tools, ecosystem, and distributed computing systems. It requires a steep learning curve, and it is hard to integrate into an existing non-ROS based stack.  Handwriting code for ROS requires using ROS’s package build system and compiling against its dependencies.

    This webinar discusses ROS and ROS 2 support from MATLAB and benefits of using MATLAB in programming robot applications.

    Highlights

    Through several examples, we will cover:

    • MATLAB and Simulink ROS and ROS 2 network interface
    • How to leverage built-in functionality in MATLAB toolboxes related to robot programming
    • Automated C++ node generation from MATLAB and Simulink
    • CUDA-compatible ROS node generation with embedded deep learning inference targeted for NVIDIA hardware
    • Recording, importing, and visualizing ROS data (rosbag files) to analyze and debug ROS-based applications

    About the Presenters

    YJ Lim is a Principal Technical Product Manager of robotics and autonomous systems at the MathWorks. He has over 20 years of experience in robotics and autonomous systems area. Lim’s responsibility in MathWorks includes long-term strategy development and product management of robotics and autonomous systems. Before joining MathWorks, Lim worked on various projects at Vecna Robotics, Hstar Technologies, and Energid Technologies. Lim received his Ph.D. in mechanical engineering from Rensselaer Polytechnic Institute (RPI) and his Master from KAIST in S. Korea. 

    Murat Belge is a consulting engineer at the MathWorks. He is the technology team lead for ROS Toolbox. He worked in MathWorks for over 15 years in various areas including code generation for embedded systems, networking protocols, middleware representation and integration, IOT, device driver abstraction and integration, targeting embedded Linux systems, embedded vision and motor control. His experience includes developing embedded vision algorithms for SIMD processors and developing time domain reflectometry diagnostics systems for digital subscriber lines.  Belge received his Ph.D. from Northeastern University, M.S. and B.S. from Bilkent University in Turkey, all in electrical engineering. 

    Recorded: 28 Mar 2023

    Hello and welcome. Everyone. My name is YJ Lim. I am the Technical Robotic Product Manager at MathWorks. Today, I am delighted to be presenting with Murat Belge. We have an exciting topic to discuss, which is a programming robot with ROS and ROS 2 using MATLAB. Throughout the section, we will dive deep into the detail of this powerful programming environment and show you how to use it to program lower.

    Before we start, let us introduce ourselves. I'm the product lead of the robotics and autonomous systems at MathWorks based in Natick, Massachusetts. With over 20 years of experience in developing robotics and autonomous systems, I have worked on several robotics projects at the companies such as Vecna, Hstar, Energid, and GM R&D center.

    I had been with The MathWorks for about five years now, where I am responsible for managing a couple of robotic products and developing product strategy. Murat, would you introduce yourself?

    Sure, YJ. Hello, my name is Murat Belge. I am a consulting engineer at MathWorks, Natick. I have been with MathWorks for 19 years. During my tenure at MathWorks, I worked on various aspects of automatic code generation, including scheduling, device driver, block development, middleware support, method targeting, and so on. My prior experience includes image and signal processing algorithm development for embedded systems. Over to you, YJ.

    Thank you. To begin with, I will introduce what ROS is and why it is important for robotic system development. Next, we will explore the capability of the ROS that are supported by MATLAB. We will also take a closer look at the workflow of the ROS with MATLAB and Simulink. By end of this section, you will have a solid understanding of how to use ROS and ROS 2 with MATLAB and Simulink.

    So let's get started. Let us begin by introducing ROS briefly. What is ROS? At its core, ROS provides developers with well-established tools for the messaging and distributed system development. This includes an extension packaging and a builder management system.

    The ROS user community is vital because they create application packages, drivers to talk to the hardware, interface to the other software packages, and robotic-specific tool for the task such as motion planning and the perception.

    However, the single point of failure to the ROS 1 is plumbing. So ROS 2 is the new version of ROS 1, designed to fix this reliability issue of the ROS 1 by using the DDS communication standard. This makes ROS 2 surer for both research and production environment.

    ROS has its advantages and disadvantages. On the positive side, it is a modular and distributed architecture, making it easy to develop and manage large and complex robotic systems. ROS has a large and active community of developers and user who contribute the development of the software.

    ROS also provide a rich set of tools and library for the robot development. On the other hand, it require a steep learning curve for the beginners. It is difficult to integrate into an existing non-ROS-based based stack. Complexity is also an issue in the design and functionality, which may require more expertise and development time to work with it.

    And then lastly, there is a lack of standardization, which can make it challenging to ensure compatibility between different ROS-based system. MATLAB and Simulink can complement ROS functionality and address some of each weakness.

    I want to start with what we heard from our customer while using ROS. The customers, such as Voyage, developing self-driving cars, and Yaskawa, making the robot manipulator, has seen the value in combining the strengths of ROS and MATLAB toolboxes.

    They have been able to complement ROS solution with MATLAB and Simulink tooling to enhance their development workflow. As ROS become more popular in industry, even engineers with limited coding background-- such as mechanical or control engineers-- are interested in using ROS for the prototyping and validating their robot applications.

    The powerful tools and intuitive interface of the MATLAB and Simulink can make it easier for these engineers to work with ROS and develop digital robotics systems. Here is how we support ROS and ROS 2 under the hood of MATLAB API. As you can see, MATLAB has a full implementation of the ROS, which allows for the seamless integration with ROS-based system.

    For the ROS 2, we offer options choosing from various DDS with a higher performance, such as latency. This makes it easier for the user to work with ROS 2 and take advantage of its improved capabilities.

    MATLAB and Simulink provide a powerful set of tools for modeling, simulation, and analysis. As you saw in the previous slide, since MATLAB and Simulink has built-in support for ROS, within one integrated development environment, you can create once a node and load ROS data into ROS spec file while you are simulating and testing robotics applications.

    MATLAB also provides a seamless interface to connect to an existing ROS network or create a ROS network with a desktop simulation and hardware interface with MATLAB and Simulink. In the right-hand side in this slide, this is how a traditional ROS-enabled system might look like.

    This involves a host of ROS-enabled simulator or hardware that communicate with the node through the ROS network, enabling autonomy, visualization, and more. MATLAB with a ROS Toolbox offer three workflows.

    The first is offline data analysis with rosbag file. You can use rosbag import and playback function to bring simulation data or hardware data into the MATLAB and Simulink or the offline design task. There are new functionality that we recently introduced, so we will cover them later this webinar.

    Next is desktop simulation. This involves a live connection between ROS and an instance of MATLAB for the ROS-based system development. And finally, deployment workflow. This allows you for the automatic generation of standalone C++ ROS nodes. So now we no longer require MATLAB to be executed.

    MATLAB and Simulink tooling can complement traditional ROS solution to enhance functionality. Later in this webinar, we will dive deeper into each workflow. In here, I will show you a simple example that use ROS with Simulink model to control a robot.

    For autonomous system development like low band system might have functional component of perception, planning, and control. With the use of ROS, you can design a test of simulation to test and verify new robotics algorithm.

    This is a simple ball-tracking robot example with the simulator using ROS 2 by control algorithm designed in Simulink. One thing to note here, which is important, you see these blocks that I highlight which include ROS in it? These blocks are ROS blocks coming from the ROS toolbox to read the data, subscribe topic, as well as create a message and publish ROS topics. These blocks allow you to communicate with ROS without writing any code.

    We will now explore the ROS and ROS 2 workflows when used with MATLAB and Simulink. The first workflow involves offline analysis using rosbag file. This enables you to store ROS topic in the log file and easily access and filter information from this file in The MATLAB and Simulink.

    For both ROS and ROS 2, MATLAB now has rosbag writer and reader function to write logos to rosbag file and read these log files and help with filtering and extracting message data. Additionally in the Simulink, you can playback ROS and ROS 2 bag file based on current simulation time.

    You can use ROS logger to record ROS message during the Simulink simulation. Murat will now provide a detailed demonstration of how to perform offline analysis with rosbag file.

    Thanks, YJ. MATLAB has supported reading rosbag files for a while. The ability to read rosbag files and work with specialized ROS message types for sensors allows you to bring in your design data into MATLAB and Simulink without the need for preprocessing. We made sure that the data read from the rosbag file can easily be fed into functions that are available in other toolboxes.

    The example here is from Sensor Fusion and Tracking Toolbox. Here's a quick demo. By using the MATLAB APS, I'm going to create a ROS 2 bag file and add some data points and read it back. So I just create a bag writer and create a message. This is a ROS 2 message of type geometry_msgs/Point. I set the XYZ coordinates arbitrary numbers and write it to the bag file. Note to timestamp, I am using the current time.

    I do the same process again. So we have two data points in the rosbag file. And then we finally close the file. Now I'm going to create a reader object. I'm opening the same rosbag files. And I'm going to read all of the messages within the files. So this will return you a 2 by 1 cell array. The first is the first data point we recorded and the second indexed element is the second data point recorded.

    It is that easy to basically create rosbag files and then read it back in MATLAB and Simulink. ROS Toolbox also gives you the capability to log Simulink signals in a rosbag file. We have a Simulink app called ROS Logger to help you with logging data. This app allows you to choose message topics and gives you a couple of nodes to optimize rosbag creation. You can save simulation results as a rosbag file for analysis, testing, and verification.

    I'm quite excited to tell you that we have a new app called rosbag Viewer in R2023a that allows you to visualize messages in a rosbag file. You can immediately see images, point clouds in a rosbag file with just a few clicks, analyze time series data, and visualize time series data as XY plots. The app provides real-time playback, fast forward and reverse, and ability to jump anywhere within recorded timeline through a playback scrubber.

    Here I am demonstrating the basic capabilities of the app. We are opening a rosbag file. On the left-hand side, you can see the topics that are available here, number of messages, and the rate that they appear in the file. I can choose various visualizers and then view the images, lidar point clouds and whatnot within the file.

    I can do XY plots within the data. So I can plot one element of a message against another element of the message, and I can basically review the time plot of the data. I can arrange all of the data in one continuous tile, and then I can do playback on that data-- first the regular playback, and I can do fast forward and fast reverse, which is very convenient for analyzing data.

    Note also that we have the capability to display the raw content of the message in the middle window. Now back to you, YJ.

    Thank you, Murat. The second workflow is desktop simulation, which enables MATLAB or Simulink to interface with external simulator or hardware through a live connection. With this live connection, MATLAB Simulink user can connect to external sensor, devices, simulator, or ROS development ecosystem tools.

    In another way, ROS user can find the value of combining the strengths of the MATLAB toolboxes and ROS to enhance their development process. You can connect MATLAB and Simulink with low-cost hardware to stream sensor data. You can also utilize ROS to control robot hardware through the use of ROS driver.

    A variety of simulator provide ROS plugins. By integrating with these ROS and a robotic simulator, you can perform pitch-based and sensor simulation for the robotics and system development. This integration allows MATLAB Simulink to use a sensor input and provide a control algorithm with capabilities such as image processing, computer vision, and deep learning.

    However, many robotics simulators do not provide an explicit time synchronization schemes, which means that time synchronization becomes a crucial issue for the closed-loop core simulation in order to make sure determinism. So Murat will now introduce you how to perform core simulation with the Gazebo simulator or other simulator.

    Thanks, YJ. You can perform core simulation with Simulink and Gazebo with time synchronization. Typically, you read sensor data from Gazebo and feed it to Simulink-- an algorithm running Simulink. You can then send commands back to Gazebo to complete the loop. Both Gazebo and Simulink are time-based simulators. The execution happens in lockstep, and time synchronization is maintained across simulators, which means Simulink takes one time step and the Gazebo takes the exact same time step in unison.

    There are multiple ways to use Gazebo from MATLAB and Simulink. ROS provides a standard interface to use Gazebo through ROS network interfaces such as publisher subscribers, services, and parameters. Robotic System Toolbox supports a direct interface with Gazebo without ROS for deterministic simulation. You can use these two interfaces together for time-synchronized simulation. You can, for example, use Gazebo Pacer block from Robotic System Toolbox in a ROS Simulink model.

    So how about other simulators for time-synchronized simulation? Simulink offers a simulation pacing feature that allows you to control the simulation time with respect to wall clock. You can use simulation pacing to approximately synchronize the simulation time of Simulink with data for an external simulator. And using Simulation Pacing, note that both Simulink and external symmetries running in their own timelines. We are simply trying to ensure that time in a Simulink environment approximately flows at the same rate as an external simulator.

    For this to work, you must ensure that the algorithm and Simulink completes within the specified wall clock time in Simulation Pacing GUI. This is not exactly lockstep simulation, but works surprisingly well with gaming engines such as Carla, Unreal Engine, or Unity.

    You may ask, how about time synchronization with deployed ROS nodes? How does that work? In a ROS network, you can run all nodes off of an artificial clock. Typically, one node publishes time to a clock topic and the other nodes advance their time by reading the value in clock topic instead of the system time. This allows all nodes to synchronize themselves to a master clock.

    This is very useful when working with rosbags. We offer a ROS time stepping feature whereby a deployed Simulink model clock topic as the time source. You can use this feature to synchronize time in a deployed node with data of an overall system. Back to you, YJ.

    Thank you, Murat. The third workflow is an automated C++ ROS node generation. MATLAB and Simulink has code generation capability. With this, you can generate efficient, consistent, and portable code for the development once the system design is working as intended.

    This approach minimize error in hand-coding and reduce the time to the market for the systems development. By code generation, you can allocate more time on the design and the algorithm development rather than coding.

    The basic idea behind automated code generation is you write the code in a high-level language and code generator takes that high-level language and translate it to our general-purpose programming language. The MathWorks code generator provide support generating C, C++, HDL, and GPU code. Additionally, you can generate C++ code that is compatible with industrial standards such as AUTOSAR Adaptive and DDS.

    So here, when we talk about ROS node generation, there are three options. The firstly, you can choose to only create one package. This option will allow you to review the generated code and make any necessary customization, if needed.

    Secondly, you can opt to automatically create and compile the ROS package. This option is a good choice when you want to manually put generated ROS node to the remote ROS device, or when you have another note to compile and edit together, or when your robot is not computer yet to learn ROS node.

    Finally, you can generate as well as the launch the ROS node on the target system. This option allows you to speed up the simulation speed or you can run a standalone ROS node without the need for connection to them in a hosted machine. So Murat will provide more details on the process of ROS nodes generation from MATLAB and Simulink.

    Thanks, YJ. Now I'd like to provide a somewhat simplified overview of MATLAB and Simulink-based workflows. In both cases, you take a high-level implementation of your ROS node and deploy it to hardware via automatic code generation. In one case, the programming medium is textual, while in the other, it's graphical.

    One key difference is that the MATLAB code is event-based while the Simulink code is time-based. You can use Stateflow charts in both MATLAB code and Simulink models. Simulink also allows you to take advantage of System Composer to design interface and components of your system. Both support processor-in-the-loop profiling, but Simulink also supports external mode features, which is really convenient for monitoring a ROS node in real time.

    Finally, you can embed MATLAB code to your Simulink models through MATLAB function block. We will see an example of that later on. You can generate a ROS 2 node from either MATLAB or Simulink. The MATLAB code on the left shows a typical subscriber node. We subscribe to the location topic and print out the XYZ coordinates in the received message.

    The Simulink model on the right does essentially the same thing. The fundamental difference is that Simulink, we poll for this. So I'm going to talk about this in a second. You can run both examples directly in MATLAB environment. This is called interpreted mode or simulation mode. You can also generate code and run the generated code as a standalone C++ ROS node.

    Now I'm going to make a quick demonstration on how you can deploy the code on the left to your local computer. Here's an example of deploying MATLAB code as a C++ ROS 2 node. I'm going to deploy my node at hand that I just showed in the previous slide. First, we create a test node, which we will be using in subsequent operations.

    We then create a configuration object that tells MATLAB Coder to generate an executable target ROS 2 and automatically build and run the generated code on the local host. After creating the configuration object, I can kick off the code-generation process by executing the codegen command.

    After a while, the generated node is compiled and it's launched outside of MATLAB, and the command window pops up showing the console output. Now I'm going to create a publisher within MATLAB and send messages to the running node. As you can see, the node receives and prints out the message content.

    As soon as we execute the send, you'll see that the node that's running standalone-- that's outside of MATLAB-- received the message and printed it out. You'll also recognize some of the printout that's coming from the node is actually what we put into MATLAB code.

    So this provides a very easy workflow for generating code out of your MATLAB scripts and deploying it onto a remote device and in this case, your local host computer. And this is furthermore supported in all platforms, meaning that you have support for this exact scenario for your host computer in Windows, Linux, or Mac platforms.

    Now we repeat the same process in Simulink. Here we have a Simulink model which does the same thing as the MATLAB code I presented a minute ago. The subscriber block is configured to receive messages from location topic. Note that the block checks for a new message every 0.001 seconds or 100 times a second. If it receives a message, the enabled subsystem is activated, and we should see a printout on the console.

    Let's open the ROS Toolbox Simulink app and set ROS network to ROS 2 and choose local host as the deployment target. Note that the Simulink has a configuration object similar to MATLAB, which can be accessed via hardware settings. Here you see the configuration object represented as a GUI in Simulink.

    Now to device settings. We are targeting our host computers, which are the Windows 64 operating system. Now let's go to deploy tab and click on the build and run to initiate code generation, Simulink called the compiler to generate code and launch the ROS node. I can then send a message from MATLAB to the running node. Observe that the node successfully received messages and prints them out at the console window.

    As soon as we click on Send, you'll will see the message printed out on the console output. So these two examples illustrate the code generation process from MATLAB or Simulink. And the process is pretty similar in MATLAB. You have mCode. You generate a configuration set for your MATLAB coder. And then you basically feed that into a command, which is called codegen, which generates the compiled ROS node, which is a C++ ROS node.

    And in Simulink, you have the similar thing. You have your canvas-- a Simulink canvas. You have a configuration set that's attached to the Simulink. You basically click on the build and deploy and then out comes to C++ code, which is compiled and then enlarged-as a separate ROS node.

    As a bonus, you can incorporate deep neural networks into your ROS nodes. We make it easy to generate CUDA-enabled ROS nodes for deployment to an Nvidia device such as NVIDIA Jetson. You can also generate and run the CUDA ROS node on the host computer, provided that you have an Nvidia GPU card installed. Please note that we are working with the NVIDIA hardware support package team to make sure that the hardware clocks that are shipped in that support package work with the ROS Toolbox.

    A recurrent question that we get is, how do you model multiple nodes in your design and development in Simulink? To answer that question, I'd like to mention that we have an example called the automated parking world example that shows how Simulink can be used to model any application which comprises of multiple nodes.

    You partition the design into components represented as model references. Each model reference is an independent model on its own, but can also be integrated into a larger Simulink model. Here we see behavioral planner, path planner, controller, and vehicle components.

    We can deploy behavioral planner, paired planner, and controller components as separate ROS nodes in a distributed system. The vehicle component is typically not deployed, and it's either replaced by simulator such as Carla or Gazebo, or replaced by an actual vehicle dynamics model. When deployed, each of these nodes use ROS network to exchange data, while in simulation, they use Simulink signals.

    This is an important distinction. This provides deterministic and repeatable results in simulation environment. While in deployment, because of the network latencies, the effects of processor are in the time that it takes to process the data and whatnot, you might get different results from one run to the other. Whereas in simulation, you can pass the signals around using Simulink signal lines, and this allows you to do deterministic simulations every time. Now, back to you, YJ.

    Great Thank you, Murat. So we have now in the generated ROS. Hey, Murat. This is my last question. What are the different ways one can communicate with generated node?

    Yes, there are several ways you can work with a node from within MATLAB environment. Here on the right, we have a generated ROS node. First, you can use a ROS device object. Second, you can use MATLAB APIs for standard ROS constructs such as parameters, pops-up messaging, services, or actions. Or you can use external mode to tune parameters and visualize data directly in a Simulink model.

    Let's explore these options in more detail. By using a ROS device object, you can create a connection with a ROS device, then start and stop a ROS node on the connected ROS device and execute system commands. The ROS device can be a remote device or the local host.

    A quick demonstration. A couple of slides earlier, I showed you how to generate ROS nodes from MATLAB or Simulink code. Now I'm going to use ROS 2 device object to start and stop the generated nodes and pass some messages from MATLAB to the running nodes.

    First, I create a ROS 2 device object with a special keyword, "localhost." When ROS 2 device sees localhost is the input parameter, it does not try to establish an SSH connection. It just returns an object that you can use to execute commands, start and stop nodes on your host computer. The returned object also shows the number of available nodes that's found in the current workspace.

    Note that in this case, we are seeing two nodes available in the ROS 2 workspace. Let's launch the first node, myNode, by using the Run Node command. Now I'm going to start the other node. Recall that both of these nodes receive messages from the location topic and print out XYZ coordinates stored in the message.

    I ran the second node and I am positioning the command window so that we can see the messages printed out on the console windows. And I'm going to create a publisher from the MATLAB. Oops, we need a node. So I create a test node. And then let's try to create the publisher again.

    All right, so this time, it went through. Send a message. As soon as we send the message, we see that it's being published in both the Simulink-generated code and the MATLAB-generated node. And we assume that they both respond almost simultaneously.

    To stop the node, we use the stopNode command line API. I am stopping the MATLAB node first, or the node generated from MATLAB, and then the node generated from Simulink.

    I previously touched on pops-up messaging and shown an example where I send messages to running ROS nodes. You can also use ROS parameters, services, and actions to communicate with nodes in the ROS network. For example, by using ROS param utility, you can conveniently modify tunable parameters in your deployed algorithm. This lets you make runtime changes to your algorithm configuration from within MATLAB without the need for recompiling compiling the code.

    In Simulink, you can use external mode to modify what we call as tunable parameters in a model. These are, for example, the gain value in a gain block or the constant value in a constant block. External mode allows you to change the gain value from Simulink canvas interactively without executing any code. In external mode, the Simulink acts as a cockpit that allows you to monitor and tune the application locally, even though the code itself is running on the target device.

    External mode can be an invaluable debugging tool while you are still designing your algorithm. With that, I'd like to conclude my section of the presentation. And I'd like to hand over the presentation to YJ for conclusion.

    OK, thank you, Murat. So this brings us to the end of today's webinar. Let me summarize what we discussed. Today, we discussed how MATLAB and Simulink can be used in conjunction with ROS to improve robot development and control.

    We cover three main workflow-data analysis using rosbag, desktop simulation and prototyping, and the deployment workflow. Specifically, automated C++ ROS node generation allow you to speed up the development process and remove the manual implementation errors.

    Using MATLAB and Simulink, you can process directly from algorithm development to implementation without hand-coding. We also discussed some of the challenges involved in using simulator with ROS, such as the time synchronization issue, and how to overcome them.

    Overall, what we learned is that by combining the strengths of ROS and MATLAB Simulink, developers can accelerate the development and the testing of robotics algorithm, and ultimately bring the developer's robot to the market faster.

    I encourage you to download the trial to check out the example on the ROS and automate the ROS node degeneration from MATLAB and Simulink. To learn more on The MathWorks robotics tools and the ROS capability, please visit our product web page and the robotics solution page. You can also check out our MATLAB and Simulink robotic GitHub repository, which contains a wealth of resource and example to assist you in your development effort.

    Additionally, we have published numerous example. We have published numerous examples on our web page to help accelerate your development process. Should you have any question or require support with you special use cases, please don't hesitate to reach out to us. We are always happy to assist you.

    Thank you for your attention. So now please post your question in the Q&A panel. We will take a few moments to review them and then come back online to answer your question.

    View more related videos