How to Perform Data Labeling for Camera and Lidar Sensor Data - MATLAB
Video length is 6:04

How to Perform Data Labeling for Camera and Lidar Sensor Data

Data labeling with the Ground Truth Labeler app enables you to label camera and lidar data. Using the app interface, you can label data manually or use algorithms in MATLAB® to automate the data labeling process. Once the process is complete, you can use the data to train deep learning and machine learning models or to validate ADAS algorithms.

Published: 17 Jun 2018

Hey, everyone. This is Pitambar from MathWorks. Today I'm going to show you how to label your data using the ground truth label or app in MATLAB. OK, let's get onto the video. First, we're going to open the app. So I'm going to go to the Apps tab and scroll to the automotive section. There's a bunch of different apps, but let's choose the ground truth labeler.

Now, we're going to click Open and hit Add signals. Here, we have the option to load in signals from different sources. Video, images, point clouds, and more. So let's load in some video data. I'm going to hit Browse, and load in a clip that ships with automated driving tool box.

Next, I'll add in a point cloud sequence. I'll change the source type to point cloud sequence, and load in the PCD files. I'll use time stamps from my workspace that correspond to this file. So I'm only going to load in two signals, but it's worth noting that you can load in multiple. Let's take a look at the two signals I selected.

As we can see, the video shows our ego vehicle approaching a car at a stoplight. If we switch to the point cloud data, this shows the same car movement but in point cloud form. If we wanted, we could actually view these simultaneously by putting them in grid mode. And when I hit Play, we watch as both of these sequences play at the same time. Let's go back to our nongrided view and talk about labeling our data starting with the video.

On the left hand side, I have the option of creating ROI labels, or region of interest labels. I'll create one for vehicle and I'll draw a bounding box around the cars that I see. Within my label, I have the option of creating sub labels. So here I'll create one to indicate the taillights on the car.

We can also give attributes to our labels and sub labels. So let's define an attribute for the vehicles to indicate the type that we're looking at. The one on the right, I'll label it a sedan, and the one on the left as a minivan. Now, when we create labels we're not limited to bounding boxes. Let's create a new ROI label, and this time I'll hit the drop down menu.

You'll see that I can create ground truth data with line labels and pixel labels as well. So let's create an ROI label called lane boundaries. We'll use the line label and define them directly in our image. Finally, we have the option to define a sin label. Let's create a sunny scene label.

So you have the option to use the scene label for a frame or part of the video. In this case, it's sunny the entire video, so I'm going to use it for the whole clip. Now, let's jump over to our Lidar data to see how we can label it. First, I'll show you a visualization option. I'll start by hiding the ground so we can see the car more clearly. Now, if I want to toggle my view I have these three options here.

So for this video, I'm going to work in the default view labeling, and let's talk about that. We can label these images using cuboid bounding boxes. We select the vehicle label and a box appears in our Lidar window. Now, the part that's going to be labeled is highlighted in yellow, and when I click it snaps a box around this highlighted portion. Once I have a box in place, I can adjust it from the sides to fine tune the fitting. But the key is whatever is highlighted in yellow is going to be part of our label.

That wraps up the basics of labeling an individual frame. Let's talk a little bit about automation. So you'll notice that if I move to the next frame, all of my labels are gone. If I had to label every single detail in every single frame manually, I would just call my friends and family and say, hey, guys. I'll see you in 10 years because it would take forever. But luckily, we have automation.

So if we go to the automate section of our labeling toolbar, we see an option to select algorithm. The dropdown shows a handful of built-in algorithms. So you can pause the video and take a look at the details, but I'm going to use this one. It's a point tracker. I'll click the option and hit automate.

Now, the right hand side here has instructions on how to use the algorithm. Let's go through it. I'll draw a bounding box around all of the objects that I want to track in my first frame. Now, when I hit Run, the algorithm will track this box across the time frame. Within these labeled frames, you can individually adjust the bounding boxes. And once I'm happy with the results, I'll click Accept.

So let's just verify that the results are there. I can scrub through the timeline like this or I can view labels summary. So as I showed before automation extends beyond just this algorithm. You can actually create your own algorithm. When you click Create New algorithm, you get a class template which you can modify to create a custom algorithm.

So you've labeled your data, it's time to export. Up here, go to Export labels and you can export it to a file or in the workspace. A quick look at our ground truth object shows the properties that we would expect to see: the data source, label definitions, ROI labels, and c labels.

And that's where I'm going to wrap up this video. If you'd like to learn more about ground truth labeler, take a look at our documentation. I based this video entirely off of one of our doc examples. So if you have automated driving toolbox, you can try all of these steps out for yourself. Thanks for tuning in, and I'll see you in another video.

Related Products

View more related videos