This example shows you how to capture and process images from a Raspberry Pi Camera Module V2 connected to the NVIDIA® Jetson Nano using the
GPU Coder™ Support Package for NVIDIA GPUs. The GPU Coder Support Package for NVIDIA GPUs allows you to capture images from the Camera Module V2 and bring them right into the MATLAB® environment for processing. In this example you will learn how to develop a Sobel edge detection algorithm by using this capability.
Target Board Requirements
NVIDIA Jetson Nano embedded platform.
Raspberry Pi Camera Module V2 connected to the CSI host port of the target.
Ethernet crossover cable to connect the target board and host PC (if the target board cannot be connected to a local network).
V4L2 library on the target.
GStreamer libraries on the target.
Environment variables on the target for the compilers and libraries. For information on the supported versions of the compilers and libraries and their setup, see installing and setting up pre-requisites for NVIDIA boards.
Development Host Requirements
GPU Coder for code generation. If you are new to GPU Coder, visit the GPU Coder product page for an overview and tutorials.
GPU Coder Support Package for NVIDIA GPUs. If you are new to this support package, we recommend completing the Getting Started with GPU Coder Support Package for NVIDIA GPUs example.
NVIDIA CUDA® toolkit on the host.
Environment variables on the host for the compilers and libraries. For information on the supported versions of the compilers and libraries, see Third-party Products. For setting up the environment variables, see Environment Variables.
Connect to the NVIDIA Jetson Nano
The GPU Coder Support Package for NVIDIA GPUs uses an SSH connection over TCP/IP to execute commands while building and running the generated CUDA code on the DRIVE or Jetson platforms. You must therefore connect the target platform to the same network as the host computer or use an Ethernet crossover cable to connect the board directly to the host computer. Refer to the NVIDIA documentation on how to set up and configure your board.
To communicate with the NVIDIA Jetson Nano, you must create a live hardware connection object by using the drive or jetson function. You must know the host name or IP address, username, and password of the target board to create a live hardware connection object. For example, use the following command to create live object for Jetson hardware,
hwobj = jetson('jetson-nano-name','ubuntu','ubuntu');
getCameraList function of the
hwobj object to find the available cameras. If this function outputs an empty table, then try re-connecting the camera and execute the function again.
Use the coder.checkGpuInstall function and verify that the compilers and libraries needed for running this example are set up correctly.
envCfg = coder.gpuEnvConfig('jetson'); envCfg.BasicCodegen = 1; envCfg.Quiet = 1; envCfg.HardwareObject = hwobj; coder.checkGpuInstall(envCfg);
Create a camera object using the name from the
camObj = camera(hwobj,"vi-output, imx219 6-0010",[640 480]);
camObj is a handle to a camera object. To display the images captured from Camera Module V2 in MATLAB, use the following commands.
for i = 1:100 img = snapshot(camObj); imagesc(img); drawnow; end
This camera object captures RGB and 3-channel gray scale images.
imageDisplay function to create a display object. This is a system object that uses
imshow function to display the images in MATLAB.
dispObj = imageDisplay(hwobj); img = snapshot(camObj); image(dispObj,img);
The Sobel edge detection algorithm is a popular yet simple edge detection algorithm. In this algorithm, a 2-D spatial gradient operation on a grayscale image is performed. This operation emphasizes the high spatial frequency regions which corresponds to edges.
Calculate the Gradients
We will find horizontal gradient(h) and vertical gradient (v) of the input image with respective Sobel kernels. These two Sobel kernels are orthogonal to each other. We will make sure our algorithm works on the test image before moving on to live data.
kern = [1 2 1; 0 0 0; -1 -2 -1]; img = imread('peppers.png'); imagesc(img); h = conv2(img(:,:,2),kern,'same'); v = conv2(img(:,:,2),kern','same');
Calculate the Gradient Magnitude
Next we find the gradient magnitude from the horizontal and vertical gradients (h and v).
e = sqrt(h.*h + v.*v);
Threshold the Edge Image
We threshold the image to find the regions of image that we consider to be edges.
edgeImg = uint8((e > 100) * 240); imagesc(edgeImg);
We can create a MATLAB function,
sobelEdgeDetectionAlg.m, out of the MATLAB code we developed in the previous sections of this example. View the MATLAB function in the editor.
sobelEdgeDetectionAlg() takes image and threshold input for edge detection and returns the results of edge detection algorithm. We will call this function on the images captured in a loop. The threshold variable
thresh can be varied to get a proper edge image. This way we can use the camera access capability of the support package to tune the algorithm suitable for the specified camera.
for i = 1:200 img = snapshot(camObj); thresh = 100; edgeImage = sobelEdgeDetectionAlg(img, thresh); image(dispObj,edgeImage); end
To deploy the above example as a stand-alone application on the target, follow the example Deploy and Run Sobel Edge Detection with I/O on NVIDIA Jetson Nano.
This example introduced an application where images coming from a camera connected to an NVIDIA Jetson Nano are processed in MATLAB using a Sobel edge detection algorithm.