Main Content

stereoParametersFromOpenCV

Convert stereo camera parameters from OpenCV to MATLAB

Since R2021b

Description

example

stereoParams = stereoParametersFromOpenCV(intrinsicMatrix1,distortionCoefficients1,intrinsicMatrix2,distortionCoefficients2,rotationOfCamera2,translationOfCamera2,imageSize) converts the OpenCV stereo parameters, specified by the input arguments, into a MATLAB® stereoParameters object stereoParams.

The OpenCV spatial coordinate system specifies the upper-left pixel center at (0,0), whereas the MATLAB spatial coordinate system specifies the pixel center at (1,1). The stereoParametersFromOpenCV function compensates for this difference by adding 1 to both of the x and y-values for the converted principal point.

OpenCV stereo parameters cannot be converted to a MATLAB stereoParameters object when:

  • The OpenCV pinhole camera model uses more than five distortion coefficients.

  • An OpenCV fisheye model is used.

In these cases, you can recalibrate the stereo camera using the Stereo Camera Calibrator app.

stereoParams = stereoParametersFromOpenCV(___,WorldUnits=worldUnits) specifies a string worldUnits, that describes the units of the world points in addition to the input arguments from the previous syntax. Specify worldUnits as a character vector or string scalar. For example, stereoParametersFromOpenCV( ___ ,WorldUnits="mm") describes all parameters in terms of the world units "mm".

Examples

collapse all

Load OpenCV stereo parameters from a MAT file into the workspace.

load openCVStereoParameters.mat

Convert the loaded stereo parameters from OpenCV to MATLAB format.

stereoParams = stereoParametersFromOpenCV(intrinsicMatrix1, ...    
    distortionCoefficients1,intrinsicMatrix2,distortionCoefficients2, ...
    rotationOfCamera2,translationOfCamera2,imageSize);

Load the images to rectify.

imageDir = fullfile(toolboxdir('vision'),'visiondata','calibration','stereo');
I1 = imread(fullfile(imageDir,'left','left01.png'));
I2 = imread(fullfile(imageDir,'right','right01.png'));

Display the unrectified images.

imshow(stereoAnaglyph(I1,I2));

Rectify the images using the full output view.

[J1,J2] = rectifyStereoImages(I1,I2,stereoParams,OutputView='full');

Display the resulting rectified images.

imshow(stereoAnaglyph(J1,J2));

The ROS camera calibration package estimates stereo camera parameters using the OpenCV camera calibration tools [1]. After calibrating a stereo camera in ROS, you can export its camera parameters to an INI file using the camera calibration parser. To use the calibrated stereo camera with Computer Vision Toolbox™ functions, such as rectifyStereoImages, you must read the camera parameters from the INI file and convert them into a stereoParameters object using stereoParametersFromOpenCV.

Note: The stereoParametersFromOpenCV function supports importing stereo camera parameters for only those pinhole camera models that use the ROS plumb-bob distortion model.

Read Stereo Camera Parameters from ROS INI File

Read the stereo camera parameters stored in stereoParams.ini using the helper function helperReadINI.

stereoParamsINI = helperReadINI("stereoParams.ini");

Compute Baseline Parameters of Stereo Camera

The baseline parameters of a stereo camera describe the relative translation and rotation of the two cameras in the stereo camera pair. The relative rotation and translation of camera 2 with respect to camera 1 is required to create the stereoParameters object using stereoParametersFromOpenCV. You can compute these from the rectification and projection matrices read from the ROS INI file [2].

Extract the two camera parameters from the stereoParams structure.

cameraParams1 = stereoParamsINI.narrow_stereo_left;
cameraParams2 = stereoParamsINI.narrow_stereo_right;

Extract the translation of camera 2 relative to camera 1 from the last column of the projection matrix.

translationOfCamera2 = cameraParams2.projection(:,end);

The rotation of camera 2 relative to camera 1, R21, is derived from the rectification matrices of the stereo pair R1 and R2. The rectification matrices are the rotation matrices that align the camera coordinate system to the ideal stereo image plane such that epipolar lines in both stereo images are parallel. Compute the rotation of camera 2 relative to camera 1 as R21= R2*R1T.

rotationOfCamera2 = cameraParams2.rectification*cameraParams1.rectification';

Create stereoParameters Object using stereoParametersFromOpenCV

Extract the intrinsic matrices and distortion coefficients of the two cameras from the stereoParams structure.

intrinsicMatrix1 = cameraParams1.camera_matrix;
intrinsicMatrix2 = cameraParams2.camera_matrix;

distortionCoefficients1 = cameraParams1.distortion;
distortionCoefficients2 = cameraParams2.distortion;

Obtain the image size from the image field of the stereoParams structure.

imageSize = [stereoParamsINI.image.height stereoParamsINI.image.width];

Use stereoParametersFromOpenCV to create a stereoParameters object from the ROS stereo camera parameters.

stereoParametersObj = stereoParametersFromOpenCV(intrinsicMatrix1, ...
    distortionCoefficients1,intrinsicMatrix2,distortionCoefficients2, ...
    rotationOfCamera2,translationOfCamera2,imageSize);

Rectify Pair of Stereo Images

Use the imported stereo parameters with rectifyStereoImages to rectify an image pair captured using the calibrated stereo camera.

% Load the image pair.
imageDir = fullfile(toolboxdir("vision"),"visiondata","calibration","stereo");
leftImages = imageDatastore(fullfile(imageDir,"left"));
rightImages = imageDatastore(fullfile(imageDir,"right"));
I1 = readimage(leftImages,1);
I2 = readimage(rightImages,1);

% Rectify the image pair.
[J1,J2] = rectifyStereoImages(I1,I2,stereoParametersObj,OutputView="full");

% Display the results.
figure
J = stereoAnaglyph(J1,J2);
imshow(J)

Supporting Functions

helperReadINI

The helperReadINI function reads the camera parameters from its input INI file that has been exported from ROS.

function cameraParams = helperReadINI(filename)
% helperReadINI reads a ROS INI file, filename, and returns a structure with
% these fields: image, <camera_name1>, <camera_name2>. image is a
% structure describing the height and width of the image captured by the
% cameras of the stereo pair. The fields <camera_name1> and <camera_name2>
% are structures named after the camera names present in the INI file, and they contain
% these fields: camera_matrix, distortion, rectification_matrix,
% and projection_matrix. These fields are stored in the INI file with their
% values placed in a new line followed by their name.

    f = fopen(filename,"r");
    sectionName = '';
    
    while ~feof(f)
        % Read line from file.
        line = fgetl(f);

        % Trim leading and trailing whitespaces.
        line = strtrim(line);
        
        if isempty(line) || line(1)=='#'
            % Skip empty line and comments.
            continue
        elseif line(1) == '[' && line(end) == ']'
            % Identify section names and continue reading.
            sectionName = line(2:end-1);
            sectionName = strrep(sectionName,'/','_');
            continue
        end

        % Replace blankspaces with underscores to create valid MATLAB variable
        % name.
        name = line;
        name(name == ' ') = '_';
        
        % Read the value data in upcoming lines.
        value = [];
        while ~feof(f)
            line = fgetl(f);
            line = strtrim(line);

            if isempty(line)
                % A empty line indicates end of value data.
                break
            elseif line(1)=='#'
                % Skip comment lines.
                continue
            end
            line = str2num(line); %#ok
            value = [value; line]; %#ok
        end
    
        % Store post-processed value.
        if isempty(sectionName)
            cameraParams.(name) = value;
        else
            cameraParams.(sectionName).(name) = value;
        end
    end
    
    fclose(f);
end

References

[1] http://wiki.ros.org/camera_calibration

[2] http://docs.ros.org/en/melodic/api/sensor_msgs/html/msg/CameraInfo.html

Input Arguments

collapse all

Intrinsics matrix of camera 1 from OpenCV, specified as a 3-by-3 matrix of the form:

[fx0cx0fycy001]

where fx and fy are the focal lengths in the x and y-directions, and cx,cy) is the principal point in specified in OpenCV.

Intrinsics matrix of camera 2 from OpenCV, specified as a 3-by-3 matrix of the form:

[fx0cx0fycy001]

where fx and fy are the focal lengths in the x and y-directions, and cx,cy is the principal point in specified in OpenCV.

Camera 1 radial and tangential distortion coefficients, returned as a five-element vector in the form [k1 k2 p1 p2 k3]. The values of k1, k2, and k3 describe the radial distortion and p1 and p2 describe the tangential distortion, specified in OpenCV.

Camera 2 radial and tangential distortion coefficients, returned as a five-element vector in the form [k1 k2 p1 p2 k3]. The values of k1, k2, and k3 describe the radial distortion and p1 and p2 describe the tangential distortion, specified in OpenCV.

Rotation of camera 2 relative to camera 1 from OpenCV, specified as a 3-by-3 matrix.

Translation of camera 2 relative to camera 1 from OpenCV, specified as a three-element vector.

Image size, specified as a two-element vector in the form [mrows,ncols].

Output Arguments

collapse all

Stereo parameters, returned as a stereoParameters object.

Version History

Introduced in R2021b