Using the Single Camera Calibrator App

Camera Calibrator Overview

You can use the Camera Calibrator app to estimate camera intrinsic parameters. You can use these camera parameters for various computer vision applications. These applications, such as removing the effects of lens distortion from an image, measuring planar objects, or reconstructing 3-D scenes from multiple cameras.

Use this workflow to calibrate your camera and to follow best practices to prepare and capture calibration images and to evaluate calibration accuracy.

1. Prepare Camera and Capture Images For Camera Calibration

2. Add Images and Select Camera Model

3. Calibrate

4. Evaluate Calibration Results

5. Improve Calibration

6. Export Camera Parameters

The Camera Calibrator app incorporates a suite of functions to implement the camera calibration workflow. You can use these functions directly in the MATLAB^® workspace. For a list of these functions, see the Camera Calibration topic.

Choose a Calibration Pattern

The Camera Calibrator app supports checkerboard, circle grid, AprilTag, ChArUco, and custom detector patterns. For details on each of these patterns and PDF files containing printable patterns, see Calibration Patterns. The app can calibrate cameras with a field of view (FOV) of up to 195 degrees.

Capture Calibration Images

For best calibration results, use between 10 and 20 images of the calibration pattern. The calibrator requires at least three images. Use uncompressed images or lossless compression formats such as PNG. The calibration pattern and the camera setup must satisfy a set of requirements to work with the calibrator. For more details on camera setup and capturing images, see Prepare Camera and Capture Images For Camera Calibration.

Using the Camera Calibrator App

Open the App

• MATLAB Toolstrip: On the Apps tab, in the Image Processing and Computer Vision section, click the Camera Calibrator icon.

• MATLAB command prompt: Enter cameraCalibrator

Add Images and Select Camera Model

To begin calibration, you must add images. You can add saved images from a folder or add images directly from a camera. The calibrator analyzes the images to ensure they meet the calibrator requirements. The calibrator then detects the points of the selected pattern. For details on camera setup and capturing images, see Prepare Camera and Capture Images For Camera Calibration.

 Add Images from File

On the Calibration tab, in the File section, click Add Images, and then select From file. You can add images from multiple folders by repeating the process for each folder.

 Acquire Live Images

The Camera Calibrator app works with UVC compliant webcams. You can acquire live images from a webcam using MATLAB Webcam support. To use this feature, you must install the MATLAB Support Package for USB Webcams. See Webcam Acquisition Overview. To add live images, follow these steps.

1. On the Calibration tab, in the File section, click Add Images, then select From camera.

   This opens the Camera tab. If only one webcam is connected to your system, the app selects it by default and a live preview pane opens. If you have multiple cameras connected and want to use one other than the default, select that camera in the Camera list.

2. (Optional) Set properties for the camera to control the image. Select the Camera Properties to open the Camera Properties dialog box for the selected camera. The available properties vary depending on your device.

   Use the sliders and lists to change the available property settings. The preview pane updates dynamically when you change a setting. When you are done setting properties, click anywhere outside of the dialog box to dismiss it.

3. Enter a location to save the acquired image files in the Save Location box. You can type the path to a folder or use the Browse button. You must have permission to write to the folder you specify.

4. Set the capture parameters.

   • To set the number of seconds between image captures, use the Capture Interval (sec) box or slider. The default is 5 seconds, the minimum is 1 second, and the maximum is 60 seconds.

   • To set the number of image captures, use the Number of images to capture box or slider. The default is 20 images, the minimum is 2 images, and the maximum is 100 images.

   The default configuration captures a total of 20 images, one every 5 seconds.

5. The preview pane shows the live images streamed as RGB data. After you adjust any device properties and capture settings, use the Preview window as a guide to line up the camera to acquire the pattern image you want to capture.

6. Select Capture. The app captures the specified number of images, and the thumbnails of the snapshots appear in the Data Browser pane. They are automatically named incrementally, and are captured as .png files.

   You can stop the image capture before the designated number of images are captured by selecting Stop Capture.

   When you are capturing images of a pattern, after the designated number of images are captured, the app displays the Image and Pattern Properties dialog box. Select the calibration pattern in the image and specify the pattern properties. Click OK.

   The app then calculates and displays the detection results.

   

7. To dismiss the Detection Results dialog box, click OK.

8. When you have finished acquiring live images, select Close Image Capture to close the Camera tab.

After you add images, the Image and Pattern Properties dialog box to your session, appears. Before the calibrator can analyze the calibration patterns, you must select the calibration pattern to detect and set image properties for the pattern structure. For more details on this dialog, see Select Calibration Pattern and Set Properties.

 Analyze Images

The calibrator attempts to detect a pattern in each of the added images, and displays a progress bar dialog box, that indicates detection progress. If any of the images are rejected, the Detection Results dialog box appears and displays diagnostic information. The results indicate how many total images have been processed and, of those processed, how many have been accepted, rejected, or skipped. The calibrator skips duplicate images.

To view the rejected images, click view images. In addition to duplicate images, the calibrator also rejects images where it could not detect the entire pattern. Possible reasons for no detection are a blurry image or an extreme angle of the pattern. Detection takes longer for larger images and patterns that contain a large number of elements.

 View Images and Detected Points

The Data Browser pane displays a list of images with their IDs. Each image contains a detected pattern. To view an image, select it from the Data Browser pane.

The Image pane displays the selected checkerboard image with green circles overlaid to indicate detected points. You can verify that the corners have been detected correctly by using the zoom controls. The yellow square indicates the (0,0) origin. The X and Y arrows indicate the checkerboard axes orientation.

Calibrate

Once you are satisfied with the accepted images, on the Calibration tab, select Calibrate. The default calibration settings use a minimum set of camera parameters. Start by running the calibration with the default settings. After evaluating the results, you can try to improve calibration accuracy by adjusting the settings or adding or removing images, and then calibrating again. If you switch between the standard and fisheye camera models, you must recalibrate.

 Select Camera Model

You can select either a standard or fisheye camera model. On the Calibration tab, in the Camera Model section, select Standard or Fisheye.

You can switch camera models at any point in the session. You must recalibrate after changing the camera model. Select Options to access settings and optimizations for either camera model.

 Standard Model Options

When the camera has severe lens distortion, the app can fail to compute the initial values for the camera intrinsics. If you have the manufacturer specifications for your camera and know the pixel size, focal length, or lens characteristics, you can manually set initial guesses for the camera intrinsics and radial distortion. To set initial guesses, select Options > Optimization Options.

• Select Specify initial intrinsics as a 3-by-3 matrix of the form [fx 0 0; s fy 0; cx cy 1], and then enter a 3-by-3 matrix to specify initial intrinsics. If you do not specify an initial guess, the function computes the initial intrinsic matrix using linear least squares.

• Select Specify initial radial distortion as 2- or 3-element vector, and then enter a 2- or 3-element vector to specify the initial radial distortion. If you do not provide a value, the function uses 0 as the initial value for all the coefficients.

For more details on calibration parameters, see What Is Camera Calibration?.

 Fisheye Model Options

In the Camera Model section, with Fisheye selected, click Options. Select Estimate Alignment to enable estimation of the axes alignment when the optical axis of the fisheye lens is not perpendicular to the image plane.

For details about the fisheye camera model calibration algorithm, see Fisheye Calibration Basics.

 Calibration Algorithm

For fisheye camera model calibration, see Fisheye Calibration Basics.

The standard camera model calibration algorithm assumes a pinhole camera model:

$$w\left[\begin{array}{ccc}x& y& 1\end{array}\right]=\left[\begin{array}{cccc}X& Y& Z& 1\end{array}\right]\left[\begin{array}{c}R\\ t\end{array}\right]K$$

• (X,Y,Z) — World coordinates of a point.

• (x,y) — Image coordinates of the corresponding image point in pixels.

• w — Arbitrary homogeneous coordinates scale factor.

• K — Camera intrinsic matrix, defined as:

  $$\left[\begin{array}{ccc}{f}_x& 0& 0\\ s& f_y& 0\\ c_x& c_y& 1\end{array}\right]$$

  The coordinates (c_x, c_y) represent the optical center (the principal point), in pixels. When the x- and y-axes are exactly perpendicular, the skew parameter, s, equals 0. The matrix elements are defined as:

  • f_x = F*s_x, expressed in pixels.

  • f_y = F*s_y, expressed in pixels.

  • F is the focal length in world units, typically expressed in millimeters.

  • s_x and s_y are the number of pixels per world unit in the x- and y- respectively.

• R — Matrix representing the 3-D rotation of the camera.

• t — Translation of the camera relative to the world coordinate system.

The camera calibration algorithm estimates the values of the intrinsic parameters, the extrinsic parameters, and the distortion coefficients. Camera calibration involves these steps:

1. Solve for the intrinsics and extrinsics in closed form, assuming that lens distortion is zero. [1]

2. Estimate all parameters simultaneously, including the distortion coefficients, using nonlinear least-squares minimization (Levenberg–Marquardt algorithm). Use the closed-form solution from the preceding step as the initial estimate of the intrinsics and extrinsics. Set the initial estimate of the distortion coefficients to zero. [1][2]

Evaluate Calibration Results

You can evaluate calibration accuracy by examining the reprojection errors, examining the camera extrinsics, or viewing the undistorted image. For best calibration results, use all three methods of evaluation.

 Examine Reprojection Errors

The reprojection errors are the distances, in pixels, between the detected and the corresponding reprojected points. The Camera Calibrator app calculates reprojection errors by projecting points from the world coordinates defined by the pattern into image coordinates. The app then compares the reprojected points to the corresponding detected points. As a general rule, mean reprojection errors of less than one pixel are acceptable.

The Camera Calibrator app displays, in pixels, the reprojection errors as a bar graph. The graph helps you identify which images adversely impact the calibration. You can select a bar graph entry to select an image, and then remove the image from the list of images in the Data Browser pane.

Reprojection Errors Bar Graph
The bar graph displays the mean reprojection error per image, along with the overall mean error. The bar labels correspond to the image IDs. The highlighted bars correspond to the selected images.

Select an image in one of these ways:

• Click a corresponding bar in the graph.

• Select an image from the list of images in the Data Browser pane.

• Adjust the overall mean error. Slide the red line up or down to automatically select all images with a mean error greater than the specified value.

 Examine Extrinsic Parameter Visualization

The 3-D extrinsic parameters plot provides a camera-centric view of the patterns and a pattern-centric view of the camera. The camera-centric view is helpful if the camera is stationary when the images are captured. The pattern-centric view is helpful if the pattern is stationary. You can click and drag the figure to rotate it. Click a checkerboard (or camera) to select it. The highlighted data in the visualizations corresponds to the selected image in the list. Examine the relative positions of the pattern and the camera to determine if they match what you expect. For example, a pattern that appears behind the camera indicates a calibration error.

 View Undistorted Image

To view the effects of removing lens distortion, on the Calibration tab, in the View section, select Show Undistorted in the View section of the Calibration tab. If the calibration is accurate, the distorted lines in the image preview become straight.

Note

Checking the undistorted images is important even if the reprojection errors are low. For example, if the pattern covers only a small percentage of the image, the distortion estimation can be incorrect, even though the calibration resulted in few reprojection errors. This image shows an example of this type of incorrect estimation for a single camera calibration.

For the fisheye camera model, while viewing the undistorted images, you can examine the fisheye images more closely by, on the Calibration tab, in the View section, specifying the Fisheye Scale. Enter a value in the Fisheye Scale box, or use the arrows to adjust the scale up or down.

Improve Calibration

To improve the calibration, you can remove high-error images, add more images, or modify the calibrator settings.

Consider adding more images if:

• You have fewer than 10 images.

• The calibration patterns do not cover enough of the image frame.

• The calibration patterns do not have enough variation in orientation with respect to the camera.

Consider removing images if the images:

• Have a high mean reprojection error.

• Are blurry.

• Contain a calibration pattern at an angle greater than 45 degrees relative to the camera plane.

  

• Incorrectly detected calibration pattern points.

 Standard Model: Change the Number of Radial Distortion Coefficients

You can specify two or three radial distortion coefficients. On the Calibrations tab, in the Camera Model section, with Standard selected, click Options. Specify the Radial Distortion as either two or three coefficients by selecting 2 Coefficients or 3 Coefficients, respectively.

Radial distortion is the displacement of image points along radial lines extending from the principal point.

• As image points move away from the principal point (positive radial displacement), image magnification decreases and a pincushion-shaped distortion occurs on the image.

• As image points move toward the principal point (negative radial displacement), image magnification increases and a barrel-shaped distortion occurs on the image.

The radial distortion coefficients model this type of distortion. The distorted points are denoted as (x_distorted, y_distorted):

x_distorted = x(1 + k₁*r² + k₂*r⁴ + k₃*r⁶)

y_distorted= y(1 + k₁*r² + k₂*r⁴ + k₃*r⁶)

• x, y — Undistorted pixel locations. x and y are in normalized image coordinates. Normalized image coordinates are calculated from pixel coordinates by translating to the optical center and dividing by the focal length in pixels. Thus, x and y are dimensionless.

• k₁, k₂, and k₃ — Radial distortion coefficients of the lens.

• r² = x² + y²

Typically, two coefficients are sufficient for calibration. For severe distortion, such as in wide-angle lenses, you can select three coefficients to include k₃.

The undistorted pixel locations are in normalized image coordinates, with the origin at the optical center. The coordinates are expressed in world units.

 Standard Model: Compute Skew

To estimate the skew of the image axes, on the Calibration tab, in the Camera Model section, select Options > Compute > Skew. Some camera sensors contain imperfections that cause the x- and y-axes of the image to not be perpendicular. You can model this defect using a skew parameter. If you do not select this option, the image axes are perpendicular, which is true for most modern cameras.

 Standard Model: Compute Tangential Distortion

Tangential distortion occurs when the lens and the image plane are not parallel. The tangential distortion coefficients model this type of distortion.

The distorted points are denoted as (x_distorted, y_distorted):

x_distorted = x + [2 * p₁ * x * y + p₂ * (r² + 2 * x²)]

y_distorted = y + [p₁ * (r² + 2 *y²) + 2 * p₂ * x * y]

• x, y — Undistorted pixel locations. x and y are in normalized image coordinates. Normalized image coordinates are calculated from pixel coordinates by translating to the optical center and dividing by the focal length in pixels. Thus, x and y are dimensionless.

• p₁ and p₂ — Tangential distortion coefficients of the lens.

• r² = x² + y²

To estimate the tangential distortion coefficients, on the Calibration tab, in the Camera Model section, select Options > Compute > Tangential Distortion. Otherwise, the calibrator sets the tangential distortion coefficients to zero.

 Fisheye Model: Estimate Alignment

On theCalibration tab, in Camera Model section, with Fisheye selected, click Options. Select Estimate Alignment to enable estimation of the axes alignment when the optical axis of the fisheye lens is not perpendicular to the image plane.

Export Camera Parameters

When you are satisfied with your calibration accuracy, select Export Camera Parameters for a standard camera model or Export Camera Parameters for a fisheye camera model. You can either export the camera parameters to an object in the MATLAB workspace or generate the camera parameters as a MATLAB script. If the default values work well, then you do not need to make any adjustments before exporting the parameters

 Export Camera Parameters

Under Export Camera Parameters, for a standard camera model, or Export Fisheye Parameters, for a fisheye camera model, select Export Parameters to Workspace to create a cameraParameters object in your workspace. The object contains the intrinsic and extrinsic parameters of the camera and its distortion coefficients. You can use this object for various computer vision tasks, such as image undistortion, measuring planar objects, and 3-D reconstruction. For more information on measuring planar objects, see Measuring Planar Objects with a Calibrated Camera. You can optionally export a cameraCalibrationErrors object that contains the standard errors of the estimated camera parameters by selecting Export estimation errors.

 Generate MATLAB Script

Under Export Camera Parameters, for a standard camera model, or Export Fisheye Parameters, for a fisheye camera model, select Generate MATLAB script to save your camera parameters to a MATLAB script, enabling you to reproduce the steps from your calibration session.

Note

You cannot generate a MATLAB script for custom pattern camera parameters defined using the vision.calibration.PatternDetector class.

References

[1] Zhang, Z. “A Flexible New Technique for Camera Calibration.” IEEE Transactions on Pattern Analysis and Machine Intelligence. 22, no. 11 (November 2000): 1330–34. https://doi.org/10.1109/34.888718.

[2] Heikkila, J., and O. Silven. “A Four-step Camera Calibration Procedure with Implicit Image Correction.” In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 1106–12. San Juan, Puerto Rico: IEEE Comput. Soc, 1997. https://doi.org/10.1109/CVPR.1997.609468.

[3] Scaramuzza, Davide, Agostino Martinelli, and Roland Siegwart. "A Toolbox for Easily Calibrating Omnidirectional Cameras." In Proceedings of IEEE International Workshop on Intelligent Robots and Systems 2006 (IROS 2006), 5695–701. Beijing, China: IEEE, 2006. https://doi.org/10.1109/IROS.2006.282372

[4] Urban, Steffen, Jens Leitloff, and Stefan Hinz. “Improved Wide-Angle, Fisheye and Omnidirectional Camera Calibration.” ISPRS Journal of Photogrammetry and Remote Sensing 108 (October 2015): 72–79. https://doi.org/10.1016/j.isprsjprs.2015.06.005.