Fundamental matrix and epipolar lines from stereo calibration slightly off

Hello,

I am using a pair of two GoPro's as a stereo system for estimating the 3D location of some pixels in my image. In order to do this, I use the Stereo Camera Calibrator. I calibrate my cameras using a checkerboard pattern glued on a wooden plate and take paired pictures of the checkerboard at different angles and different distances with respect to the camera and I obtain a mean reprojection error of ~ 0.4 [px.] (for 94 paired pictures).

After the calibration, in a separate trial, I project a different checkerboard pattern on a screen in order to test whether the calibration is correct. In Figure 1 you can see a point (extracted using detectCheckerboardPoints) at the top left of the checkerboard pattern from the two camera perspectives along with the epipolar line in the second perspective (computed using the Fundamental matrix from the stereo calibration session, the function epipolarLine and the function lineToBorderPoints). As it can be observed, the epipolar line intersects the edge in the second perspective quite well. In Figure2, I choose another point of the checkerboard and it can be observed that the correspondence with the epipolar line is not that good anymore.

Does anybody have an intuition why this would be the case?

[Matt J]

Wouldn't it be because of the accuracy of the calibration? Or the accuracy of the corner detection?

[Alexandru Iancu]

Thanks for your answer! My intuition says that it can be a limitation of the camera model and therefore of my calibration, since during the stereo calibration my checkerboard was not in the pose that I am using for the separate test.

[SABINA LAVEGLIA]

Do you used all the image for strereo calibration or specify the different heinghts?

Answer 1

[Matt J]

The position of the checkerboard isn't supposed to be part of the camera model. It's the relative poses of the two cameras that matters. The perspective of the two cameras does look a bit similar, however. You might get a better calibration if you move them further apart.

[Alexandru Iancu]

Thanks for your response. I also tried to triangulate the two pairs of points, and I obtain a higher reprojection error for the points in figure 2 compared to the points in figure 1.