stereo vision question- translation of pixels from left camera to the reconstructed scene image

Hello everyone! Through our computer science degree we are using 2 cameras and building a 3d scene of the enviorment. Our estimate errors are very low, the image we get back from the process is also pretty accurate.Now, through object tracking algorithm presented in matlab tutorials we are tracking multiple objects and our problem is as follows: We find a moving object through the left camera and try to calculate its distance from the cameras. In order to do that, we need the ability to know how to match a few pixels in the left camera image to their appropriate pixels in the reconstructed image ( we use the reconstructScene function that is attached in the link below). Is there any way to "reverse" ,so to say ,the process of creating a 3d image and to know in a genral form, maybe a formula, that helps to translate a pixel from the left image to its appropriate pixel in the reconstructed scene image? Worth to mention, we of course have access to the rectified images that help generate the reconstructed scene and the stereo parameters of the cameras.

Thanks in adavance! Ohad

Answer 1

[ohad assour]

If the question is unclear or you are not certain in the manner we could achieve the wanted task i would very much appreciate a feedback here. We dont understand why all the questions submitted after ours gets an answer and ours doesnt. Thanks again for your help