Segmentation of color image using depth information

I have an already acquired database of color and depth images. My goal is to use the depth image to segment the color image using the information from the nearest object. The problem I have is that the 2 images have different resolutions (depth -> 240*320, color ->480*640) so I found a problem during segmentation. To remedy this problem I looked everywhere to find a solution and what I found is as follows : 1) the use of registration based on calibration information 2) mapping always using calibration information My problem is that I haven't found the right approach that I can follow and that is very clear. Note that the calibration information I have is : fx, fy ,cx ,cy ,k1 ,k2 ,k3 ,p1 ,p2, for the color image and the depth image. I will be grateful if someone can help me and any information can be useful.