Crop and create a matrix from an image

Hey guys, for an upcoming project I have to identify which tiles are on the board of a screenshot from the 2048 game.

For this I wanted to be able to crop down the screenshot to just have the board no matter where the board is on the screenshot, then take the RGB values for each individual tile and turn this into a 4x4 matrix!

If anyone could help me figure this out it would be much appreciated, I have tried numerous ways to do this and just cant seem to get it at all

Answer 1

[Image Analyst]

This could be very easy if the images are all like the two you have posted. Steps are

1. Find all pixels that are the color of the gray grid. For example use the Color Thresholder app on the Apps tab of the tool ribbon to export code.
2. Call mask = imfill(mask, 'holes') on the blobs to fill in the holes.
3. If the square is the largest blob, find the square either by calling mask = bwareafilt(mask, 1)
4. If it's not the largeste blob, call props = regionprops(mask, 'BoundingBox') to the the bounding box and find the square by finding the blob where the width and height are pretty close to the same. bb = vertcat(props.BoundingBox); widths = bb(:, 3); heights = bb(:, 4);
5. Now, get the rows and columns of some spot that will be in color, like in the upper left corner of each of the 16 squares. Read that color.
6. Have a look up table that tells you what the number is based on the color of the square. You should know in advance what the square background color is for each number. lut = [r1, g1, b1; r2, g2, b2; r3, g3, b3, etc. up to 128 or the max]

Should be an easy project for you - maybe 15-30 minutes. Good luck.