Optical Character Recognition (OCR) from an app screen shot.

Hello,

I am trying to create a function that can convert the following image:

into a matrix that contains the value of each frame.

So, for the above image, the output would be

frames = [30 60 90 120 149 169 189 219 248 268; 
          18 28 58 88  118 148 175 194 203 212;
          28 48 68 98  128 158 187 207 226 245];

The app can output a screenshot, so the frame locations will always be consistent.

I have managed to find these frame locations, crop them, and create a 3,10 cell containing the cropped images of each frame.

I have also managed to convert these images to binary, however I don't know if this gets me any closer to solving my problem.

I have looked into OCR and limiting the CharacterSet to numeric characters, but this is not yielding successful results.

I'm aware that 100% accuracy may not be possible, but it's what I'm striving for.

Any help would be greatly appreciated!

Answer 1

[Cindy Solomon]

Hi Steven,

When you mentioned that you looked into OCR, I assume you meant the "ocr" function in MATLAB, but if you haven't seen this doc page, thought it would be helpful to you:

http://www.mathworks.com/help/vision/examples/recognize-text-using-optical-character-recognition-ocr.html

However, I've done something similar without using the computer vision toolbox at all- if you know what the numbers will look like (for example, you take a screenshot of each of the numbers separately), you can essentially adapt what this person was doing for integer recognition:

http://www.mathworks.com/matlabcentral/fileexchange/9477-morphological-character-recognition/content/Intrecog/intrec.m

This essentially entails creating a "test" image for each number as a training set, expanding that image and subtracting the original image so that it can test if the numbers are equivalent. This code in particular uses the binary hit-or-miss algorithm found here to identify the number. It recognizes the "left" or "right" of two digit numbers by just identifying which side of the image there is a match.

Attached is my adapted version of the File Exchange code with my own additions to it from when I used it for my project. I called that function within another function I named "process", which includes the following lines of code:

X= imread(File);                                %Read screenshot you want

im_gray = rgb2gray(X);                          %Convert to grayscale/1-channel image
im_bw = im2bw(im_gray,0.5);                      %Convert to black and white with threshold of 1/2

im_sub = im_bw(210:270,1636:1714);              %Select region of interest for numbers

[Number]= intrec(im_sub);                       %Pattern recognition on the numbers (Integer recognition)

Output= Number;                                 %Output the number

Hope this helps! It definitely makes it much easier that you know what the numbers should look like and where they would be.