Data format for multiple categories in R-CNN

I am trying to train an R-CNN to identify objects of different categories in an image. However, there is one part in the official documentation that I do not understand. It is clear that one needs to increase the number of nodes in the last fully-connected layer to take the categories into account, but I cannot understand the explanation for how the training data table should be structured. It says;

'Labeled ground truth images, specified as a table with two or more columns. The first column must contain path and file names to images that are either grayscale or true color (RGB). The remaining columns must contain bounding boxes related to the corresponding image. Each column represents a single object class, such as a car, dog, flower, or stop sign.'

So each column after the first column corresponds to a category. But what if not every category is present in an image? Should the content in those columns just be an empty cell, or a [0 0 0 0] bounding box? Also, how should one define the bounding box if there are more than one instance of a category in the same image?

Answer 1

[Farah Sarwar]

You can design ground truth data/table using "Image Labeler" app in Matlab (available in Image processing and computer vision tool box). Then you can export them in your workspace as a table or ground truth data. These bounding boxes are linked with respective image but are not over the image. You can create as many categories there as you want and annotate the whole data set as per your own requirement. Check following links for more guidance.

https://au.mathworks.com/help/driving/ref/groundtruthlabeler-app.html

https://au.mathworks.com/help/vision/ref/imagelabeler-app.html?searchHighlight=image%20labeler&s_tid=doc_srchtitle

To view the same in MATLAB then use the following code:

annotation = 'box';

resultant_Img = insertObjectAnnotation(img1, 'rectangle', BoundingBoxes,annotation);

imagesc(resultant_Img)

But I hope that you have figured it out by now. :)

[Lucas Hedström]

Thank you for your answer!

I had a look at that tool. It however became unsuitable to use R-CNNs for my project due to the lack of experimental data, but thank you anyway!