cropping manually and automatic using matlab

i want to seperate every scene in the comic, but the problem is not all of the scene are rectangle and there are so many details.

can u guys tell me the way to cropping manually or automatic after edge detection? and what the first thing i have to do before edge detection?

thanks..

[Ryan]

Can you post a sample image that you are trying to segment? It is difficult to advise you with out an appropriate visual aid.

[Image Analyst]

Who said you have to do edge detection? Can't you just threshold, call imfill(), call regionprops(), and then call imcrop()?

http://www.mathworks.com/matlabcentral/answers/7924-where-can-i-upload-images-and-files-for-use-on-matlab-answers

[shara]

here's the image

http://imageshack.us/f/546/aluminiumcomicstrip.jpg/

so i can do it without edge detection? can u show me how? thanks..

[Image Analyst]

That image does not have separate scenes - everything is overlapped so even if you were to artificially chop it into "scenes" there would be clutter from adjacent scenes (parts of elements) that would mess up the scene. What good would that be? Why do you want to do that anyway?

[shara]

then i'm gonna change the image, what about this http://imageshack.us/f/13/hiltunenpetri.gif/ there's a dialog that's not fully in the box..

[Image Analyst]

Yes. that would make them connected but you can try to do imopen() to break that connection.

Answer 1

[Image Analyst]

Do you still need the manual method? I'd use rbbox or imrect - see the help for examples. For the automatic one, I probably did too much for you, but here is the code for the "depressingcomic" image.

% function to crop out panels of a cartoon/comic.
clc;
clearvars;
close all;
workspace;
fontSize = 14;

% Read in a standard MATLAB color demo image.
folder = 'C:\Users\shara\Documents\Temporary';
baseFileName = 'depressingcomicweek3cya.jpg';
fullFileName = fullfile(folder, baseFileName);
% Get the full filename, with path prepended.
fullFileName = fullfile(folder, baseFileName);
if ~exist(fullFileName, 'file')
  % Didn't find it there.  Check the search path for it.
  fullFileName = baseFileName; % No path this time.
  if ~exist(fullFileName, 'file')
    % Still didn't find it.  Alert user.
    errorMessage = sprintf('Error: %s does not exist.', fullFileName);
    uiwait(warndlg(errorMessage));
    return;
  end
end
rgbImage = imread(fullFileName);
% Get the dimensions of the image.  numberOfColorBands should be = 3.
[rows columns numberOfColorBands] = size(rgbImage);
% Display the original color image.
subplot(2, 3,  1);
imshow(rgbImage);
title('Original Color Image', 'FontSize', fontSize);
% Enlarge figure to full screen.
set(gcf, 'units','normalized','outerposition',[0 0 1 1]);

% Erase copyright.
rgbImage(552:end, 202:430, :) = 255;
% Erase title.
rgbImage(1:25, :, :) = 255;
% Display the color image.
subplot(2, 3,  2);
imshow(rgbImage);
title('Copyright and Title Erased', 'FontSize', fontSize);

% Binarize to find black pixels
% Find where either red, green, or blue channel is dark.
thresholdValue = 55;
binaryImage = rgbImage(:,:, 1) < thresholdValue | rgbImage(:,:, 2) < thresholdValue | rgbImage(:,:, 3) < thresholdValue;
% Display the image.
subplot(2, 3,  3);
imshow(binaryImage);
title('Binary Image', 'FontSize', fontSize);

% Fill the image
filledImage = imfill(binaryImage, 'holes');
% Display the image.
subplot(2, 3,  4);
imshow(filledImage);
title('Filled Binary Image', 'FontSize', fontSize);
drawnow;

% Label the image - find connected components.
[labeledImage numberOfBlobs] = bwlabel(filledImage);
% Get bounding boxes for all the regions.
measurements = regionprops(labeledImage, 'BoundingBox');

% Create colors for all the boxes.
boxColors = lines(numberOfBlobs);

% Fill in the bounding boxes to mask the "broken" panels.
for k = 1 : numberOfBlobs
  x1 = measurements(k).BoundingBox(1);
  y1 = measurements(k).BoundingBox(2);
  width = measurements(k).BoundingBox(3);
  height = measurements(k).BoundingBox(4);
  x2 = x1 + width;
  y2 = y1 + height;
  
  % Plot the boxes in different colors in a separate subplot.
  subplot(2, 3,  5);
  hold on;
  set(gca, 'ydir', 'reverse');
  thisColor = boxColors(k, :);
  rectangle('Position', [x1 y1 width height], 'EdgeColor', thisColor);
  drawnow;
  title('All the Bounding Boxes', 'FontSize', fontSize);

  x1 = ceil(x1); % Get rid of the 0.5
  y1 = ceil(y1); % Get rid of the 0.5
  x2 = floor(x2); % Get rid of the 0.5
  y2 = floor(y2); % Get rid of the 0.5
  filledImage(y1:y2, x1:x2) = true;
  % Display the image.
  subplot(2, 3,  6);
  imshow(filledImage);
  title('Filled Binary Image', 'FontSize', fontSize);
end

% Call bwlabel again.  This time there shoudl be 5 objects.
[labeledImage numberOfBlobs] = bwlabel(filledImage);
% Get bounding boxes for all the regions.
measurements = regionprops(labeledImage, 'BoundingBox');
% Inform user.
message = sprintf('We have now found the %d panels.\nNow we will crop them out', numberOfBlobs);
uiwait(msgbox(message));

% Create a new figure for this.
figure;
% Enlarge figure to full screen.
set(gcf, 'units','normalized','outerposition',[0 0 1 1]);

% Crop out the individual panels.
for k = 1 : numberOfBlobs
  x1 = measurements(k).BoundingBox(1);
  y1 = measurements(k).BoundingBox(2);
  width = measurements(k).BoundingBox(3);
  height = measurements(k).BoundingBox(4);
  subImage = imcrop(rgbImage, [x1 y1 width height]);
  % Display the image.
  subplot(2, 3,  k);
  imshow(subImage);
  caption = sprintf('Panel #%d', k);
  title(caption, 'FontSize', fontSize);
  
  if k < numberOfBlobs
    message = sprintf('That was panel #%d.\nDo you want to continue and show panel #%d?', k, k+1);
    % Inform user - allow to bail out if desired.
    button = questdlg(message, 'Continue?', 'Yes', 'No', 'Yes');
    drawnow;  % Refresh screen to get rid of dialog box remnants.
    if strcmpi(button, 'No')
       break;
    end
  end

end
msgbox('Thank you ImageAnalyst!');

[shara]

thank you so much it works and that's cool! :D

yes, i still need the manual method, i need to learn more about image processing.

but thank you for helping me..

[Image Analyst]

Did you try putting rbbox or imcrop into a loop like I suggested? I could do that for you but I think it's within your capabilities.

[shara]

I think it's better like this. Is that harder ?

[shara]

Sir, if i have a different image that has more panels.

Which part of the codes that should be changed?

[Walter Roberson]

No change. The code is already written to handle arbitrary numbers of panels.

[shara]

But when i try with 3panels image, it cropped into 4panels. Which is the 3panels is part of the image but 1panel isjust showing color.

And when i try with another image that has different size of panels some of the details is not get he bounding box properly and thats makes the cropped result became a half of the original panel should be. What should i do?

[Image Analyst]

The code was designed to handle the one demo image you gave. That image had a title that needed to be erased and a copyright panel at the bottom that needed to be erased. Then it had several cartoon panels which were all well separated. Obviously, you may have to edit the part about erasing title and copyright if your new image doesn't have those or they are in a different place. But after that, the code is general enough to handle any number of panels as long as they are separated. But if they are not well separated, like some of your prior images, then you're either going to havde to employ a much more complicated automatic algorithm, which may cause you to lose some of the parts of a panel that penetrate into another panel, or you're going to have to do it manually. Stuff from one panel that overlaps or penetrates into another panel will cause those panels to be connected and be considered as a single panel, according to the very simple code I gave you. The automatic way will be much, much harder than just using imrect() or rbbox(), so I'm really puzzled as to why you refuse to use them since it would be easier. Maybe you can explain why. At first you seemed to say you needed a manual way then when I tell you what functions to use to do it manually, you won't do it. Why not?

[shara]

Ya of course i've edited the part about erasing title and copyright to another image that i use

It doesn't i refuse to use it or i won't do it, that's actually because i don't really know how to use it and still learning it.

Could u help me, thanks before

[Image Analyst]

What are you trying to do? Automatic or manual? I already gave you an automatic method and I'm not going to make it more sophisticated for you to handle every possible kind of comic where panels connect to each other. Ryan gave you a manual method that I'm sure you could adapt. So, what is left to help you on? You'll have to be more explicit, plus, realize that I can't spend hours and hours to do your work for you. If I can't bang it out in a few minutes, then I leave it to you. I can give you some tips and pointers but you'll have to do the majority of coding for your project.

[shara]

I was trying to put imrect into a loop and got a problem, but now i already resolve it.

Thanks for everything

[shara]

i wanna ask something again sir,

why i got a message "??? Index exceeds matrix dimensions. "

when i use this image http://imageshack.us/f/198/wl7d.jpg/

and it only showed original color image.

[Image Analyst]

I don't know, especially without seeing the line of code it refers to. Why don't you set a breakpoint there and look at the values of all the indexes on that line, as well as the size of the array. That will tell you which index is out of range.

[shara]

ok i'll try it out

Answer 2

[Image Analyst]

Just crop it.

width = col1 - col2 + 1;
height = row2 - row1 + 1;
onePanel = imcrop(originalImage, [row1 col1 width height]);

Where col1, row1, col2, and row2 are is obviously a judgement call - you and I might not agree on where they are.

[shara]

is that can be crop by maintain the shape of the outerside? i'm a newbie in matlab so could u explain more specific thanks before

[Image Analyst]

Why don't you crop it and show us how you'd like it to be cropped? Because I have no idea. And explain why you chose the cropping locations where you did.

[Walter Roberson]

You cannot maintain the outside shape when you crop. crop is always along rectangular boundaries. This is because MATLAB cannot store arrays with irregular boundaries.

[shara]

so u mean matlab cannot crop except rectangle shape? okay then i'm gonna change the image that fully rectangle every scene. and this is the image and the scene that i've cropped

• http://imageshack.us/f/513/depressingcomicweek3cya.jpg/
• http://imageshack.us/f/225/48522388.jpg/
• http://imageshack.us/f/266/48386146.jpg/
• http://imageshack.us/f/600/84671304.jpg/
• http://imageshack.us/f/6/20995236.jpg/
• http://imageshack.us/f/268/51020284.jpg/
• http://imageshack.us/f/401/20287617.jpg/

[Walter Roberson]

Yes, I do mean that MATLAB can only crop into rectangular shapes.

You can use a combination of rectangular shapes and "masks", where the masks give information about which part of the rectangle are used. It is also common (but not always practical) to use rectangular shapes but to set the irrelevant information to a background color such as pure black.

The difficulty of finding only the objects relevant to a particular frame depends upon how complicated the edges are. It would be difficult for the first comic you showed, the one about recycling; it might be practical for the second comic you showed.

[shara]

so could u show me how to crop the second comic? thanks..

[Walter Roberson]

Sorry, I am not experienced in this kind of analysis, and I do not have access to MATLAB to experiment with. Perhaps Image Analyst will have suggestions. You should indicate, though, whether you want to proceed by rectangular regions only, or by rectangular regions and masks, or by rectangular regions and a background color.

[Image Analyst]

Can we assume that the copyright box will be in the same location for all your images? Because that's the only thing that would complicate the algorithm. Otherwise it's really simple, but that copyright connects the bottom three panels together. If I can assume that it will always be in the same place, I can erase it, threshold, fill, find bounding boxes, mask, then regionprops again to find bounding boxes again, and it will get the rectangular panels. But if the copyright box is sometimes there and sometimes not, or it's in different locations, then it would be a much more complicated algorithm.

[shara]

i think i'm not gonna use the copyright, maybe i'll erase it. so what step should i do? could u give me the example code? thanks before..

Answer 3

[Ryan]

Manual method:

scenenum = input('Please enter the number of scenes you wish to crop: ')
for m = 1:scenenum
 mask = roipoly(Img);
 segment{m} = immultiply(Img,mask);
 Img = immultiply(Img,~mask);
 figure, imshow(segment{m})
end

[Image Analyst]

image is the name of a built in MATLAB function. It's best to use another name for that variable so you don't destroy that function.

[shara]

i've tried this and i got an error message

??? Undefined function or method 'immulitply' for input arguments of type 'uint8'.

what should i do?

[Walter Roberson]

Fix the obvious typing mistake so that it is immultiply

[shara]

i still got an error

Error in ==> immultiply at 66

    checkForSameSizeAndClass(X, Y, mfilename);

[Walter Roberson]

What is the error message?

[Ryan]

what class is your image and what is its size?

[shara]

 Size=629x701x3, Class=uint8

here's the image that i use

http://imageshack.us/f/513/depressingcomicweek3cya.jpg/

[Ryan]

What version of matlab are you using? (since you're asking about this again).

[shara]

I use 7.8 version

[Ryan]

I'm not sure why you're getting an immultiply not being compatible with uint8 error... does 7.8 have the immultiply function? If not, you'll need to use .* and multiply the mask to each individual layer of Img.

[Walter Roberson]

7.8... that's R2009a if I remember correctly.

We have not been told what the error message is. Possibly it is complaining that the two inputs are of different classes.

[Image Analyst]

This is not what shara wants to do anyway, which is cropping. This would mask, not crop. Masking blackens inside or outside some area, while cropping extracts the bounding box of the region into a new smaller image.