Defect detection with pattern variation in comparison to template

I am trying to detect defects on images comparing them to a reference template. Sample images are shown that can illustrate the problem. In this case the defects are red circled. I used image difference but due to the pattern variations and different line thickness it results in too many false positives. any Ideas or directions on how to proceed?

Reference Image

Defect Image

Answer 1

[Jim Joy]

Hi Ahmed,

I think that this issue lends itself well to pattern matching. You might have to develop several criteria to determine what constitutes a 'match' and what constitutes a 'defect', but I think that the regularity of both the arrangement of the pattern, and the pattern itself lend itself to this approach.

Here's an approach that should get you started. This will tell you how many cells are missing from your image. Here, 'image_1' is the top image, and 'image_2' is the bottom image. The code below works by creating a lattice of points, and comparing their locations with those of the actual object centers in the image. If there is not a 'real' object center near a 'lattice' center, this lattice point is counted as a missing object. Note that I have begun by defining a template to pattern match in each cell, but this step is unnecessary at this point.

im = im2bw(imread('image_1.jpg'));
im2 = im2bw(imread('image_2.jpg'));
im2 = bwareaopen(im2,5);
temp = im(20:90,15:75);
imshow(im2)
regionObj = regionprops(im2);
regionCents = vertcat(regionObj.Centroid);
hold on
plot(regionCents(:,1),regionCents(:,2),'wx','MarkerSize',12);
[latticeX,latticeY] = meshgrid(44+85*(0:1:4),60+85*(0:1:6)); 
% Make these column vectors 
latticeX = latticeX(:);
latticeY = latticeY(:);
plot(latticeX,latticeY,'yx','markersize',12);
axis equal
hold on
% Try to find the lattice points that correspond to real cells in the image
% Define empty vectors to store the lattice points that are found
foundX = [];
foundY = [];
numMissing = 0;
for ii = 1:length(latticeX)
  [dist,idx] = min(sqrt((latticeX(ii) - regionCents(:,1)).^2 + (latticeY(ii) - regionCents(:,2)).^2));
  if dist < 25
      foundX(end+1) = regionCents(idx,1);
      foundY(end+1) = regionCents(idx,2);
  else 
      numMissing = numMissing + 1;
  end
end
plot(foundX,foundY,'bx','markersize',12);

Moving forward, I would recommend experimenting with methods to match to the pattern defined above in 'temp' (or something similar). One thing you might consider is looking at the difference between a region of interest near each found cell center, and its image XOR'd with the complement of the template. I have an example shown below. However, I say this with the caveat that I haven't had the opportunity to experiment with different operations here.

 [h,w] = size(temp);
h = floor(h/2);
w = floor(w/2);
% Just choose 1 ROI to look at for now
for ii = 3
   figure;
   xCent = floor(foundX(ii));
   yCent = floor(foundY(ii));
   roi = im2(yCent-h:yCent+h,xCent-w:xCent+w);
   % This image shows the points in the roi that do not intersect with the template
   % You might consider performing this or a similar operation, and developing a metric on it to 
   % determine if there is a match or defect in the current cell
   imshow(roi - xor(roi,~temp));
end

In addition, you might consider using "bwboundaries" to look for regions that do not have holes in the center. See the documentation link here. The blog posting here also has some good ideas about how you might proceed.

Best of luck moving forward on this problem.

Jim

[Ahmed]

Hi Jim Many thanks for your answer, I will try your recommendation and see if it works here. I just had new information yesterday regarding the problem, as it will not always be a repeated pattern as the sample I have here. nevertheless I think local comparison for each region may work too.