Automatically segment the fish eye

Hi, I'm having trouble with the segmentation of fish eye

I follow the instruction from this: https://www.mathworks.com/help/images/detecting-a-cell-using-image-segmentation.html?fbclid=IwAR3MPi9mlL6Rt8R7sA20oyZdvoiMhIpxpW-dxjSUfmoNcX-g9u2_5cn6V3Y

I changed some parameters and successfully segmented the fish eye like the image I attached.

However, for the fish that is stale like the image attached, the code couldn't segment the eye.

Can anyone help me to solve the problem ? And how can I segment the fish eye successfully in any situation (Fresh and stale fishes) ?

Thanks in advance.

Here is my code:

I = imread('');
I = imresize(I,[500 500]);
I = rgb2gray(I);

[~,threshold] = edge(I,'sobel');
fudgeFactor = 1;
BWs = edge(I,'sobel',threshold * fudgeFactor);
figure
imshow(BWs)
title('Binary Gradient Mask')

se90 = strel('line',1,90);
se0 = strel('line',1,0);

BWsdil = imdilate(BWs,[se90 se0]);
figure
imshow(BWsdil)
title('Dilated Gradient Mask')

BWdfill = imfill(BWsdil,'holes');
figure
imshow(BWdfill)
title('Binary Image with Filled Holes')

BWnobord = imclearborder(BWdfill,4);
figure
imshow(BWnobord)
title('Cleared Border Image')

seD = strel('diamond',1);
BWfinal = imerode(BWnobord,seD);
BWfinal = imerode(BWfinal,seD);
figure
imshow(BWfinal)
title('Segmented Image');

BWmask = labeloverlay(I,BWfinal);
figure
imshow(BWmask)
title('Mask Over Original Image')

BWoutline = bwperim(BWfinal);
Segout = I; 
Segout(BWoutline) = 255; 
figure
imshow(Segout)
title('Outlined Original Image')

maskedGrayImage = bsxfun(@times, I, cast(BWfinal, 'like', I));
figure
imshow(maskedGrayImage)

Answer 1

[Image Analyst]

Try this:

clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
clearvars;
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 16;

%=======================================================================================
% Have user browse for a file, from a specified "starting folder."
% For convenience in browsing, set a starting folder from which to browse.
% startingFolder = pwd;  % or 'C:\wherever';
% if ~exist(startingFolder, 'dir')
% 	% If that folder doesn't exist, just start in the current folder.
% 	startingFolder = pwd;
% end
% % Get the name of the file that the user wants to use.
% defaultFileName = fullfile(startingFolder, 'n*.*');
% [baseFileName, folder] = uigetfile(defaultFileName, 'Select a file');
% if baseFileName == 0
% 	% User clicked the Cancel button.
% 	return;
% end

folder = pwd;
baseFileName = 'IMG_Segment_Failed.JPG';
% baseFileName = 'IMG_Segment_Success.JPEG';
fullFileName = fullfile(folder, baseFileName);
rgbImage = imread(fullFileName);

% Get the dimensions of the image.
[rows1, columns1, numberOfColorChannels1] = size(rgbImage)
% Display the original image.
subplot(2, 2, 1);
imshow(rgbImage, []);
axis('on', 'image');
caption = sprintf('Original Color Image\n"%s"', baseFileName);
title(caption, 'FontSize', fontSize, 'Interpreter', 'None');
drawnow;
hp = impixelinfo(); % Set up status line to see values when you mouse over the image.

% Set up figure properties:
% Enlarge figure to full screen.
set(gcf, 'Units', 'Normalized', 'OuterPosition', [0 0.05 1 0.95]);
% Get rid of tool bar and pulldown menus that are along top of figure.
% set(gcf, 'Toolbar', 'none', 'Menu', 'none');
% Give a name to the title bar.
set(gcf, 'Name', 'Demo by Image Analyst', 'NumberTitle', 'Off')

[mask, maskedRGBImage] = createMask(rgbImage);
% Display the image.
subplot(2, 2, 2);
imshow(mask, []);
axis('on', 'image');
title('Initial Mask Image', 'FontSize', fontSize, 'Interpreter', 'None');
drawnow;
hp = impixelinfo(); % Set up status line to see values when you mouse over the image.

% Do area filtering.
mask = bwareafilt(mask, [30000, 60000]);
% Display the image.
subplot(2, 2, 3);
imshow(mask, []);
axis('on', 'image');
title('Mask Image after area filtering', 'FontSize', fontSize, 'Interpreter', 'None');
drawnow;
hp = impixelinfo(); % Set up status line to see values when you mouse over the image.

% Measure things in the mask
labeledImage = bwlabel(mask);
props = regionprops(labeledImage, 'Area', 'Perimeter');
allAreas = [props.Area];
% sortedAreas = sort(allAreas)
allPerims = [props.Perimeter];
circularities = allPerims .^ 2 ./ (4 * pi * allAreas)
subplot(2, 2, 4);
% histogram(circularities);
% grid on;
% Extract the roundest blob - the one with circularity closest to 1
[~, index] = min(abs(circularities - 1))
mask = ismember(labeledImage, index);
imshow(mask, []);
axis('on', 'image');
title('Mask Image after circularity filtering', 'FontSize', fontSize, 'Interpreter', 'None');
drawnow;
hp = impixelinfo(); % Set up status line to see values when you mouse over the image.



function [BW,maskedRGBImage] = createMask(RGB)
%createMask  Threshold RGB image using auto-generated code from colorThresholder app.
%  [BW,MASKEDRGBIMAGE] = createMask(RGB) thresholds image RGB using
%  auto-generated code from the colorThresholder app. The colorspace and
%  range for each channel of the colorspace were set within the app. The
%  segmentation mask is returned in BW, and a composite of the mask and
%  original RGB images is returned in maskedRGBImage.

% Auto-generated by colorThresholder app on 19-Dec-2019
%------------------------------------------------------


% Convert RGB image to chosen color space
I = rgb2hsv(RGB);

% Define thresholds for channel 1 based on histogram settings
channel1Min = 0.254;
channel1Max = 0.991;

% Define thresholds for channel 2 based on histogram settings
channel2Min = 0.000;
channel2Max = 1.000;

% Define thresholds for channel 3 based on histogram settings
channel3Min = 0.000;
channel3Max = 0.523;

% Create mask based on chosen histogram thresholds
sliderBW = (I(:,:,1) >= channel1Min ) & (I(:,:,1) <= channel1Max) & ...
	(I(:,:,2) >= channel2Min ) & (I(:,:,2) <= channel2Max) & ...
	(I(:,:,3) >= channel3Min ) & (I(:,:,3) <= channel3Max);
BW = sliderBW;

% Initialize output masked image based on input image.
maskedRGBImage = RGB;

% Set background pixels where BW is false to zero.
maskedRGBImage(repmat(~BW,[1 1 3])) = 0;

end

It might need some tweaking to get it to work with all images.

Or you could try deep learning. I tried imfindcircles() and was not very successful with that function.

[Minh Le]

Thanks but the threshold you get from the image is from the colorthresholder app and when apply your code to differrent images, some is success and some failed due to different color threshold.

I am focusing on fish eye segmentation regardless of being fresh or stale. We are building up a database which comprises a bunch of fish eye image and I really need to automactically sucessfully segment the fish eye of all image in the database

I saw you recommend deep learning but is there any other ways to this problem ?

[Image Analyst]

You can use color segmentation like I did with fixed thresholds if you have good control over your lighting and exposure. If you don't, you're in trouble because a clear/good eye might look cloudy/whitish/hazy/bad with higher levels of exposure or lighting. In that case you'll have to use a variable exposure but even then you'll have to have some standard to do a color standardization on. Like hold an x-rite ColorChecker Passport chart next to each fish head.

I don't think you'll just be able to roam around the dock taking cell phone phots and reliably telling if the eye is good or bad since the reflections, angles, lighting, and exposure will all be varying and what's good could look bad with the wrong image capture conditions and what's bad could look good under some conditions.

[Minh Le]

Thanks for your help last week.

Yes I knew that this is very hard to say that the fish is good/bad by taking photo but I'm really keen on this

I try to adjust my old code a little bit with the function to increase contrast named adapthisteq

and also use your code to extract blobs with circularities closest to 1.

Now, I am thinking about how to segment the whole fish's eyes (not just the black iris but also the white iris)

Some of my images worked well with my code and the result was like this:

However, in overall, for most of the image I tested, they were segmented with black iris.

So can you help me to segment the whole fish eye (any way like preprocessing or using filters, etc.)

Another thing is that as you can see in the picture, the extract blob funtion from your code cound not work with this 2 image (after using extract blob function- the results are on the right side) but in some cases, it did

Can you help me again to segment the whole fish eyes and then extract the image with just the eye segmented.

Thank you again

I will give you 2 images attached

My old code:

I = imread('');

I = imresize(I,[500 500]);

I = rgb2gray(I);

% Increase contrast in gray image

I= adapthisteq(I);

figure

imshow(I)

[~,threshold] = edge(I,'canny');

fudgeFactor = 1.2;

BWs = edge(I,'canny',threshold * fudgeFactor);

figure

imshow(BWs)

title('Binary Gradient Mask')

% Stretch image

se90 = strel('line',1,90);

se0 = strel('line',1,0);

BWsdil = imdilate(BWs,[se0 se90]);

figure

imshow(BWsdil)

title('Dilated Gradient Mask')

BWdfill = imfill(BWsdil,'holes');

figure

imshow(BWdfill)

title('Binary Image with Filled Holes')

% BWnobord = imclearborder(BWdfill,4);

% figure

% imshow(BWnobord)

% title('Cleared Border Image')

seD = strel('sphere',1);

% BWfinal = imerode(BWnobord,seD);

BWfinal = imerode(BWdfill,seD);

figure

imshow(BWfinal)

title('Segmented Image');

BWmask = labeloverlay(I,BWfinal);

figure

imshow(BWmask)

title('Mask Over Original Image')

% BWoutline = bwperim(BWfinal);

% Segout = I;

% Segout(BWoutline) = 255;

% figure

% imshow(Segout)

% title('Outlined Original Image')

% Measure things in the mask

BWfinal = bwlabel(BWfinal);

props = regionprops(BWfinal, 'Area', 'Perimeter');

allAreas = [props.Area];

allPerims = [props.Perimeter];

circularities = allPerims .^ 2 ./ (4 * pi * allAreas)

% Extract the roundest blob - the one with circularity closest to 1

[~, index] = min(abs(circularities - 1))

mask = ismember(BWfinal, index);

figure

imshow(mask)

% %Extract the gray masked image

% maskedGrayImage = bsxfun(@times, I, cast(mask, 'like', I));

% figure

% imshow(maskedGrayImage)