32Bit RGBA Image

Hello,

i read via MicroMager Core succseedfull images in Graymode - But the device give me an 32bit RGBA Image that would not

display with function imshow()

import mmcorej.*

mmc = CMMCore;
mmc.unloadAllDevices();
mmc.loadSystemConfiguration ('C:\Program Files\Micro-Manager-2.0gamma\DijSDKcameraMgr.cfg');
mmc.prepareSequenceAcquisition('DijSDKcamera');
mmc.startContinuousSequenceAcquisition(0);

imgRaw = mmc.getLastImage(); %mmc.snapImage(); %img = mmc.getImage();

pixelType = 'uint8';%8BIT GRAY default

if depth == 4 
   pixelType = 'uint32';%32BIT RGBA can't display img as 32BIT RGBA only 16Bit RGB is possible ?
elseif depth == 2 
   pixelType = 'uint16';%16BIT RGB not supportet by Micromanger normaly 32bit or 8bit
end
    
img = typecast(imgRaw,pixelType);
img = reshape(img, [width, height]);
img = transpose(img);

cla reset; 

imshow(img);

mmc.unloadAllDevices(); %complettly destruct MMCore

How i can convert tzhe image correctly ? The Funktion image(img) Display an point cloud of image, but not in real colors given from map...

thx for any suggestions

Karsten (www.FlexxVision.de)

[Image Analyst]

I don't know what MicroMager Core and Graymode are, but what does this show:

whos img
whos imgRaw

[Karsten Schulz]

Hello Analyst,

thank u for answer, the image Aquisitation sample comes from other thread :

https://de.mathworks.com/matlabcentral/answers/404012-creating-an-acquisition-video-using-micromanager-matlab-core

I have adapted to my solution, all works nice so far.

But the 32 bit RGBA image can't display . works only with Gray 8bit Images for me.

Best regards

Karsten

[Guillaume]

Please answer Image Analyst question. We need to know the type of imgRaw that's returned by your device. Is it double, uint8 or something else? We also need to know the size. whos imgRaw will give us all that information.

Once we know that, we can probably figure out a way to display your image. It's probably trivial, just split the transparency channel from the colour channels.

Also, doesn't your library document the memory arrangement of the data returned by getLastImage? That would also be useful information for us to have.

[Karsten Schulz]

My Camera give me back an RGBA Image R8|G8|B8|A8 = 32 Bit . The alpha channel is zero always i have written the Micromanger Core driver self , so i can say there works in all other cases.

For each colorchannel 8bit its an BYTE Array.

size is : width*height*(bpp/3)

it's also an normal BYTE Buffer , that is waht lastimage or getimage give me back : an field of bytes.

it's not an 16Bit RGB that comes from webcams. But i use High end Color camera in Microscopy.

thx for Help !

Regards Karsten

Answer 1

[Guillaume]

"My Camera give me back an RGBA Image R8|G8|B8|A8 = 32 Bit"

"size is : width*height*(bpp/3)"

Shouldn't that be width*height*(bpp/4) ?

Assuming it's 4 bytes per pixel, and that imgRaw is indeed of class uint8:

img = permute(reshape(imgRaw, 4, width, height), [3, 2, 1])
imgRGB = img(:, :, 1:3);  %discard alpha channel which is all 0
imshow(imgRGB);

This assumes that the image pixel are stored in row-major order, which your code also assumed.

[Karsten Schulz]

In first Thank's for helping, with this formation reshape i have try tests and got an series of errors :

Error using images.internal.imageDisplayValidateParams>validateCData (line 131)

RGB images must be uint8, uint16, single, or double.

Error in images.internal.imageDisplayValidateParams (line 30)

common_args.CData = validateCData(common_args.CData,image_type);

Error in images.internal.imageDisplayParseInputs (line 79)

common_args = images.internal.imageDisplayValidateParams(common_args);

Error in imshow (line 253)

images.internal.imageDisplayParseInputs({'Parent','Border','Reduce'},preparsed_varargin{:});

 drawnow();%pause(0) doe's the same: In this case think it's poll an msg handler..
    
    picnr = picnr + 1; %incrase actioncounter
    
    try
      imgRaw = mmc.getLastImage(); %mmc.snapImage(); %img = mmc.getImage();
    catch
      continue; %format change or buffer MMCore failure
    end
    
    width  = mmc.getImageWidth();
    height = mmc.getImageHeight();
    depth  = mmc.getBytesPerPixel();
  
    cla reset; %reset any what for new frame, without u see all times the first image
    
    img = permute(reshape(imgRaw, 4, width, height), [3, 2, 1])
    
    imgRGB = img(:, :, 1:3);  %discard alpha channel which is all 0
    
    imshow(imgRGB);

[Guillaume]

As I wrote, Assuming [..] that imgRaw is indeed of class uint8

Clearly, that's not the case. Still waiting for the output of whos imgRaw which would clarify that instantly...

But we probably need an explanation as to why it's not uint8 (aka BYTE).

[Karsten Schulz]

I have explained two posts befor, its an BYTE linear memory block each DWORD is filled with rgba

so has each pixel 4Byte and the alpha channel is always null

But if is usenow image(imgRGB); instand imshow(imgRGB);

i take an better picture see attachment, but not the rigth one, i hope i can find the problem.

[Guillaume]

I understood the memory layout the first time round, it still doesn't tell us what the type is in matlab. So ONCE AGAIN, what is the output of

whos imgRaw

we could also do with the values of width, height, depth for sanity check.

[Guillaume]

Ah, Finally! Not what was asked, but I can see from your massive screenshot that imgRaw is int8, that's a signed byte. Matlab doesn't use signed integers to store images. If actual values don't need to be preserved, just relative intensity, the image could be converted to uint8. Otherwise, double for a massive increase in memory consumption.

So,

img = permute(reshape(imgRaw, 4, width, height), [3, 2, 1])
imgRGB = img(:, :, 1:3);  %discard alpha channel which is all 0
imguint8 = typecast(bitxor(imgRGB, -128), 'uint8');  %rescales -128:127 to 0:255
imshow(imgRGB);

or

img = permute(reshape(imgRaw, 4, width, height), [3, 2, 1])
imgRGB = img(:, :, 1:3);  %discard alpha channel which is all 0
imgdouble = double(imgRGB); %will result in values from -128 to 127
imshow(imgdouble, [-128 127]);  %so have to tell imshow that range is not 0-1

[Karsten Schulz]

i will in first display the data only that all works on 8bit gray. what mathlab use for an pixel i dont now.

in the output i have now an image in false color from the function image(imgRGB);

but not the preview picture is it me equal how matlab convert the image in own datatypes i will only see the image inthe right bar , your transform arry function do also something right . thx for hep me here.

all values from my imgRaw are 0-255 per channel .

Your result with the function image can see in attachment, i hope i can bring out it with imgShow in right colors

wifth = 1800 height = 1200 depth 4

[Karsten Schulz]

Ohh holly shit much better ...

see the result, in attachment, i will try understod this operations , i cant see this image in my live output after stop aquiring i see this wrong color one but much better

im c/c++ programmer and a bit confused with the high universal matrix operations here in matlab sorry i work on it

we can bring out an better one like the original ?

[Karsten Schulz]

Hi Guillaume,

now i have integrate your code sniped into the fall case 8Bit gry or 32Bit RGB (0-255) per channel.

Resume are two different pictures in attachment with code and output .

ImageA5.jpg gray with this code

imageA4.jpg color with this code

Only the Gray Mode 8Bit image works succseed full. The Matrix convert operation

send many text into the output can i stop the text output of conversation so i see only the images ?

I use

imshow()

for top picture and

image()

for the bottom one

in Gry mode i see my camera images in realtime on the output based on :

drawnow();

and

cla reset;

that the code for now : (dosent work correctly on color images 32bit rgba

%MATLAB setup for   MICROSCOPY CAMERAMODULES VIA MICROMANGER2.0-GAMMA 
%from 18.02.2020 www.FlexxVision.de
%https://valelab4.ucsf.edu/~MM/doc/MMCore/html/class_c_m_m_core.html
%https://stackoverflow.com/questions/30855352/rgba-image-from-intel-realsense-to-matlab
%https://micro-manager.org/wiki/Using_the_Micro-Manager_python_library

%edit classpath.txt
%YOU HAVE TO ADD CLASS PATH ! CHECK OUR OWN PATHES HERE
%C:\Program Files\Micro-Manager-2.0gamma\ij.jar
%C:\Program Files\Micro-Manager-2.0gamma\plugins\Micro-Manager\MMCoreJ.jar
%C:\Program Files\Micro-Manager-2.0gamma\plugins\Micro-Manager\MMJ_.jar
%C:\Program Files\Micro-Manager-2.0gamma\plugins\Micro-Manager\swingx-0.9.5.jar

clear java %destruct references
import mmcorej.*

try
    mmc = CMMCore;
    mmc.unloadAllDevices();
    mmc.loadSystemConfiguration ('C:\Program Files\Micro-Manager-2.0gamma\DijSDKcameraMgr.cfg');
catch
    mmc.unloadAllDevices(); %complettly destruct MMCore
    msgbox("Initalizing failed (Old Instance is already running)");
    return
end
    
try
    mmc.setProperty('DijSDKcamera', '00-Extend property','Show');
    
    %mmc.setProperty('DijSDKcamera', '02-Camera select', 'GRYPHAX::ARKTUR::0300000263'); 
    %mmc.setProperty('DijSDKcamera', '03-Live', '1920 x 1080'); 
    %mmc.setProperty('DijSDKcamera', '04-Capture shot', '3840 x 2160'); 
    
    %mmc.setProperty('DijSDKcamera', '02-Camera select', 'GRYPHAX::NAOS::0200000225'); 
    %mmc.setProperty('DijSDKcamera', '03-Live', '1800 x 1200'); 
    %mmc.setProperty('DijSDKcamera', '04-Capture shot', '5400 x 3600'); 
    
    mmc.setProperty('DijSDKcamera', '01-Color mode', 'Gray'); 
catch
    mmc.unloadAllDevices(); %complettly destruct MMCore
    msgbox("PropSet failed");
    return
end

try
    mmc.prepareSequenceAcquisition('DijSDKcamera');
    mmc.startContinuousSequenceAcquisition(0); %%mmc.StartSequenceAcquisition(1,1,1);
catch
    mmc.unloadAllDevices(); %complettly destruct MMCore
    msgbox("StartAcq failed");
    return
end

picnr = 0;

while picnr < 30  %capture fast 30 images without snapImage (scan version)
    
    drawnow();%pause(0) doe's the same: In this case think it's poll an msg handler.. 
    cla reset; %reset any what for new frame, without u see all times the first image
    
    picnr = picnr + 1; %incrase actioncounter
    
    try
      imgRaw = mmc.getLastImage(); %mmc.snapImage(); %img = mmc.getImage();
    catch
      continue; %format change or buffer MMCore failure
    end
    
    width  = mmc.getImageWidth();
    height = mmc.getImageHeight();
    depth  = mmc.getBytesPerPixel();
    
    if depth == 4 
     pixelType = 'uint32';%32BIT RGBA can't display img as 32BIT RGBA only 16Bit RGB is possible ?
     img = permute(reshape(imgRaw, 4, width, height), [3, 2, 1])
     img = img(:, :, 1:3);  %discard alpha channel which is all 0
         %imguint8 = typecast(bitxor(imgRGB, -127), 'uint8');  %rescales -128:127 to 0:255
     img = double(img); %will result in values from -128 to 127
    elseif depth == 2 
     pixelType = 'uint16';%16BIT RGB not supportet by Micromanger normaly 32bit or 8bit
     return; %cant handle
    elseif depth == 1
     pixelType = 'uint8';%8BIT GRAY default
     img = typecast(imgRaw,pixelType);
     img = reshape(img, [width, height]);
     img = transpose(img);
    end

    subplot(2,1,1);
    imshow(img);%show bw 8 only, (dosen't work here by given RGBA32 from MMCore  
    subplot(2,1,2);
    image(img); %show as point cloud 32bit RGBA will be display ugly 
   
end 

mmc.unloadAllDevices(); %complettly destruct MMCore

disp("End capture");

sorry for my easy english

best regards from Berlin

Karsten

[Guillaume]

Can you attach a mat file containing:

• imgRaw
• width
• height

when depth == 4.

Also, I've just noticed that in the depth==1 you just typecast to uint8, which would hint that imgRaw should have been returned as uint8 to start with (as I initially assumed) and that your library returns the wrong type. In that case:

img = permute(reshape(typecast(imgRaw, 'uint8'), 4, width, height), [3, 2, 1])
imgRGB = img(:, :, 1:3);  %discard alpha channel which is all 0
imshow(imgRGB);

should work.

[Karsten Schulz]

Ah the new code snipe works !

Need 1 Minute per frame on i7x6950 ten core :)

Is it possible to speed up ? That works men , nice thank u then tey must wait for an image a while

Can we rotate R an B channel ?

Can i stop Text Output generated by conversion ?

Many thx for all the help

k

[Guillaume]

"Is it possible to speed up ?"

The expensive part is the rearranging from RGBARGBARGBA... to RRRR...GGGG...BBBB...AAAA... and the transposition from row-major to column-major. Both of these are required for the image to be displayed properly by matlab and they're both done at once with the permute, so the only way that could be sped up would be by a change in the mmc library.

"Can i stop Text Output generated by conversion"

Sure, just add the semicolon af the end of

img = permute(reshape(typecast(imgRaw, 'uint8'), 4, width, height), [3, 2, 1]);

that I forgot by mistake. It will probably help with speed as well.

"Can we rotate R an B channel ?"

If you mean swap:

imgRGB = img(:, [3, 2, 1]);  %discard alpha and swap R and B

Note that the swapping is also an expensive operation.

[Karsten Schulz]

Taht work's and speed's up much by add the semicolon, never seen befor^^ THX!

But the statement img = img(:, [3, 2, 1]); versus img = img(:, :, 1:3);

doe's not the conversation right. And yes i will an convert part integrate in the MMCore 's camera dll will see how i can signal it,

So far very thanks for help, i can work now with that. And think in this thread are some questions about MMCore resolved.

Many Thx from Berlin

Karsten

[Guillaume]

"But the statement [..] doe's not the conversation right"

It's probably because I haven't understand what you want. The code swaps the red and blue channel.

Anyway, if your question is answered please accept the answer.

[Karsten Schulz]

Nope is not swap R and B. It's destruct the image result last view.

documented in the middle of attached project , all wors nice only the R B Swap dosent work atm.

thx

K.

[Guillaume]

Oh! I made a typo. It should have been:

imgRGB = img(:, :, [3, 2, 1]);  %was missing one :

[Karsten Schulz]

Oh Holly Shit men, wonderfull , it's works and do ~8 FPS that is more what i have hope.

Very thx men , and if u need anything in Vision let me now. All Proplematics solved thx

Best regards

Karsten