How to implement Grad-CAM in 3D?

Hi,

I've got a trained 3D CNN model and I would like to visualize its classification result. However, Grad-CAM seems to be implemented for 2D models for now. To be more exact, the predict function in gradcam expects the input to be 2D image:

dlImg = dlarray(single(img),'SSC'); % 224x224x224 dlarray
dlnet = dlnetwork(lgraph); % 1x1 dlnetwork
%Specify the names of the softmax and feature map layers to use with the Grad-CAM helper function. For the feature map layer, specify either the last ReLU layer with non-singleton spatial dimensions, or the last layer that gathers the outputs of ReLU layers (such as a depth concatenation or an addition layer). If your network does not contain any ReLU layers, specify the name of the final convolutional layer that has non-singleton spatial dimensions in the output. Use the function analyzeNetwork to examine your network and select the correct layers. For GoogLeNet, the name of the softmax layer is 'prob' and the depth concatenation layer is 'inception_5b-output'.
softmaxName = 'prob';
featureLayerName = 'res5b_relu';
%To use automatic differentiation, convert the sherlock image to a dlarray.
%Compute the Grad-CAM gradient for the image by calling dlfeval on the gradcam function.
[featureMap, dScoresdMap] = dlfeval(@gradcam, dlnet, dlImg, softmaxName, featureLayerName, classfn);

function [featureMap,dScoresdMap] = gradcam(dlnet, dlImg, softmaxName, featureLayerName, classIdx)
    [scores,featureMap] = predict(dlnet, dlImg, 'Outputs', {softmaxName, featureLayerName});
    classScore = scores(classIdx);
    dScoresdMap = dlgradient(classScore,featureMap);
end

This function is used and is taken from the example Grad-CAM Reveals the Why Behind Deep Learning Decisions. The exact error I get:

Error using dlfeval (line 43)
Layer 'VideoInputLayer': Invalid input data. Invalid number of spatial dimensions. Layer expects 3 but received 2.

Error in adniviz (line 58)
    [featureMap, dScoresdMap] = dlfeval(@gradcam, dlnet, dlImg, softmaxName, featureLayerName, classfn);

I get the same error with just:

[scores,featureMap] = predict(dlnet, dlImg, 'Outputs', {softmaxName, featureLayerName});

Error using dlnetwork/predict (line 406)
Layer 'VideoInputLayer': Invalid input data. Invalid number of spatial dimensions. Layer expects 3 but received 2.

[Juuso Korhonen]

I actually got it. The dlImg was in wrong format. For volumes, it has to be:

dlImg = dlarray(single(img),"SSSCB"); 

[Ava Dolores]

Hello,

I have been training my CNN on alzheimer's vs control 3D Nii files of the brain. I want to use gradCAM to visualize why I am getting specific classification results.

Using your code as reference, I am getting a 5-D dlarray for the featureMap and dScoresdMap. I beleive this is because when using dlarray we are putting in SSSCB, so the channel and batch are the fourth and fifth dimensions. The 5-D dlarray makes it very difficult to visualize the gradCAM data using volshow becuase it is not a 3D array.

Why am I getting a 5-D array, how can I fix this, and how should I visualize the data?

I have attached my workspace, my modified version of your code, and the error I get when trying volshow.

Thank you for your time and help!