Matlab trainNetwork: The output size of the last layer does not match the response size. Semantic classification.

dataSetDir = fullfile('Airplane');
imageDir = fullfile(dataSetDir,'images');
labelDir = fullfile(dataSetDir,'labels');

% .mat files each with size [1 X 3]
imds = customDataStore(imageDir);

classNames = ["body" "engine" "tail" "wing"];
labelIDs   = [1 2 3 4];

% read corresponding png labels size:[1 X], where each pixel is 1,2,3 or 4
pxds = pixelLabelDatastore(labelDir,classNames,labelIDs);

pximds = pixelLabelImageDatastore(imds,pxds);

lgraph = pixelSeg(1024, 4);
% analyzeNetwork(lgraph)

options = trainingOptions('adam', ...
    'MaxEpochs',20,...
    'InitialLearnRate',0.001, ...
    'MiniBatchSize', 128, ...
    'Plots','training-progress');

net = trainNetwork(pximds, lgraph, options);

Error using trainNetwork (line 170)

Invalid training data. The output size ([1 1024 4]) of the last layer does not match the response size ([1 1 4]).

pixelSeg in a custom network with a PixelClassificationLayer as the output layer (it should have an output size of [1 1024 4] because the image input is [1 1024 3] and theres 4 output categories). How come the response size changed from [1 1024] to [1 1 4]? I'm pretty sure it is not network that is wrong because it does accepts the right input and gives out the right output.

I've also refered to: trainNetwork with response as categorical tensor which suggests do use pixelLabelImagedatastore. Does it have some limitiation with using single row images?

Answer 1

[faris azhari]

I was right. i solved the problem by changing:

1. The size of the inputs [1 X 3] to [Y Z 3], where Y*Z=X
2. The size of the labels [1 X 1] to [Y Z 1], where Y*Z=X
3. Add a custom layer after the input layer to convert the input to [1 X 3] so that it could be used in the network.
4. Add a custom layer before softmax layer to convert the output [1 X numClass] back to [Y Z numClasses] so that it could be compared to the response labels

Therefore, I would say the problem is either one or all of these functions [imageDatastore, pixelLabelDatastore, pixelLabelImageDatastore] where it doesn't know how to handle single row or column images (note: I also tried having inputs and labels of the size [X 1 3] and [X 1 1] respectively but received the same error shown in the question).