How to concatenate video frames into single matrix for cnn?

I have multiple videos. Each video has different number of frames. I am trying to combine all the frames of all videos for training of CNN. But I faced errors.

[KSSV]

Each frame should be a input right? Why you want to concatenate? Do you want to concatente along 3rd dimension? Show us your code, along with error.

[Shilpa Sonawane]

clear all;

close all;

clc;

idx=1;

files={'trainsp1_A1.avi';'trainsp1_A2.avi';'trainsp1_B1.avi';'trainsp1_B2.avi';}%'trainsp1_A5.avi';'trainsp1_A6.avi';'trainsp1_A7.avi';};

numFiles = numel(files);

all_frames=[]

for mm=1:numFiles

video = readVideo(files{mm});

all_vid{mm}=video;

fprintf("Reading file %d of %d...\n", i, numFiles)

[s1 s2 s3 s4]=size(video);

for ii=1:s4

comb_video=video(:,:,:,ii);

all_frames= [all_frames ;comb_video];

[t1 t2 t3]=size(all_frames)

end

end

aa=[ones(1,31) ones(1,29)]';

sslbl= categorical(aa);

imageAugmenter = imageDataAugmenter;

imageSize = [38 62 3];

augimds = augmentedImageDatastore(imageSize,all_frames,sslbl','DataAugmentation',imageAugmenter);

layers = [

imageInputLayer(imageSize)

convolution2dLayer(3,8,'Padding','same')

batchNormalizationLayer

reluLayer

maxPooling2dLayer(2,'Stride',2)

convolution2dLayer(3,16,'Padding','same')

batchNormalizationLayer

reluLayer

maxPooling2dLayer(2,'Stride',2)

convolution2dLayer(3,32,'Padding','same')

batchNormalizationLayer

reluLayer

fullyConnectedLayer(2)

softmaxLayer

classificationLayer];

%

% opts = trainingOptions('sgdm', ...

% 'MaxEpochs',25, ...

% 'Shuffle','every-epoch', ...

% 'Plots','training-progress', ...

% 'Verbose',false, ...

% 'ValidationData',{XValidation,YValidation});

opts = trainingOptions('sgdm', ...

'MaxEpochs',25, ...

'Shuffle','every-epoch', ...

'Plots','training-progress', ...

'Verbose',false);

net = trainNetwork(augimds,layers,opts);

____________________________________

Error using augmentedImageDatastore (line 251)

Number of observations in X and Y disagree.

Error in JULY6_2022 (line 23)

augimds = augmentedImageDatastore(imageSize,all_frames,sslbl','DataAugmentation',imageAugmenter);

_______________________________

Total 4 videos. First 2 videos for'A' letter & remaining 2 are for 'B' letter.

First video has 17 frames. second video has 14 frames. so for 'A' letter , total frames are 31.

For 'B' letter, total frames are 29. I have concatednated all frames of 4 videos vertically. Therefore 'all_frames' variable size is '2280x62x3'.

While labelling, I have done this. 'aa=[ones(1,31) ones(1,29)]';'

what is expected while labelling?

Please guide.

Thank you.

[Shilpa Sonawane]

clear all;

close all;

clc;

global acnt bcnt ccnt dcnt ecnt fcnt gcnt hcnt icnt jcnt kcnt lcnt mcnt;

global ncnt ocnt pcnt qcnt rcnt scnt tcnt ucnt vcnt wcnt xcnt ycnt zcnt;

acnt=0,bcnt=0,ccnt=0,dcnt=0,ecnt=0,fcnt=0,gcnt=0,hcnt=0,icnt=0,jcnt=0,kcnt=0, lcnt=0, mcnt=0, ...

ncnt=0, ocnt=0, pcnt=0, qcnt=0, rcnt=0, scnt=0, tcnt=0,ucnt=0,vcnt=0,wcnt=0,xcnt=0,ycnt=0,zcnt=0;

global acntr bcntr ccntr dcntr ecntr fcntr gcntr hcntr icntr jcntr kcntr lcntr mcntr;

global ncntr ocntr pcntr qcntr rcntr scntr tcntr ucntr vcntr wcntr xcntr ycntr zcntr;

acntr=0,bcntr=0,ccntr=0,dcntr=0,ecntr=0,fcntr=0,gcntr=0,hcntr=0,icntr=0,jcntr=0,kcntr=0, lcntr=0, mcntr=0, ...

ncntr=0, ocntr=0, pcntr=0, qcntr=0, rcntr=0, scntr=0, tcntr=0,ucntr=0,vcntr=0,wcntr=0,xcntr=0,ycntr=0,zcntr=0;

idx=1;

index2=1;

files={'trainsp1_A1.avi';'trainsp1_A2.avi';'trainsp1_A3.avi';'trainsp1_A4.avi';'trainsp1_A5.avi';'trainsp1_A6.avi';'trainsp1_A7.avi'; ...

'trainsp1_B1.avi';'trainsp1_B2.avi';'trainsp1_B3.avi';'trainsp1_B4.avi';'trainsp1_B5.avi';'trainsp1_B6.avi';'trainsp1_B7.avi'; ...

'trainsp1_C1.avi';'trainsp1_C2.avi';'trainsp1_C3.avi';'trainsp1_C4.avi';'trainsp1_C5.avi';'trainsp1_C6.avi';'trainsp1_C7.avi'; ...

'trainsp2_C1.avi';'trainsp2_C2.avi';'trainsp2_C3.avi';'trainsp2_C4.avi';'trainsp2_C5.avi';'trainsp2_C6.avi';'trainsp2_C7.avi'; ...

'trainsp3_C1.avi';'trainsp3_C2.avi';'trainsp3_C3.avi';'trainsp3_C4.avi';'trainsp3_C5.avi';'trainsp3_C6.avi';'trainsp3_C7.avi'; ...

'trainsp4_C1.avi';'trainsp4_C2.avi';'trainsp4_C3.avi';'trainsp4_C4.avi';'trainsp4_C5.avi';'trainsp4_C6.avi';'trainsp4_C7.avi'; ...

'trainsp5_C1.avi';'trainsp5_C2.avi';'trainsp5_C3.avi';'trainsp5_C4.avi';'trainsp5_C5.avi';'trainsp5_C6.avi';'trainsp5_C7.avi'; ...

'trainsp1_D1.avi';'trainsp1_D2.avi';'trainsp1_D3.avi';'trainsp1_D4.avi';'trainsp1_D5.avi';'trainsp1_D6.avi';'trainsp1_D7.avi'; ...

'trainsp2_D1.avi';'trainsp2_D2.avi';'trainsp2_D3.avi';'trainsp2_D4.avi';'trainsp2_D5.avi';'trainsp2_D6.avi';'trainsp2_D7.avi'; ...

'trainsp3_D1.avi';'trainsp3_D2.avi';'trainsp3_D3.avi';'trainsp3_D4.avi';'trainsp3_D5.avi';'trainsp3_D6.avi';'trainsp3_D7.avi'; ...

'trainsp4_D1.avi';'trainsp4_D2.avi';'trainsp4_D3.avi';'trainsp4_D4.avi';'trainsp4_D5.avi';'trainsp4_D6.avi';'trainsp4_D7.avi'; ...

'trainsp5_D1.avi';'trainsp5_D2.avi';'trainsp5_D3.avi';'trainsp5_D4.avi';'trainsp5_D5.avi';'trainsp5_D6.avi';'trainsp5_D7.avi'; ...

'trainsp1_E1.avi';'trainsp1_E2.avi';'trainsp1_E3.avi';'trainsp1_E4.avi';'trainsp1_E5.avi';'trainsp1_E6.avi';'trainsp1_E7.avi'; ...

'trainsp2_E1.avi';'trainsp2_E2.avi';'trainsp2_E3.avi';'trainsp2_E4.avi';'trainsp2_E5.avi';'trainsp2_E6.avi';'trainsp2_E7.avi'; ...

'trainsp3_E1.avi';'trainsp3_E2.avi';'trainsp3_E3.avi';'trainsp3_E4.avi';'trainsp3_E5.avi';'trainsp3_E6.avi';'trainsp3_E7.avi'; ...

'trainsp4_E1.avi';'trainsp4_E2.avi';'trainsp4_E3.avi';'trainsp4_E4.avi';'trainsp4_E5.avi';'trainsp4_E6.avi';'trainsp4_E7.avi'; ...

'trainsp5_E1.avi';'trainsp5_E2.avi';'trainsp5_E3.avi';'trainsp5_E4.avi';'trainsp5_E5.avi';'trainsp5_E6.avi';'trainsp5_E7.avi'; ...

'trainsp1_F1.avi';'trainsp1_F2.avi';'trainsp1_F3.avi';'trainsp1_F4.avi';'trainsp1_F5.avi';'trainsp1_F6.avi';'trainsp1_F7.avi'; ...

'trainsp2_F1.avi';'trainsp2_F2.avi';'trainsp2_F3.avi';'trainsp2_F4.avi';'trainsp2_F5.avi';'trainsp2_F6.avi';'trainsp2_F7.avi'; ...

'trainsp3_F1.avi';'trainsp3_F2.avi';'trainsp3_F3.avi';'trainsp3_F4.avi';'trainsp3_F5.avi';'trainsp3_F6.avi';'trainsp3_F7.avi'; ...

'trainsp4_F1.avi';'trainsp4_F2.avi';'trainsp4_F3.avi';'trainsp4_F4.avi';'trainsp4_F5.avi';'trainsp4_F6.avi';'trainsp4_F7.avi'; ...

'trainsp5_F1.avi';'trainsp5_F2.avi';'trainsp5_F3.avi';'trainsp5_F4.avi';'trainsp5_F5.avi';'trainsp5_F6.avi';'trainsp5_F7.avi'; ...

'trainsp1_G1.avi';'trainsp1_G2.avi';'trainsp1_G3.avi';'trainsp1_G4.avi';'trainsp1_G5.avi';'trainsp1_G6.avi';'trainsp1_G7.avi'; ...

'trainsp2_G1.avi';'trainsp2_G2.avi';'trainsp2_G3.avi';'trainsp2_G4.avi';'trainsp2_G5.avi';'trainsp2_G6.avi';'trainsp2_G7.avi'; ...

'trainsp3_G1.avi';'trainsp3_G2.avi';'trainsp3_G3.avi';'trainsp3_G4.avi';'trainsp3_G5.avi';'trainsp3_G6.avi';'trainsp3_G7.avi'; ...

'trainsp4_G1.avi';'trainsp4_G2.avi';'trainsp4_G3.avi';'trainsp4_G4.avi';'trainsp4_G5.avi';'trainsp4_G6.avi';'trainsp4_G7.avi'; ...

'trainsp5_G1.avi';'trainsp5_G2.avi';'trainsp5_G3.avi';'trainsp5_G4.avi';'trainsp5_G5.avi';'trainsp5_G6.avi';'trainsp5_G7.avi'; ...

'trainsp1_H1.avi';'trainsp1_H2.avi';'trainsp1_H3.avi';'trainsp1_H4.avi';'trainsp1_H5.avi';'trainsp1_H6.avi';'trainsp1_H7.avi'; ...

'trainsp2_H1.avi';'trainsp2_H2.avi';'trainsp2_H3.avi';'trainsp2_H4.avi';'trainsp2_H5.avi';'trainsp2_H6.avi';'trainsp2_H7.avi'; ...

'trainsp3_H1.avi';'trainsp3_H2.avi';'trainsp3_H3.avi';'trainsp3_H4.avi';'trainsp3_H5.avi';'trainsp3_H6.avi';'trainsp3_H7.avi'; ...

'trainsp4_H1.avi';'trainsp4_H2.avi';'trainsp4_H3.avi';'trainsp4_H4.avi';'trainsp4_H5.avi';'trainsp4_H6.avi';'trainsp4_H7.avi'; ...

'trainsp5_H1.avi';'trainsp5_H2.avi';'trainsp5_H3.avi';'trainsp5_H4.avi';'trainsp5_H5.avi';'trainsp5_H6.avi';'trainsp5_H7.avi'; ...

'trainsp1_I1.avi';'trainsp1_I2.avi';'trainsp1_I3.avi';'trainsp1_I4.avi';'trainsp1_I5.avi';'trainsp1_I6.avi';'trainsp1_I7.avi'; ...

'trainsp2_I1.avi';'trainsp2_I2.avi';'trainsp2_I3.avi';'trainsp2_I4.avi';'trainsp2_I5.avi';'trainsp2_I6.avi';'trainsp2_I7.avi'; ...

'trainsp3_I1.avi';'trainsp3_I2.avi';'trainsp3_I3.avi';'trainsp3_I4.avi';'trainsp3_I5.avi';'trainsp3_I6.avi';'trainsp3_I7.avi'; ...

'trainsp4_I1.avi';'trainsp4_I2.avi';'trainsp4_I3.avi';'trainsp4_I4.avi';'trainsp4_I5.avi';'trainsp4_I6.avi';'trainsp4_I7.avi'; ...

'trainsp5_I1.avi';'trainsp5_I2.avi';'trainsp5_I3.avi';'trainsp5_I4.avi';'trainsp5_I5.avi';'trainsp5_I6.avi';'trainsp5_I7.avi'; ...

'trainsp1_J1.avi';'trainsp1_J2.avi';'trainsp1_J3.avi';'trainsp1_J4.avi';'trainsp1_J5.avi';'trainsp1_J6.avi';'trainsp1_J7.avi'; ...

'trainsp2_J1.avi';'trainsp2_J2.avi';'trainsp2_J3.avi';'trainsp2_J4.avi';'trainsp2_J5.avi';'trainsp2_J6.avi';'trainsp2_J7.avi'; ...

'trainsp3_J1.avi';'trainsp3_J2.avi';'trainsp3_J3.avi';'trainsp3_J4.avi';'trainsp3_J5.avi';'trainsp3_J6.avi';'trainsp3_J7.avi'; ...

'trainsp4_J1.avi';'trainsp4_J2.avi';'trainsp4_J3.avi';'trainsp4_J4.avi';'trainsp4_J5.avi';'trainsp4_J6.avi';'trainsp4_J7.avi'; ...

'trainsp5_J1.avi';'trainsp5_J2.avi';'trainsp5_J3.avi';'trainsp5_J4.avi';'trainsp5_J5.avi';'trainsp5_J6.avi';'trainsp5_J7.avi'; ...

'trainsp1_K1.avi';'trainsp1_K2.avi';'trainsp1_K3.avi';'trainsp1_K4.avi';'trainsp1_K5.avi';'trainsp1_K6.avi';'trainsp1_K7.avi'; ...

'trainsp2_K1.avi';'trainsp2_K2.avi';'trainsp2_K3.avi';'trainsp2_K4.avi';'trainsp2_K5.avi';'trainsp2_K6.avi';'trainsp2_K7.avi'; ...

'trainsp3_K1.avi';'trainsp3_K2.avi';'trainsp3_K3.avi';'trainsp3_K4.avi';'trainsp3_K5.avi';'trainsp3_K6.avi';'trainsp3_K7.avi'; ...

'trainsp4_K1.avi';'trainsp4_K2.avi';'trainsp4_K3.avi';'trainsp4_K4.avi';'trainsp4_K5.avi';'trainsp4_K6.avi';'trainsp4_K7.avi'; ...

'trainsp5_K1.avi';'trainsp5_K2.avi';'trainsp5_K3.avi';'trainsp5_K4.avi';'trainsp5_K5.avi';'trainsp5_K6.avi';'trainsp5_K7.avi'; ...

'trainsp1_L1.avi';'trainsp1_L2.avi';'trainsp1_L3.avi';'trainsp1_L4.avi';'trainsp1_L5.avi';'trainsp1_L6.avi';'trainsp1_L7.avi'; ...

'trainsp2_L1.avi';'trainsp2_L2.avi';'trainsp2_L3.avi';'trainsp2_L4.avi';'trainsp2_L5.avi';'trainsp2_L6.avi';'trainsp2_L7.avi'; ...

'trainsp3_L1.avi';'trainsp3_L2.avi';'trainsp3_L3.avi';'trainsp3_L4.avi';'trainsp3_L5.avi';'trainsp3_L6.avi';'trainsp3_L7.avi'; ...

'trainsp4_L1.avi';'trainsp4_L2.avi';'trainsp4_L3.avi';'trainsp4_L4.avi';'trainsp4_L5.avi';'trainsp4_L6.avi';'trainsp4_L7.avi'; ...

'trainsp5_L1.avi';'trainsp5_L2.avi';'trainsp5_L3.avi';'trainsp5_L4.avi';'trainsp5_L5.avi';'trainsp5_L6.avi';'trainsp5_L7.avi'; ...

'trainsp1_M1.avi';'trainsp1_M2.avi';'trainsp1_M3.avi';'trainsp1_M4.avi';'trainsp1_M5.avi';'trainsp1_M6.avi';'trainsp1_M7.avi'; ...

'trainsp2_M1.avi';'trainsp2_M2.avi';'trainsp2_M3.avi';'trainsp2_M4.avi';'trainsp2_M5.avi';'trainsp2_M6.avi';'trainsp2_M7.avi'; ...

'trainsp3_M1.avi';'trainsp3_M2.avi';'trainsp3_M3.avi';'trainsp3_M4.avi';'trainsp3_M5.avi';'trainsp3_M6.avi';'trainsp3_M7.avi'; ...

'trainsp4_M1.avi';'trainsp4_M2.avi';'trainsp4_M3.avi';'trainsp4_M4.avi';'trainsp4_M5.avi';'trainsp4_M6.avi';'trainsp4_M7.avi'; ...

'trainsp5_M1.avi';'trainsp5_M2.avi';'trainsp5_M3.avi';'trainsp5_M4.avi';'trainsp5_M5.avi';'trainsp5_M6.avi';'trainsp5_M7.avi'; ...

'trainsp1_N1.avi';'trainsp1_N2.avi';'trainsp1_N3.avi';'trainsp1_N4.avi';'trainsp1_N5.avi';'trainsp1_N6.avi';'trainsp1_N7.avi'; ...

'trainsp2_N1.avi';'trainsp2_N2.avi';'trainsp2_N3.avi';'trainsp2_N4.avi';'trainsp2_N5.avi';'trainsp2_N6.avi';'trainsp2_N7.avi'; ...

'trainsp3_N1.avi';'trainsp3_N2.avi';'trainsp3_N3.avi';'trainsp3_N4.avi';'trainsp3_N5.avi';'trainsp3_N6.avi';'trainsp3_N7.avi'; ...

'trainsp4_N1.avi';'trainsp4_N2.avi';'trainsp4_N3.avi';'trainsp4_N4.avi';'trainsp4_N5.avi';'trainsp4_N6.avi';'trainsp4_N7.avi'; ...

'trainsp5_N1.avi';'trainsp5_N2.avi';'trainsp5_N3.avi';'trainsp5_N4.avi';'trainsp5_N5.avi';'trainsp5_N6.avi';'trainsp5_N7.avi'; ...

'trainsp1_O1.avi';'trainsp1_O2.avi';'trainsp1_O3.avi';'trainsp1_O4.avi';'trainsp1_O5.avi';'trainsp1_O6.avi';'trainsp1_O7.avi'; ...

'trainsp2_O1.avi';'trainsp2_O2.avi';'trainsp2_O3.avi';'trainsp2_O4.avi';'trainsp2_O5.avi';'trainsp2_O6.avi';'trainsp2_O7.avi'; ...

'trainsp3_O1.avi';'trainsp3_O2.avi';'trainsp3_O3.avi';'trainsp3_O4.avi';'trainsp3_O5.avi';'trainsp3_O6.avi';'trainsp3_O7.avi'; ...

'trainsp4_O1.avi';'trainsp4_O2.avi';'trainsp4_O3.avi';'trainsp4_O4.avi';'trainsp4_O5.avi';'trainsp4_O6.avi';'trainsp4_O7.avi'; ...

'trainsp5_O1.avi';'trainsp5_O2.avi';'trainsp5_O3.avi';'trainsp5_O4.avi';'trainsp5_O5.avi';'trainsp5_O6.avi';'trainsp5_O7.avi'; ...

'trainsp1_P1.avi';'trainsp1_P2.avi';'trainsp1_P3.avi';'trainsp1_P4.avi';'trainsp1_P5.avi';'trainsp1_P6.avi';'trainsp1_P7.avi'; ...

'trainsp2_P1.avi';'trainsp2_P2.avi';'trainsp2_P3.avi';'trainsp2_P4.avi';'trainsp2_P5.avi';'trainsp2_P6.avi';'trainsp2_P7.avi'; ...

'trainsp3_P1.avi';'trainsp3_P2.avi';'trainsp3_P3.avi';'trainsp3_P4.avi';'trainsp3_P5.avi';'trainsp3_P6.avi';'trainsp3_P7.avi'; ...

'trainsp4_P1.avi';'trainsp4_P2.avi';'trainsp4_P3.avi';'trainsp4_P4.avi';'trainsp4_P5.avi';'trainsp4_P6.avi';'trainsp4_P7.avi'; ...

'trainsp5_P1.avi';'trainsp5_P2.avi';'trainsp5_P3.avi';'trainsp5_P4.avi';'trainsp5_P5.avi';'trainsp5_P6.avi';'trainsp5_P7.avi'; ...

'trainsp1_Q1.avi';'trainsp1_Q2.avi';'trainsp1_Q3.avi';'trainsp1_Q4.avi';'trainsp1_Q5.avi';'trainsp1_Q6.avi';'trainsp1_Q7.avi'; ...

'trainsp2_Q1.avi';'trainsp2_Q2.avi';'trainsp2_Q3.avi';'trainsp2_Q4.avi';'trainsp2_Q5.avi';'trainsp2_Q6.avi';'trainsp2_Q7.avi'; ...

'trainsp3_Q1.avi';'trainsp3_Q2.avi';'trainsp3_Q3.avi';'trainsp3_Q4.avi';'trainsp3_Q5.avi';'trainsp3_Q6.avi';'trainsp3_Q7.avi'; ...

'trainsp4_Q1.avi';'trainsp4_Q2.avi';'trainsp4_Q3.avi';'trainsp4_Q4.avi';'trainsp4_Q5.avi';'trainsp4_Q6.avi';'trainsp4_Q7.avi'; ...

'trainsp5_Q1.avi';'trainsp5_Q2.avi';'trainsp5_Q3.avi';'trainsp5_Q4.avi';'trainsp5_Q5.avi';'trainsp5_Q6.avi';'trainsp5_Q7.avi'; ...

'trainsp1_R1.avi';'trainsp1_R2.avi';'trainsp1_R3.avi';'trainsp1_R4.avi';'trainsp1_R5.avi';'trainsp1_R6.avi';'trainsp1_R7.avi'; ...

'trainsp2_R1.avi';'trainsp2_R2.avi';'trainsp2_R3.avi';'trainsp2_R4.avi';'trainsp2_R5.avi';'trainsp2_R6.avi';'trainsp2_R7.avi'; ...

'trainsp3_R1.avi';'trainsp3_R2.avi';'trainsp3_R3.avi';'trainsp3_R4.avi';'trainsp3_R5.avi';'trainsp3_R6.avi';'trainsp3_R7.avi'; ...

'trainsp4_R1.avi';'trainsp4_R2.avi';'trainsp4_R3.avi';'trainsp4_R4.avi';'trainsp4_R5.avi';'trainsp4_R6.avi';'trainsp4_R7.avi'; ...

'trainsp5_R1.avi';'trainsp5_R2.avi';'trainsp5_R3.avi';'trainsp5_R4.avi';'trainsp5_R5.avi';'trainsp5_R6.avi';'trainsp5_R7.avi'; ...

'trainsp1_S1.avi';'trainsp1_S2.avi';'trainsp1_S3.avi';'trainsp1_S4.avi';'trainsp1_S5.avi';'trainsp1_S6.avi';'trainsp1_S7.avi'; ...

'trainsp2_S1.avi';'trainsp2_S2.avi';'trainsp2_S3.avi';'trainsp2_S4.avi';'trainsp2_S5.avi';'trainsp2_S6.avi';'trainsp2_S7.avi'; ...

'trainsp3_S1.avi';'trainsp3_S2.avi';'trainsp3_S3.avi';'trainsp3_S4.avi';'trainsp3_S5.avi';'trainsp3_S6.avi';'trainsp3_S7.avi'; ...

'trainsp4_S1.avi';'trainsp4_S2.avi';'trainsp4_S3.avi';'trainsp4_S4.avi';'trainsp4_S5.avi';'trainsp4_S6.avi';'trainsp4_S7.avi'; ...

'trainsp5_S1.avi';'trainsp5_S2.avi';'trainsp5_S3.avi';'trainsp5_S4.avi';'trainsp5_S5.avi';'trainsp5_S6.avi';'trainsp5_S7.avi'; ...

'trainsp1_T1.avi';'trainsp1_T2.avi';'trainsp1_T3.avi';'trainsp1_T4.avi';'trainsp1_T5.avi';'trainsp1_T6.avi';'trainsp1_T7.avi'; ...

'trainsp2_T1.avi';'trainsp2_T2.avi';'trainsp2_T3.avi';'trainsp2_T4.avi';'trainsp2_T5.avi';'trainsp2_T6.avi';'trainsp2_T7.avi'; ...

'trainsp3_T1.avi';'trainsp3_T2.avi';'trainsp3_T3.avi';'trainsp3_T4.avi';'trainsp3_T5.avi';'trainsp3_T6.avi';'trainsp3_T7.avi'; ...

'trainsp4_T1.avi';'trainsp4_T2.avi';'trainsp4_T3.avi';'trainsp4_T4.avi';'trainsp4_T5.avi';'trainsp4_T6.avi';'trainsp4_T7.avi'; ...

'trainsp5_T1.avi';'trainsp5_T2.avi';'trainsp5_T3.avi';'trainsp5_T4.avi';'trainsp5_T5.avi';'trainsp5_T6.avi';'trainsp5_T7.avi'; ...

'trainsp1_U1.avi';'trainsp1_U2.avi';'trainsp1_U3.avi';'trainsp1_U4.avi';'trainsp1_U5.avi';'trainsp1_U6.avi';'trainsp1_U7.avi'; ...

'trainsp2_U1.avi';'trainsp2_U2.avi';'trainsp2_U3.avi';'trainsp2_U4.avi';'trainsp2_U5.avi';'trainsp2_U6.avi';'trainsp2_U7.avi'; ...

'trainsp3_U1.avi';'trainsp3_U2.avi';'trainsp3_U3.avi';'trainsp3_U4.avi';'trainsp3_U5.avi';'trainsp3_U6.avi';'trainsp3_U7.avi'; ...

'trainsp4_U1.avi';'trainsp4_U2.avi';'trainsp4_U3.avi';'trainsp4_U4.avi';'trainsp4_U5.avi';'trainsp4_U6.avi';'trainsp4_U7.avi'; ...

'trainsp5_U1.avi';'trainsp5_U2.avi';'trainsp5_U3.avi';'trainsp5_U4.avi';'trainsp5_U5.avi';'trainsp5_U6.avi';'trainsp5_U7.avi'; ...

'trainsp1_V1.avi';'trainsp1_V2.avi';'trainsp1_V3.avi';'trainsp1_V4.avi';'trainsp1_V5.avi';'trainsp1_V6.avi';'trainsp1_V7.avi'; ...

'trainsp2_V1.avi';'trainsp2_V2.avi';'trainsp2_V3.avi';'trainsp2_V4.avi';'trainsp2_V5.avi';'trainsp2_V6.avi';'trainsp2_V7.avi'; ...

'trainsp3_V1.avi';'trainsp3_V2.avi';'trainsp3_V3.avi';'trainsp3_V4.avi';'trainsp3_V5.avi';'trainsp3_V6.avi';'trainsp3_V7.avi'; ...

'trainsp4_V1.avi';'trainsp4_V2.avi';'trainsp4_V3.avi';'trainsp4_V4.avi';'trainsp4_V5.avi';'trainsp4_V6.avi';'trainsp4_V7.avi'; ...

'trainsp5_V1.avi';'trainsp5_V2.avi';'trainsp5_V3.avi';'trainsp5_V4.avi';'trainsp5_V5.avi';'trainsp5_V6.avi';'trainsp5_V7.avi'; ...

'trainsp1_W1.avi';'trainsp1_W2.avi';'trainsp1_W3.avi';'trainsp1_W4.avi';'trainsp1_W5.avi';'trainsp1_W6.avi';'trainsp1_W7.avi'; ...

'trainsp2_W1.avi';'trainsp2_W2.avi';'trainsp2_W3.avi';'trainsp2_W4.avi';'trainsp2_W5.avi';'trainsp2_W6.avi';'trainsp2_W7.avi'; ...

'trainsp3_W1.avi';'trainsp3_W2.avi';'trainsp3_W3.avi';'trainsp3_W4.avi';'trainsp3_W5.avi';'trainsp3_W6.avi';'trainsp3_W7.avi'; ...

'trainsp4_W1.avi';'trainsp4_W2.avi';'trainsp4_W3.avi';'trainsp4_W4.avi';'trainsp4_W5.avi';'trainsp4_W6.avi';'trainsp4_W7.avi'; ...

'trainsp5_W1.avi';'trainsp5_W2.avi';'trainsp5_W3.avi';'trainsp5_W4.avi';'trainsp5_W5.avi';'trainsp5_W6.avi';'trainsp5_W7.avi'; ...

'trainsp1_X1.avi';'trainsp1_X2.avi';'trainsp1_X3.avi';'trainsp1_X4.avi';'trainsp1_X5.avi';'trainsp1_X6.avi';'trainsp1_X7.avi'; ...

'trainsp2_X1.avi';'trainsp2_X2.avi';'trainsp2_X3.avi';'trainsp2_X4.avi';'trainsp2_X5.avi';'trainsp2_X6.avi';'trainsp2_X7.avi'; ...

'trainsp3_X1.avi';'trainsp3_X2.avi';'trainsp3_X3.avi';'trainsp3_X4.avi';'trainsp3_X5.avi';'trainsp3_X6.avi';'trainsp3_X7.avi'; ...

'trainsp4_X1.avi';'trainsp4_X2.avi';'trainsp4_X3.avi';'trainsp4_X4.avi';'trainsp4_X5.avi';'trainsp4_X6.avi';'trainsp4_X7.avi'; ...

'trainsp5_X1.avi';'trainsp5_X2.avi';'trainsp5_X3.avi';'trainsp5_X4.avi';'trainsp5_X5.avi';'trainsp5_X6.avi';'trainsp5_X7.avi'; ...

'trainsp1_Y1.avi';'trainsp1_Y2.avi';'trainsp1_Y3.avi';'trainsp1_Y4.avi';'trainsp1_Y5.avi';'trainsp1_Y6.avi';'trainsp1_Y7.avi'; ...

'trainsp2_Y1.avi';'trainsp2_Y2.avi';'trainsp2_Y3.avi';'trainsp2_Y4.avi';'trainsp2_Y5.avi';'trainsp2_Y6.avi';'trainsp2_Y7.avi'; ...

'trainsp3_Y1.avi';'trainsp3_Y2.avi';'trainsp3_Y3.avi';'trainsp3_Y4.avi';'trainsp3_Y5.avi';'trainsp3_Y6.avi';'trainsp3_Y7.avi'; ...

'trainsp4_Y1.avi';'trainsp4_Y2.avi';'trainsp4_Y3.avi';'trainsp4_Y4.avi';'trainsp4_Y5.avi';'trainsp4_Y6.avi';'trainsp4_Y7.avi'; ...

'trainsp5_Y1.avi';'trainsp5_Y2.avi';'trainsp5_Y3.avi';'trainsp5_Y4.avi';'trainsp5_Y5.avi';'trainsp5_Y6.avi';'trainsp5_Y7.avi'; ...

'trainsp1_Z1.avi';'trainsp1_Z2.avi';'trainsp1_Z3.avi';'trainsp1_Z4.avi';'trainsp1_Z5.avi';'trainsp1_Z6.avi';'trainsp1_Z7.avi'; ...

'trainsp2_Z1.avi';'trainsp2_Z2.avi';'trainsp2_Z3.avi';'trainsp2_Z4.avi';'trainsp2_Z5.avi';'trainsp2_Z6.avi';'trainsp2_Z7.avi'; ...

'trainsp3_Z1.avi';'trainsp3_Z2.avi';'trainsp3_Z3.avi';'trainsp3_Z4.avi';'trainsp3_Z5.avi';'trainsp3_Z6.avi';'trainsp3_Z7.avi'; ...

'trainsp4_Z1.avi';'trainsp4_Z2.avi';'trainsp4_Z3.avi';'trainsp4_Z4.avi';'trainsp4_Z5.avi';'trainsp4_Z6.avi';'trainsp4_Z7.avi'; ...

'trainsp5_Z1.avi';'trainsp5_Z2.avi';'trainsp5_Z3.avi';'trainsp5_Z4.avi';'trainsp5_Z5.avi';'trainsp5_Z6.avi';'trainsp5_Z7.avi'; ...

};

numFiles = numel(files);

for mm=1:numFiles

video = readVideo(files{mm});

all_vid{mm}=video;

fprintf("Reading file %d of %d...\n", mm, numFiles)

[s1 s2 s3 s4]=size(video);

chk_fn=files{mm};

chk_letter=chk_fn(10);

chk_letter_cnt_for_train_AVLETTERS(chk_letter,video) ;

for ii=1:s4

comb_video=video(:,:,:,ii);

all_frames(:,:,:,index2)=comb_video;

index2=index2+1;

end

end

%-

aa=[ones(1,acntr) 2*ones(1,bcntr) 3*ones(1,ccntr) 4*ones(1,dcntr) 5*ones(1,ecntr) 6*ones(1,fcntr) 7*ones(1,gcntr) 8*ones(1,hcntr) 9*ones(1,icntr) 10*ones(1,jcntr) ...

11*ones(1,kcntr) 12*ones(1,lcntr) 13*ones(1,mcntr) 14*ones(1,ncntr) 15*ones(1,ocntr) 16*ones(1,pcntr) 17*ones(1,qcntr) 18*ones(1,rcntr) 19*ones(1,scntr) 20*ones(1,tcntr) ...

21*ones(1,ucntr) 22*ones(1,vcntr) 23*ones(1,wcntr) 24*ones(1,xcntr) 25*ones(1,ycntr) 26*ones(1,zcntr)...

]';

sslbl= categorical(aa);

imageAugmenter = imageDataAugmenter;

imageSize = [38 62 3];

augimds = augmentedImageDatastore(imageSize,all_frames,sslbl,'DataAugmentation',imageAugmenter);

layers = [

imageInputLayer(imageSize)

convolution2dLayer(3,4,'Padding','same')

batchNormalizationLayer

reluLayer

maxPooling2dLayer(2,'Stride',2)

convolution2dLayer(3,16,'Padding','same')

batchNormalizationLayer

reluLayer

maxPooling2dLayer(2,'Stride',2)

convolution2dLayer(3,32,'Padding','same')

batchNormalizationLayer

reluLayer

fullyConnectedLayer(26)

softmaxLayer

classificationLayer];

%%%%

xval1=[5,6, 7, 36,37, 38,71,72,73,106, 107,108,141, 142, 143, 176, 177, 178, 211, 212, 213, 246, 247, 248, 281, 282, 283, 316, 317, 318, 351, 352, 353, ...

386,387,388,421,422,423,456,457,458,491,492,493,526,527,528,561,562,563,596,597,598,631,632,633,666,667,668,701,702,703,...

736,737,738,771,772,773,805,806,807,841,842,843];

index3=1;

for mm1=1:length(xval1)

ind_tst=xval1(mm1);

video1 = readVideo(files{ind_tst});

[s1 s2 s3 s4]=size(video1);

chk_fn=files{ind_tst};

chk_letter=chk_fn(10);

chk_letter_cnt_for_validation_AVLETTERS(chk_letter,video1);

for ii=1:s4

comb_video1=video1(:,:,:,ii);

XValidation(:,:,:,index3)=comb_video1;

index3=index3+1;

end

end

% -

bb=[ones(1,acnt) 2*ones(1,bcnt) 3*ones(1,ccnt) 4*ones(1,dcnt) 5*ones(1,ecnt) 6*ones(1,fcnt) 7*ones(1,gcnt) 8*ones(1,hcnt) 9*ones(1,icnt) 10*ones(1,jcnt) ...

11*ones(1,kcnt) 12*ones(1,lcnt) 13*ones(1,mcnt) 14*ones(1,ncnt) 15*ones(1,ocnt) 16*ones(1,pcnt) 17*ones(1,qcnt) 18*ones(1,rcnt) 19*ones(1,scnt) 20*ones(1,tcnt) ...

21*ones(1,ucnt) 22*ones(1,vcnt) 23*ones(1,wcnt) 24*ones(1,xcnt) 25*ones(1,ycnt) 26*ones(1,zcnt)]';

bb1= categorical(bb);

YValidation=bb1;

miniBatchSize = 400;

opts = trainingOptions('sgdm', ...

'MiniBatchSize', miniBatchSize,...

'MaxEpochs',2, ...

'Shuffle','every-epoch', ...

'Plots','training-progress', ...

'Verbose',false,...

'ValidationData',{XValidation,YValidation});

net = trainNetwork(augimds,layers,opts);

------------------------------------------

Error

Error using trainNetwork (line 184)

Invalid validation data. The output size (26) of the last layer does not match the number

of classes (25).

______________________________

Please help to resolve the error

Answer 1

[Siraj]

Hii! It is my understanding that you want to read different videos corresponding to different English alphabets and then concatenate each frame of these videos to train a classification network.

You have attached 2 codes; I believe the second code is an improvement of the first and solves the error given by the first code but generates a new error.

In the code attached in the first comment, a video is being read and all the frames of that video are extracted and vertically concatenated in the “all_frames”. This makes “all_frames” a 3d array and this variable is then passed to “augmentedImageDatastore()” which expects a 4d array.

Refer to the input arguments of “augmentedImageDatastore()” below.

https://in.mathworks.com/help/deeplearning/ref/augmentedimagedatastore.html#d124e15197

Another issue with the code attached in the first comment is that “sslbl” has only one category (1) but it should have exactly 2 categories (1,2) since you have mentioned “Total 4 videos. First 2 videos for'A' letter & remaining 2 are for 'B' letter.”

However, both above issues are fixed in the code attached in the second comment.

The new error that appears is

Error using trainNetwork (line 184)

Invalid validation data. The output size (26) of the last layer does not match the number

of classes (25).

Even though I am not able to reproduce this error at my end since I don’t have access to the data and helper functions “chk_fn”, “chk_letter_cnt_for_validation_AVLETTERS”, from the error message it is quite evident that “bb1” contains only 25 different categories instead of 26 different categories. This can be because of either “acnt” or “bct” …….or “zcnt” being 0.

Since you have not mentioned the “classes” while defining the classification layer so this parameter is set to “auto” and the number of classes will be defined at the time of training.

Refer to the document below to define the number of classes for a “classificationLayer”

https://in.mathworks.com/help/deeplearning/ref/classificationlayer.html

To resolve the error, make sure that “bb1” has exactly 26 different categories.

Hope this helps.