Error using trainNetwork. Number of observations in X and Y disagree

Hello!

This is my first time building a neural net and am slowly working through the process.

What I am trying to do:

I have ~40,000 rows of data with 61 columns. Each row corresponds to a spectrum of a body's surface, and each column is an observation of that surface at a different wavelength. I am trying to make a neural net that can take each row in as an input (1 x 60 double) and tell me whether or not it falls into group 1 or group 2. The first column in the 40,000 x 61 array is the label 1 or 2.

I have tried to follow https://www.mathworks.com/help/deeplearning/ug/create-simple-deep-learning-network-for-classification.html but it is not very useful for me since my data is not kept in an image datastore. I am also trying to make a 1-D cnn layer (though I am new to this and am not sure that is the best option for what I am trying to do).

I have tried reshaping my data into a 4-D array such that

>> whos nnTrainLabels

Name Size Bytes Class Attributes

nnTrainLabels 32373x1 32603 categorical

>> whos reshapedData

Name Size Bytes Class Attributes

reshapedData 1x32373x1x60 15539040 double

But I keep getting the error message

Error using trainNetwork (line 170)

Number of observations in X and Y disagree.

Here is the code I am trying to run:

height = 1;
widthTrain = size(nnTrainSet,1);
channels = 1;
sampleSizeTrain = size(nnTrainSet,2);
layers = [
    imageInputLayer([height, widthTrain, channels])
    %     convolution2dLayer([3 1], widthTrain)
    %     batchNormalizationLayer
    %     reluLayer
    fullyConnectedLayer(2)
    softmaxLayer
    classificationLayer];

options = trainingOptions('sgdm', ...
    'MaxEpochs', 200, ...
    'Plots', 'training-progress');
% ________________________________________________________________
% Train the network
net = trainNetwork(reshapedData,nnTrainLabels',layers,options);

I am stuck here, as I have tried to follow what users have said in response to similar questions, but to no avail.

I appreciate any help on this and constructing the neural net in general :)

Thank you!

Answer 1

[Kiran Felix Robert]

Hi Tanner,

The first two input argument of trainNetwork function - X and Y, are reshapedData and nnTrainLables’ respectively, in your program. According to the documentation of the trainNetwork function the dimension of X and Y are required to be,

X – 4-D – h-by-w-by-c-by-N

Y – 1-D – N-by-1

According to your reshaping method, the final dimensions are (as mentioned by you),

X – 4-D – 1-by-32373-by-1-by-60

Y – 1-D – 32373-by-1

Which gives you the error. This error can be resolved if you reshape the X (reshapedData) such that its fourth dimension contains N = 32373.

Hope this Helps.

Kiran Felix Robert

Answer 2

[pathakunta]

The first two input argument of trainNetwork function - X and Y, are reshapedData and nnTrainLables’ respectively, in your program. According to the documentation of the trainNetwork function the dimension of X and Y are required to be, X – 4-D – h-by-w-by-c-by-N Y – 1-D – N-by-1 According to your reshaping method, the final dimensions are (as mentioned by you), X – 4-D – 1-by-32373-by-1-by-60 Y – 1-D – 32373-by-1 Which gives you the error. This error can be resolved if you reshape the X (reshapedData) such that its fourth dimension contains N = 32373.

Answer 3

[pathakunta]

This is my first time building a neural net and am slowly working through the process. What I am trying to do: I have ~40,000 rows of data with 61 columns. Each row corresponds to a spectrum of a body's surface, and each column is an observation of that surface at a different wavelength. I am trying to make a neural net that can take each row in as an input (1 x 60 double) and tell me whether or not it falls into group 1 or group 2. The first column in the 40,000 x 61 array is the label 1 or 2. I have tried to follow https://www.mathworks.com/help/deeplearning/ug/create-simple-deep-learning-network-for-classification.html but it is not very useful for me since my data is not kept in an image datastore. I am also trying to make a 1-D cnn layer (though I am new to this and am not sure that is the best option for what I am trying to do). I have tried reshaping my data into a 4-D array such that >> whos nnTrainLabels Name Size Bytes Class Attributes nnTrainLabels 32373x1 32603 categorical >> whos reshapedData Name Size Bytes Class Attributes reshapedData 1x32373x1x60 15539040 double But I keep getting the error message Error using trainNetwork (line 170) Number of observations in X and Y disagree. Here is the code I am trying to run: height = 1; widthTrain = size(nnTrainSet,1); channels = 1; sampleSizeTrain = size(nnTrainSet,2); layers = [ imageInputLayer([height, widthTrain, channels]) % convolution2dLayer([3 1], widthTrain) % batchNormalizationLayer % reluLayer fullyConnectedLayer(2) softmaxLayer classificationLayer]; options = trainingOptions('sgdm', ... 'MaxEpochs', 200, ... 'Plots', 'training-progress'); % ______________________________________________________________ % Train the network net = trainNetwork(reshapedData,nnTrainLabels',layers,options); I am stuck here, as I have tried to follow what users have said in response to similar questions, but to no avail. I appreciate any help on this and constructing the neural net in general :) Thank you!