Predictors must be a N-by-1 cell array of sequences, where N is the number of sequences. All sequences must have the have the same feature dimension and at least one time step

Hello,

i get this error message and i tried to fix it but i didn't succeed.

"Error using trainNetwork (line 170)

Invalid training data. Predictors must be a N-by-1 cell array of sequences, where N is the number of sequences. All sequences must

have the same feature dimension and at least one time step.

Error in Untitled5 (line 64)

[net, traininfo] = trainNetwork(xTrain, yTrain, layers, options);"

Thank you if you can give me helpful tips.

Data = readtable("C:\Users\robouser\Desktop\m1\FullTrainingDataTable_roi50_dist2_16xPatches_poly11_noneExt.csv");
data_droped = unique(Data, "rows");
data_droped_normalized = normalize(data_droped,'range', [-1 1]);
[r,c] = size(data_droped_normalized) ;
P1 = 0.70 ; P2 = 0.85 ;
idx = randperm(r) ;
m = round(P1*r) ; n = round(P2*r) ;
Training = data_droped_normalized(idx(1:m),:) ;
Validation = data_droped_normalized(idx(m+1:n),:) ;
Testing = data_droped_normalized(idx(n+1:end),:) ;

x_train = Training(:, 1:end-1);
x_val = Validation(:, 1:end-1);
x_test = Testing(:, 1:end-1);

y_train = Training(:, end);
y_val = Validation(:, end);
y_test = Testing(:, end);
xTrain = table2cell(x_train);
xVal = table2cell(x_val);
xTest = table2cell(x_test);

yTrain = table2cell(y_train);
yVal = table2cell(y_val);
yTest = table2cell(y_test);
%%
numFeatures = size(xTrain(1, :), 2);
layers = [
    sequenceInputLayer(numFeatures,"Name","Input_Layer")
    fullyConnectedLayer(512,"Name","Layer-1")
    reluLayer("Name","relu_1")
    fullyConnectedLayer(256,"Name","Layer-2")
    reluLayer("Name","relu_2")
    fullyConnectedLayer(32,"Name","Layer-3")
    reluLayer("Name","relu_3")
    fullyConnectedLayer(16,"Name","Layer-4")
    reluLayer("Name","relu_4")
    fullyConnectedLayer(1,"Name","OutPut")
    regressionLayer("Name","regressionoutput")];
%%
options = trainingOptions("adam",...
    "ExecutionEnvironment","auto",...
    "InitialLearnRate",0.001,...
    "MaxEpochs",250,...
    'MiniBatchSize',128, ...
    "ValidationFrequency", 20,...
    "ValidationPatience",7,...
    "ValidationData",{x_val, y_val},...
    "Verbose",false,...
    "LearnRateSchedule","piecewise",...
    "LearnRateDropFactor", 0.5,...
    "Plots","training-progress");
[net, traininfo] = trainNetwork(xTrain, yTrain, layers, options);

[yanqi liu]

yes，sir，may be upload your csv file to do debug

[Sam Kalho]

hi,

hier is my first 1000 data.

thanks.

Answer 1

[yanqi liu]

yes，sir，may be the data should make process，such as

clc; clear all; close all;
% Data = readtable("C:\Users\robouser\Desktop\m1\FullTrainingDataTable_roi50_dist2_16xPatches_poly11_noneExt.csv");
load Data_1000
Data = Data_1000;
data_droped = unique(Data, "rows");
data_droped_normalized = normalize(data_droped,'range', [-1 1]);
[r,c] = size(data_droped_normalized) ;
P1 = 0.70 ; P2 = 0.85 ;
idx = randperm(r) ;
m = round(P1*r) ; n = round(P2*r) ;
Training = data_droped_normalized(idx(1:m),:) ;
Validation = data_droped_normalized(idx(m+1:n),:) ;
Testing = data_droped_normalized(idx(n+1:end),:) ;

x_train = Training(:, 1:end-1);
x_val = Validation(:, 1:end-1);
x_test = Testing(:, 1:end-1);

y_train = Training(:, end);
y_val = Validation(:, end);
y_test = Testing(:, end);
xTrain = table2cell(x_train);
xVal = table2cell(x_val);
xTest = table2cell(x_test);

yTrain = table2cell(y_train);
yVal = table2cell(y_val);
yTest = table2cell(y_test);
%%
numFeatures = size(xTrain(1, :), 2);
layers = [
    sequenceInputLayer(numFeatures,"Name","Input_Layer")
    fullyConnectedLayer(512,"Name","Layer-1")
    reluLayer("Name","relu_1")
    fullyConnectedLayer(256,"Name","Layer-2")
    reluLayer("Name","relu_2")
    fullyConnectedLayer(32,"Name","Layer-3")
    reluLayer("Name","relu_3")
    fullyConnectedLayer(16,"Name","Layer-4")
    reluLayer("Name","relu_4")
    fullyConnectedLayer(1,"Name","OutPut")
    regressionLayer("Name","regressionoutput")];
%%
xTrain2 = [];
for i = 1 : size(xTrain, 1)
    xi = cell2mat(xTrain(i,:));
    xTrain2{i,1} = xi(:);
end
yTrain2 = [];
for i = 1 : size(yTrain, 1)
    yi = cell2mat(yTrain(i,:));
    yTrain2{i,1} = yi(:);
end
x_val2 = [];
for i = 1 : size(x_val, 1)
    xi = x_val{i,:};
    x_val2{i,1} = xi(:);
end
y_val2 = [];
for i = 1 : size(y_val, 1)
    yi = y_val{i,:};
    y_val2{i,1} = yi(:);
end
options = trainingOptions("adam",...
    "ExecutionEnvironment","auto",...
    "InitialLearnRate",0.001,...
    "MaxEpochs",250,...
    'MiniBatchSize',128, ...
    "ValidationFrequency", 20,...
    "ValidationPatience",7,...
    "ValidationData",{x_val2, y_val2},...
    "Verbose",false,...
    "LearnRateSchedule","piecewise",...
    "LearnRateDropFactor", 0.5,...
    "Plots","training-progress");
[net, traininfo] = trainNetwork(xTrain2, yTrain2, layers, options);

[Sam Kalho]

thank you very much.

thank you for your Help and time.

Answer 2

[Katja Mogalle]

Hi Seyed,

I see you've chosen to use a sequenceInputLayer in your network. Sequences are data with a "time" dimension as well as a feature dimension. So a sequence could for example be some sort of weather data over time (temperature, humidity, etc.), or an EEG signal in the medical domain.

From looking at your network (only uses fullyConnectedLayers) I suspect you don't actually have sequence data, i.e. you don't have a time dimension. You just have 526734 observations and each observation has 56 features plus a scalar response. But no time/sequence dimension.

If my assumption is correct, then what you'd actually need is the featureInputLayer which is available from R2022b onwards.

So your network would look as follows:

layers = [
    featureInputLayer(numFeatures,"Name","Input_Layer")
    fullyConnectedLayer(512,"Name","Layer-1")
    reluLayer("Name","relu_1")
    fullyConnectedLayer(256,"Name","Layer-2")
    reluLayer("Name","relu_2")
    fullyConnectedLayer(32,"Name","Layer-3")
    reluLayer("Name","relu_3")
    fullyConnectedLayer(16,"Name","Layer-4")
    reluLayer("Name","relu_4")
    fullyConnectedLayer(1,"Name","OutPut")
    regressionLayer("Name","regressionoutput")];

If you go down this route, you also won't need to convert your data to a cell array. Check out this documentation example to see how the data and network needs to look like: https://www.mathworks.com/help/deeplearning/ug/train-network-on-data-set-of-numeric-features.html

(In the case you are working with an earlier MATLAB version, there is a workaround to use imageInputLayer and reshape your input data into a 4-dimensional array such that the first two dimensions are scalar, the third dimension is for your features and the fourth dimension for the observations. Or in short, convert your data from shape numObservations-by-numFeatures to 1-by-1-by-numFeatures-by-numObservations. )

I hope this solves your problem.

Good luck!

Katja

[Sam Kalho]

Hi Katja,

Thank you for your answer.

Yes, actually I don't need sequence layer because my data is not sequential,

Feutures are 56, output only 1.

I installed the new version 2021b and tried to run my script with feautureInputLayer.

but I still get an error message.

" Error using trainNetwork (line 184)

Too many input arguments.

Error in script_1 (line 76)

[net, traininfo] = trainNetwork(x_train, y_train, layers, options);

Caused by:

Error using nnet.internal.cnn.trainNetwork.DLTInputParser>iParseInputArguments (line 83)

Too many input arguments. "

clc

close all

clear all

Data = readtable("C:\Users\saman\Downloads\Datenbank\Datenbank\FullTrainingDataTable_roi50_dist2_16xPatches_poly11_noneExt.csv");

data_droped = unique(Data, "rows");

data_droped_normalized = normalize(data_droped,'range', [-1 1]);

[r,c] = size(data_droped_normalized) ;

P1 = 0.70 ; P2 = 0.85 ;

idx = randperm(r) ;

m = round(P1*r) ; n = round(P2*r) ;

Training = data_droped_normalized(idx(1:m),:) ;

Validation = data_droped_normalized(idx(m+1:n),:) ;

Testing = data_droped_normalized(idx(n+1:end),:) ;

x_train = Training(:, 1:end-1);

x_val = Validation(:, 1:end-1);

x_test = Testing(:, 1:end-1);

y_train = Training(:, end);

y_val = Validation(:, end);

y_test = Testing(:, end);

%xTrain = table2cell(x_train);

%xVal = table2cell(x_val);

%xTest = table2cell(x_test);

%yTrain = table2cell(y_train);

%yVal = table2cell(y_val);

%yTest = table2cell(y_test);

%%

numFeatures = size(x_train, 2);

%response = size(y_train, 2);

%numFeatures = size(xTrain(:, 1));

%numFeatures = xTrain(:, 1);

layers = [

featureInputLayer(numFeatures,"Name","Input_Layer")

fullyConnectedLayer(512,"Name","Layer-1")

reluLayer("Name","relu_1")

fullyConnectedLayer(256,"Name","Layer-2")

reluLayer("Name","relu_2")

fullyConnectedLayer(32,"Name","Layer-3")

reluLayer("Name","relu_3")

fullyConnectedLayer(16,"Name","Layer-4")

reluLayer("Name","relu_4")

fullyConnectedLayer(1,"Name","OutPut")

regressionLayer("Name","regressionoutput")];

%%

options = trainingOptions("adam",...

"ExecutionEnvironment","auto",...

"InitialLearnRate",0.001,...

"MaxEpochs",250,...

'MiniBatchSize',128, ...

"ValidationFrequency", 20,...

"ValidationPatience",7,...

"ValidationData",{x_val, y_val},...

"Verbose",false,...

"LearnRateSchedule","piecewise",...

"LearnRateDropFactor", 0.5,...

"Plots","training-progress");

%%

[net, traininfo] = trainNetwork(x_train, y_train, layers, options);

Thank you in advance for your help, efforts and understanding.

Best regards

saman

[Katja Mogalle]

Thanks for providing your script and data. I can understand now why you've been struggling and I imagine other people will face exactly the same difficulties.

The error "Too many input arguments." is clearly unhelpful and even incorrect in this case. I've reported this to the MathWorks development department for improvement.

Using featureInputLayer is still the correct approach for your kind of data (features with no temporal dimension) and type of network (fully connected network).

The way to call trainNetwork is to have only ONE table containing both predictors AND responses. The second input argument to trainNetwork is then the name of the column(s) containing the response(s).

See the description in the doc page for the first input argument, features, and the second input argument, responses.

Here is also an example for training a network using numeric features, in case anybody needs it: https://uk.mathworks.com/help/deeplearning/ug/train-network-on-data-set-of-numeric-features.html

I hope this information helps.

[Sam Kalho]

thanks a lot katja.