"Invalid training data. Predictors and responses must have the same number of observations"
71 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
massoud pourmandi
el 30 de Jun. de 2022
Comentada: Image Analyst
el 6 de Oct. de 2023
I'd be appreciated it if someone help me to debug my code. Thanks in advance.
clc;
clear;
close all;
check_node_degs_vec=[1,2,3,4,5];
var_node_degs_vec=[1,2,3,4,5];
x=0.5;
decoding_threshold=0.3;
sample_size=1000;
Xtemp=decoding_threshold*rand([1,sample_size]);
for i=1:sample_size
Xtrain{i}=Xtemp(i);
end
Xtrain=reshape(Xtrain, [1,sample_size]);
Ytrain=zeros(size(Xtrain));
lgraph = layerGraph;
layer_1=[sequenceInputLayer(1,"Name","seq_input_1")
functionLayer(@(x) nodeDegreeLayer(x, check_node_degs_vec),"Name","func_ro")
fullyConnectedLayer(1,"Name","ro")
reluLayer('Name','relu_1');
functionLayer(@(x) nodeDegreeLayer(x, var_node_degs_vec),"Name","func_lam")
fullyConnectedLayer(1,"Name","lam")
reluLayer('Name','relu_2');
additionLayer(2,'Name','add_1');
reluLayer('Name','relu_3');
regressionLayer
];
lgraph = addLayers(lgraph,layer_1);
res_input=functionLayer(@(X) -X,"Name","res_func");
lgraph = addLayers(lgraph,res_input);
lgraph = connectLayers(lgraph,'seq_input_1','res_func');
lgraph = connectLayers(lgraph,'res_func','add_1/in2');
%dlnet = dlnetwork(lgraph);
% figure
% plot(lgraph)
options = trainingOptions("adam", ...
MaxEpochs=60, ...
miniBatchSize=1, ...
Plots="training-progress", ...
Verbose=0);
net = trainNetwork(Xtrain,Ytrain,lgraph,options);
function y = nodeDegreeLayer(x, node_degs_vec)
y = (1-x).^(node_degs_vec-1);
end
Error using trainNetwork
Invalid training data. Predictors and responses must have the same number of observations.
Error in DAGNET (line 49)
net = trainNetwork(Xtrain,Ytrain,lgraph,options);
2 comentarios
Respuesta aceptada
Cris LaPierre
el 1 de Jul. de 2022
Note that, in the image you shared, they also all collapse back to a single node. My suspicion, then, is that you would use the lambdas to weight the result of each value. Again, you would know more about the particulars here than me.
I defined a vector of lambdas and used pagemtimes to compute the output of nodeDegreeLayer
clc;
clear;
close all;
check_node_degs_vec=[1,2,3,4,5];
var_node_degs_vec=[1,2,3,4,5];
lambda = [0.1 0.3 0.2 0.15 0.25]';
x=0.5;
decoding_threshold=0.3;
sample_size=1000;
Xtemp=decoding_threshold*rand([1,sample_size]);
% for i=1:sample_size
% Xtrain{i}=Xtemp(i);
% end
Xtrain=Xtemp;
Xtrain=reshape(Xtrain, [1,sample_size]);
Ytrain=zeros(size(Xtrain));
lgraph = layerGraph;
layer_1=[sequenceInputLayer(1,"Name","seq_input_1")
functionLayer(@(x) nodeDegreeLayer(x, check_node_degs_vec,lambda),"Name","func_ro")
fullyConnectedLayer(1,"Name","ro")
reluLayer('Name','relu_1');
functionLayer(@(x) nodeDegreeLayer(x, var_node_degs_vec,lambda),"Name","func_lam")
fullyConnectedLayer(1,"Name","lam")
reluLayer('Name','relu_2');
additionLayer(2,'Name','add_1');
reluLayer('Name','relu_3');
regressionLayer
];
lgraph = addLayers(lgraph,layer_1);
res_input=functionLayer(@(X) -X,"Name","res_func");
lgraph = addLayers(lgraph,res_input);
lgraph = connectLayers(lgraph,'seq_input_1','res_func');
lgraph = connectLayers(lgraph,'res_func','add_1/in2');
%dlnet = dlnetwork(lgraph);
% figure
% plot(lgraph)
options = trainingOptions("adam", ...
MaxEpochs=60, ...
miniBatchSize=1, ...
Plots="training-progress", ...
Verbose=0);
net = trainNetwork(Xtrain,Ytrain,lgraph,options);
function y = nodeDegreeLayer(x, node_degs_vec,lambda)
y = pagemtimes((1-x).^(node_degs_vec-1),lambda);
end
2 comentarios
l
el 6 de Oct. de 2023
Hello,I have the same problem, but it hasn't been solved yet , so I hope you can give me some advice.My code is as follows:
gwsdata = reshape(gwsdata,205,3627);
tempdata = reshape(tempdata,205,3627);
ndvidata = reshape(ndvidata,205,3627);
precdata = reshape(precdata,205,3627);
petdata = reshape(petdata,205,3627);
lstdata = reshape(lstdata,205,3627);
smdata = reshape(smdata,205,3627);
%划分训练集,验证集
[trainInd, testInd] = dividerand(205, 0.8, 0.2); % 80%的数据作为训练集,20%的数据作为测试集
temp_train = reshape(tempdata(trainInd, :), 39, 93, 1, 142);
ndvi_train = reshape(ndvidata(trainInd, :), 39, 93, 1, 142);
prec_train = reshape(precdata(trainInd, :), 39, 93, 1, 142);
pet_train = reshape(petdata(trainInd, :), 39, 93, 1, 142);
lst_train = reshape(lstdata(trainInd, :), 39, 93, 1, 142);
sm_train = reshape(smdata(trainInd, :), 39, 93, 1, 142);
gws_train = reshape(gwsdata(trainInd, :), 39, 93, 1, 142);
temp_test = reshape(tempdata(testInd, :), 39, 93, 1, 36);
ndvi_test = reshape(ndvidata(testInd, :), 39, 93, 1, 36);
prec_test = reshape(precdata(testInd, :), 39, 93, 1, 36);
pet_test = reshape(petdata(testInd, :), 39, 93, 1, 36);
lst_test = reshape(lstdata(testInd, :), 39, 93, 1, 36);
sm_test = reshape(smdata(testInd, :), 39, 93, 1, 36);
gws_test = reshape(gwsdata(testInd, :), 39, 93, 1, 36);
%数据归一化
temp_train_norm = normalize(temp_train);
temp_test_norm = normalize(temp_test);
%ndvi_train_norm = normalize(ndvi_train);
%ndvi_test_norm = normalize(ndvi_test);
prec_train_norm = normalize(prec_train);
prec_test_norm = normalize(prec_test);
pet_train_norm = normalize(pet_train);
pet_test_norm = normalize(pet_test);
lst_train_norm = normalize(lst_train);
lst_test_norm = normalize(lst_test);
sm_train_norm = normalize(sm_train);
sm_test_norm = normalize(sm_test);
%gws_train_norm = normalize(gws_train);
%gws_test_norm = normalize(gws_test);
% 构造网络结构
layers = [
imageInputLayer([39 93 6],"Name","imageinput")
convolution2dLayer([3 3],50,"Name","conv2d_1","Padding",[1 1 1 1])
reluLayer("Name","Relu_1")
maxPooling2dLayer([2 2],"Name","pool_1","Padding","same","Stride",[2 2])
dropoutLayer(0.2,"Name","dropout_1")
convolution2dLayer([3 3],100,"Name","conv2d_2","Padding",[1 1 1 1])
reluLayer("Name","Relu_2")
maxPooling2dLayer([2 2],"Name","pool_2","Padding","same","Stride",[2 2])
dropoutLayer(0.2,"Name","dropout_2")
convolution2dLayer([3 3],200,"Name","conv2d_3","Padding",[1 1 1 1])
reluLayer("Name","Relu_3")
maxPooling2dLayer([2 2],"Name","pool_3","Padding","same","Stride",[2 2])
dropoutLayer(0.3,"Name","dropout_3")
convolution2dLayer([3 3],300,"Name","conv2d_4","Padding",[1 1 1 1])
reluLayer("Name","Relu_4")
fullyConnectedLayer(200,"Name","fc_1")
reluLayer("Name","relu_5")
dropoutLayer(0.4,"Name","dropout_4")
fullyConnectedLayer(1164*2783,"Name","fc_2")
regressionLayer("Name","Output")];
lgraph = layerGraph(layers);
options = trainingOptions('adam', ...
'MaxEpochs', 32, ...
'MiniBatchSize', 15, ...
'Shuffle', 'every-epoch', ...
'InitialLearnRate', 1e-2, ...
'LearnRateDropFactor', 0.01, ...
'LearnRateDropPeriod', 10, ...
'LearnRateSchedule','piecewise',...
'Plots', 'training-progress',...
'ExecutionEnvironment', 'cpu', ...
"L2Regularization", 0.001,...
'Verbose', true);
net = trainNetwork({temp_train, ndvi_train, prec_train,pet_train,sm_train,lst_train}, gws_train, lgraph, options);
Error for trainnetwork:Invalid training data. The predictor and response must have the same number of observations.
But when I checked the data, I found that the data seemed to be correct. Could you tell me why? Thanks a lot!
Cris LaPierre
el 6 de Oct. de 2023
Without your data, it is pretty hard to provide a specific answer. However, if you have the same error, then your predictor and response inputs do not have the same number of observations.
Más respuestas (3)
Image Analyst
el 1 de Jul. de 2022
Instead of
Xtemp=decoding_threshold*rand([1,sample_size]);
for i=1:sample_size
Xtrain{i}=Xtemp(i);
end
Xtrain=reshape(Xtrain, [1,sample_size]);
Ytrain=zeros(size(Xtrain));
maybe
Xtemp=decoding_threshold*rand([sample_size, 1]); % Column vector. But why random numbers?!?
Xtrain=Xtemp;
Ytrain=zeros(size(Xtrain)); % Huh?!?
but it looks like you're training on random inputs, and a desired ground truth of all zeros. What's up with that? That makes no sense, at least not to me.
3 comentarios
l
el 6 de Oct. de 2023
Hello,I have the same problem, but it hasn't been solved yet , so I hope you can give me some advice.My code is as follows:
gwsdata = reshape(gwsdata,205,3627);
tempdata = reshape(tempdata,205,3627);
ndvidata = reshape(ndvidata,205,3627);
precdata = reshape(precdata,205,3627);
petdata = reshape(petdata,205,3627);
lstdata = reshape(lstdata,205,3627);
smdata = reshape(smdata,205,3627);
%划分训练集,验证集
[trainInd, testInd] = dividerand(205, 0.8, 0.2); % 80%的数据作为训练集,20%的数据作为测试集
temp_train = reshape(tempdata(trainInd, :), 39, 93, 1, 142);
ndvi_train = reshape(ndvidata(trainInd, :), 39, 93, 1, 142);
prec_train = reshape(precdata(trainInd, :), 39, 93, 1, 142);
pet_train = reshape(petdata(trainInd, :), 39, 93, 1, 142);
lst_train = reshape(lstdata(trainInd, :), 39, 93, 1, 142);
sm_train = reshape(smdata(trainInd, :), 39, 93, 1, 142);
gws_train = reshape(gwsdata(trainInd, :), 39, 93, 1, 142);
temp_test = reshape(tempdata(testInd, :), 39, 93, 1, 36);
ndvi_test = reshape(ndvidata(testInd, :), 39, 93, 1, 36);
prec_test = reshape(precdata(testInd, :), 39, 93, 1, 36);
pet_test = reshape(petdata(testInd, :), 39, 93, 1, 36);
lst_test = reshape(lstdata(testInd, :), 39, 93, 1, 36);
sm_test = reshape(smdata(testInd, :), 39, 93, 1, 36);
gws_test = reshape(gwsdata(testInd, :), 39, 93, 1, 36);
%数据归一化
temp_train_norm = normalize(temp_train);
temp_test_norm = normalize(temp_test);
%ndvi_train_norm = normalize(ndvi_train);
%ndvi_test_norm = normalize(ndvi_test);
prec_train_norm = normalize(prec_train);
prec_test_norm = normalize(prec_test);
pet_train_norm = normalize(pet_train);
pet_test_norm = normalize(pet_test);
lst_train_norm = normalize(lst_train);
lst_test_norm = normalize(lst_test);
sm_train_norm = normalize(sm_train);
sm_test_norm = normalize(sm_test);
%gws_train_norm = normalize(gws_train);
%gws_test_norm = normalize(gws_test);
% 构造网络结构
layers = [
imageInputLayer([39 93 6],"Name","imageinput")
convolution2dLayer([3 3],50,"Name","conv2d_1","Padding",[1 1 1 1])
reluLayer("Name","Relu_1")
maxPooling2dLayer([2 2],"Name","pool_1","Padding","same","Stride",[2 2])
dropoutLayer(0.2,"Name","dropout_1")
convolution2dLayer([3 3],100,"Name","conv2d_2","Padding",[1 1 1 1])
reluLayer("Name","Relu_2")
maxPooling2dLayer([2 2],"Name","pool_2","Padding","same","Stride",[2 2])
dropoutLayer(0.2,"Name","dropout_2")
convolution2dLayer([3 3],200,"Name","conv2d_3","Padding",[1 1 1 1])
reluLayer("Name","Relu_3")
maxPooling2dLayer([2 2],"Name","pool_3","Padding","same","Stride",[2 2])
dropoutLayer(0.3,"Name","dropout_3")
convolution2dLayer([3 3],300,"Name","conv2d_4","Padding",[1 1 1 1])
reluLayer("Name","Relu_4")
fullyConnectedLayer(200,"Name","fc_1")
reluLayer("Name","relu_5")
dropoutLayer(0.4,"Name","dropout_4")
fullyConnectedLayer(1164*2783,"Name","fc_2")
regressionLayer("Name","Output")];
lgraph = layerGraph(layers);
options = trainingOptions('adam', ...
'MaxEpochs', 32, ...
'MiniBatchSize', 15, ...
'Shuffle', 'every-epoch', ...
'InitialLearnRate', 1e-2, ...
'LearnRateDropFactor', 0.01, ...
'LearnRateDropPeriod', 10, ...
'LearnRateSchedule','piecewise',...
'Plots', 'training-progress',...
'ExecutionEnvironment', 'cpu', ...
"L2Regularization", 0.001,...
'Verbose', true);
net = trainNetwork({temp_train, ndvi_train, prec_train,pet_train,sm_train,lst_train}, gws_train, lgraph, options);
Error for trainnetwork:Invalid training data. The predictor and response must have the same number of observations.
But when I checked the data, I found that the data seemed to be correct. Could you tell me why? Thanks a lot!
Image Analyst
el 6 de Oct. de 2023
Start your own question and put all these variables (temp_train, ndvi_train, prec_train,pet_train,sm_train,lst_train}, gws_train, lgraph, options) into a .mat file and attach it with the paperclip icon. Also give your code there in your new question.
Cris LaPierre
el 30 de Jun. de 2022
I believe the problem is due to the fact that Xtrain is a cell array. I would replace your for loop with the following code instead:
Xtrain=Xtemp;
However, that will lead to a second error:
Error using trainNetwork
Targets and network outputs must be the same size. The networks outputs have size 1×5×1000 and the targets have size 1×1×1000.
This is because the network output (result of nodeDegree) is a 1000 x 5. You need to apply your knowledge is this space to have the network output and target variable be the same size. Here is a configuration that got the training to run for me.
clc;
clear;
close all;
check_node_degs_vec=[1,2,3,4,5];
var_node_degs_vec=[1,2,3,4,5];
x=0.5;
decoding_threshold=0.3;
sample_size=1000;
Xtemp=decoding_threshold*rand([1,sample_size]);
% for i=1:sample_size
% Xtrain{i}=Xtemp(i);
% end
Xtrain=Xtemp;
Xtrain=reshape(Xtrain, [1,sample_size]);
Ytrain=zeros(size(Xtrain));
lgraph = layerGraph;
layer_1=[sequenceInputLayer(1,"Name","seq_input_1")
functionLayer(@(x) nodeDegreeLayer(x, check_node_degs_vec),"Name","func_ro")
fullyConnectedLayer(1,"Name","ro")
reluLayer('Name','relu_1');
functionLayer(@(x) nodeDegreeLayer(x, var_node_degs_vec),"Name","func_lam")
fullyConnectedLayer(1,"Name","lam")
reluLayer('Name','relu_2');
additionLayer(2,'Name','add_1');
reluLayer('Name','relu_3');
regressionLayer
];
lgraph = addLayers(lgraph,layer_1);
res_input=functionLayer(@(X) -X,"Name","res_func");
lgraph = addLayers(lgraph,res_input);
lgraph = connectLayers(lgraph,'seq_input_1','res_func');
lgraph = connectLayers(lgraph,'res_func','add_1/in2');
%dlnet = dlnetwork(lgraph);
% figure
% plot(lgraph)
options = trainingOptions("adam", ...
MaxEpochs=60, ...
miniBatchSize=1, ...
Plots="training-progress", ...
Verbose=0);
net = trainNetwork(Xtrain,Ytrain,lgraph,options);
function y = nodeDegreeLayer(x, node_degs_vec)
y = sum((1-x).^(node_degs_vec-1),2);
end
0 comentarios
Ver también
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!