How to split matrix data into training, validation and testing data based on a categorical label?
4 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
Kimberly Cardillo
el 20 de Ag. de 2020
Respondida: Srivardhan Gadila
el 23 de Ag. de 2020
I have a 184 x 1 cell array called aefm_4 and each cell is 1 x 1599. Each row (184) has a corresponding categorical label in a separate variable called f and its either labeled 'crack-related' or 'noise'. I want to split up my data so that I have 70% training data, 10% validation data and 20% testing data. I want about even amounts of noise and crack related signals in each training, validation and testing dataset. How would I do that?
Right now, I have been manually separating them as seen below but I will be running this for more data and it is too time consuming to do this.
dataTrain = aefm_4([1:9,14:23,28:47,55:75,87:90,93:101,108:152,169:177],:); %127 hits (70%)
fTrain = f([1:9,14:23,28:47,55:75,87:90,93:101,108:152,169:177],:);
dataVal = aefm_4([12:13,26:27,53:54,83:86,92,105:107,163:168],:); %20 hits (10%)
fVal = f([12:13,26:27,53:54,83:86,92,105:107,163:168],:);
dataTest = aefm_4([10:11,24:25,48:52,76:82,91,102:104,151:162,178:184],:); %39 hits (20%)
fTest = aefm_4([10:11,24:25,48:52,76:82,91,102:104,151:162,178:184],:);
0 comentarios
Respuesta aceptada
Srivardhan Gadila
el 23 de Ag. de 2020
You can do something like below:
%% Create categorical label data
numLabels = 18
labels = randi([1 2],numLabels,1);
labelsCat = categorical(labels,[1 2],{'crack-related','noise'});
%% find lndices w.r.t each label
crackInd = find(labelsCat==categorical("crack-related"));
noiseInd = find(labelsCat==categorical("noise"));
%% divide into train, val, test for each label
[trainCrack,valCrack,testCrack] = dividerand(numel(crackInd),0.7,0.1,0.2);
[trainNoise,valNoise,testNoise] = dividerand(numel(noiseInd),0.7,0.1,0.2);
%% get final indices
trainInd = [crackInd(trainCrack); noiseInd(trainNoise)];
valInd = [crackInd(valCrack); noiseInd(valNoise)];
testInd = [crackInd(testCrack); noiseInd(testNoise)];
You can optimize the above code based on your requirement like, dividing the indices randomly or not. Refer to Divide Data for Optimal Neural Network Training, dividerand, divideblock, divideint, divideind for more information.
trainInd = trainInd(randperm(numel(trainInd)));
0 comentarios
Más respuestas (0)
Ver también
Categorías
Más información sobre Statistics and Machine Learning Toolbox en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!