Transform and Combine fileDatastores to Train CNN

15 views (last 30 days)
Hi I would appreciatte your help,
I have a collection of 50x1x12 mat files, that I need to upload into some datastore to subsequently pass into a convolutional neural network, how can I Train my
but instead of having .mat file for the labels I am using this function to get the labels from the folders.
function label = readLabel(filename,classNames)
filepath = fileparts(filename);
[~,label] = fileparts(filepath);
label = categorical(string(label),classNames);
end
I am uploading the files into the fileDatastore
inputData=fileDatastore(fullfile('inputData'),'ReadFcn',@load,'FileExtensions','.mat');
%getting the labels from the folders
classNames = string(1:2)
targetData=fileDatastore(fullfile('targetData'),'ReadFcn',@load,'FileExtensions','.mat',readLabel(filename,classNames));
Should I transform before combine the filedatastores? if so, how should my transform function be?
I tried combining without transforming and I got this error when I tried to train my CNN: (trainNetwork)
%combininig
cdsTrain = combine(inputData,targetData);
%Training
net = trainNetwork(cdsTrain,layers,options);
"The training images are of size 1x1x1 but the input layer expects images of size 50x1x12"
Thanks for your help!
  2 Comments
Mauricio Piraquive
Mauricio Piraquive on 21 Feb 2021
I am working with a "Image" like file when
50 would be the heigth of the "Image"
1 would be the width
12 would be the number of channels. (like 3 for rgb but in this case I have 12 channels).
These file correspond to the reading of a "material" using a lab instrument,
for example, one file could correspond to the results of the reading of material type-A
another file correspond to the results of the reading of material type-B
So I want to create this convolutional neural network, pass a reading and the network should identify the materia and tell me what type of Material that data correspond.
Right now I dont have any target data, I am getting the labels from the folders that contains each file. However, if needed I can create this output data files,
What Size this label files should be?
My current .mat files dont have the label in it.
Really appreciatte your help.
Regards,
Mauricio

Sign in to comment.

Accepted Answer

luisa di monaco
luisa di monaco on 22 Feb 2021
Edited: luisa di monaco on 22 Feb 2021
Hi, this is my code. I know it is not efficient, but it should work! I hope it can help as a starting point to be optimized.
For simplicity, I added information about category inside each .mat file as string variable. However, I'm sure you can get labels from folder names too.
The key idea is to use fileDatastore for input data and tabularTextDatastore for categorical data. Then you can combine them succesfully.
Regards
clear
close all
clc
%% create dummy inputs and targets (.mat file)
% inputs are images of size 50x1x12 [heigth, width, channels]
% target are type of material: categorical array
mkdir trainingData
cd trainingData
n=10; % number of training cases
for i=1:n
material=ones(50,1,12)*i;
if i<n/2
matType='typeA';
else
matType='typeB';
end
mkdir(matType)
filename=sprintf('material_%d', i);
save(fullfile(matType,filename),'material', 'matType')
end
cd ..
%% load data
allData=fileDatastore(fullfile('trainingData'),'ReadFcn',@load,'FileExtensions','.mat', 'IncludeSubfolders', true);
%% create inputs
inputData = transform(allData,@(data) rearrange_input(data)); % extract and rearrange input
%% create targets (can be optimized...)
targetData = transform(allData,@(data) rearrange_target(data));
myLabels=targetData.readall;
writematrix(myLabels,'myLabels.txt');
labelStore = tabularTextDatastore('myLabels.txt','TextscanFormats','%C',"ReadVariableNames",false);
read_size=1; % this line is foundamental... I don't know why...
labelStore.ReadSize = read_size;
labelStoreCell = transform(labelStore,@setcat_and_table_to_cell);
%% combininig
cdsTrain = combine(inputData,labelStoreCell);
%% training
% dummy layers and options
numClasses=2;
layers=[
imageInputLayer([50 1 12],"Name","imageinput")
batchNormalizationLayer()
leakyReluLayer(0.1)
fullyConnectedLayer(numClasses,'Name','fc')
softmaxLayer('Name','soft')
classificationLayer('Name','classification')];
options = trainingOptions('adam');
net = trainNetwork(cdsTrain,layers,options);
%% functions
function inputData = rearrange_input(data)
inputData=data.material;
inputData= {inputData};
end
function targetData = rearrange_target(data)
targetData=data.matType;
targetData=categorical(cellstr(targetData));
end
function [dataout] = setcat_and_table_to_cell(datain)
validcats = ["typeA", "typeB"];
datain.(1) = setcats(datain.(1),validcats);
dataout = table2cell(datain);
end
  1 Comment
Mauricio Piraquive
Mauricio Piraquive on 26 Mar 2021
@luisa di monaco Thank you very much this was very helpful. Really appreciate it.

Sign in to comment.

More Answers (1)

Mahesh Taparia
Mahesh Taparia on 24 Feb 2021
Hi
One possible approach is already suggested by Luisa. In your approach, replace your read function with the following line of code
function label = readLabel(filename)
filepath = fileparts(filename);
[~,label] = fileparts(filepath);
label = categorical(string(label));
end
Also, the fileDatastore of label data is not created properly, replace it with following line of code
targetData=fileDatastore(fullfile('targetData'),'ReadFcn',@readLabel,'FileExtensions','.mat');
Try it, it may work. If it won't work, apply break points in readLabel function and try to create the function readLabel such that it should returns the categorical label. Hope it will help!

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by