How to read in a lot of files, average a column, and output in a new file?

Hi, I want to read in a bunch of different .csv files from a folder, average the 8th column of each individual one, and then output the averages into a new .csv file as a single column list. I have tried some code so far, but I cannot figure out how to apply it to multiple files in a folder rather than just reading in one file.

M = csvread()
average = mean(M(:,8));
csvwrite('output.csv',average);

I also read to do something like this to read in multiple files but I'm not sure how to integrate these two chunks of code.

csvFiles = dir('*.csv');
numfiles = length(csvFiles);
mydata = cell(1, numfiles);
for k = 1:numfiles
   mydata{k} = imread(csvFiles(k).name);
end

Thanks for your help, I'm pretty new to Matlab so please bear with me :)

Answer 1

[Jan]

folder   = 'C:\Your\Folder\';
csvFiles = dir(fullfile(folder, '*.csv'));  % Use absolute path names
numfiles = length(csvFiles);
average  = zeros(1, numfiles);
for k = 1:numfiles
  M          = csvread(fullfile(folder, csvFiles(k).name));
  average(k) = mean(M(:,8));
end
csvwrite(fullfile(folder, 'output.csv'), average);

[amcgough]

Thanks, would there be a way to add in that I want to start at the 2nd row of the matrix to exclude headers? I know the syntax for csvread adds 1,2 at the end to do this but not sure how I'd add it into this code.

[Jan]

If "csvread adds 1,2 at the end" means, that the arguments "1, 2" are appended to the inputs of csvread, simply do this: Replace

M = csvread(fullfile(folder, csvFiles(k).name));

by

M = csvread(fullfile(folder, csvFiles(k).name), 1, 2);

[Ayesha Batool]

Hi Jan

my current folder in is

participant_folders = dir('D:\PhD\Research\Experimental Design\Experiment 1\DATA\Analysis\Nystrom');

I want the code to read each folder in folder_in but not the files in it, instead read two folders in, 100514\detectectionrecults\ then read the csv file ending in 'Fixaduration.csv' and output a file csv with an average.

If I want to access this folder D:\PhD\Research\Experimental Design\Experiment 1\DATA\Analysis\Nystrom\100514\DetectionResults\1_1\averages

it takes up the list of csv and outputs the average in a csv.

However I want to automate the algorithm to go through hundreds of folders and output one average for a file ending in a certain name.

the file path way is If I want to access this folder D:\PhD\Research\Experimental Design\Experiment 1\DATA\Analysis\Nystrom\100514\DetectionResults\1_1\averages\1_1_FixationDuration.csv

This is working for one folder:

folder = 'D:\PhD\Research\Experimental Design\Experiment 1\DATA\Analysis\Nystrom\100514\DetectionResults\1_1\averages\';

csvFiles = dir(fullfile(folder, '*.csv')); % Use absolute path names

numfiles = length(csvFiles);

average = zeros(1, numfiles);

for k = 1:numfiles

M = csvread(fullfile(folder, csvFiles(k).name));

average(k) = mean(M(:,1));

end

csvwrite(fullfile(folder_out, 'output.csv'), average);

but

I'm trying to write it for all participants:

participant_folders = dir('D:\PhD\Research\Experimental Design\Experiment 1\DATA\Analysis\Nystrom');

isub = [participant_folders(:).isdir]; %# returns logical vector

nameFolds = {participant_folders(isub).name}';

nameFolds(ismember(nameFolds,{'.','..'})) = [];

for participant = 1:size(nameFolds,1)

folder_in = ['D:\PhD\Research\Experimental Design\Experiment 1\DATA\Analysis\Nystrom\',...

nameFolds{participant},'\'];

files = dir([folder_in, '*.csv']);

for f = 1:size(files,1)

Answer 2

[Angana Borah]

What if I want find the average of only a few files from that directory, instead of the fullfile, whose ".name" I know?