Reading data from multiple files in ascending order.

I have many folders and in each I have potentially 480 txt files (“potentially” because in some folders some files are missing) which names changing in ascending order from:

aaaa001a00.txt

aaaa001a30.txt

aaaa001b00.txt

aaaa001b30.txt

…

aaaa001x00.txt

aaaa001x30.txt

aaaa002a00.txt

…

aaaa010x30.txt

(bold number change from 1 to 10, bold italic letter changes from “a” to “x” (24 letters) and number after this letter is 00 or 30).

In each file there are four columns with data, as below:

0.5031 0.5709 0.1937 0.7852

0.5074 0.5719 0.1878 0.7873

0.5127 0.5727 0.1823 0.7900

…

0.5170 0.5729 0.1787 0.7921

0.5211 0.5733 0.1730 0.7938

But the number of line can be different in each file…

In each folder I would like to read all 480 “potentially” files in ascending order (as listed).

If file exist I would like to read data from the last line… (for example: 0.5211 0.5733 0.1730 0.7938)

If file don’t exist I would like “to read”: NaN NaN NaN NaN.

How it could be done in matlab?

Answer 1

[Mathieu NOE]

hello

this is my suggestion , based on natsortfiles (from FEX : Natural-Order Filename Sort - File Exchange - MATLAB Central (mathworks.com), also attached here ) that allows you to load the files according to natural name sorting

then each last line of the data is stored in a cell ,and the final array is then saved as excel file

now I don't understand how you could do the "NaN" line for a file that does not exist and therefore cannot be identified in the folder with dir ??

my code :

fileDir = cd;

outfile = 'OUT.xlsx';

fileNames = dir(fullfile(cd,'*.txt')); % get list of files in directory
fileNames_sorted = natsortfiles({fileNames.name}); % sort file names into order
M= length (fileNames_sorted);

for f = 1:M 
    raw = importdata( fullfile(fileDir, fileNames_sorted{f}));
    [m,n] = size(raw);
    last_line{f} = raw(m,:);
end

% write all last lines into a matrix and store it in excel file 
writecell(last_line',fullfile(cd,outfile));

[Stephen23]

The leading zeros on the integers means that NATSORTFILES is not required:

C = {'aaaa001a30.txt';'aaaa001b00.txt';'aaaa001b30.txt';'aaaa001a00.txt';'aaaa001x00.txt';'aaaa010x30.txt';'aaaa001x30.txt';'aaaa002a00.txt'};
C = C(randperm(numel(C)))

C = 8×1 cell array
    {'aaaa002a00.txt'}
    {'aaaa001b30.txt'}
    {'aaaa001a30.txt'}
    {'aaaa001a00.txt'}
    {'aaaa010x30.txt'}
    {'aaaa001x00.txt'}
    {'aaaa001b00.txt'}
    {'aaaa001x30.txt'}

D = sort(C) % no need for NATSORTFILES

D = 8×1 cell array
    {'aaaa001a00.txt'}
    {'aaaa001a30.txt'}
    {'aaaa001b00.txt'}
    {'aaaa001b30.txt'}
    {'aaaa001x00.txt'}
    {'aaaa001x30.txt'}
    {'aaaa002a00.txt'}
    {'aaaa010x30.txt'}

The challenge is to detect the missing files.

[UWM]

Thank you very much for your suggestions - they were very helpful.