Read multiple text files and store data

Hi,

I want to read Multiple text files and store the data. I used the following code (I attached two text files,the number of text files will be more):

my_files = dir('*.txt');
N_files = numel( my_files );
A = zeros( numel(my_files),50 ); % initialize matrix to hold data
for k = 3:N_files %%(k start from 3 because, when I read the files, the first two are ".&,..)
  file2read = my_files(k).name;
  fid = fopen(file2read);
  A(k,:) = fscanf(fid, '%s', [1 inf]); % data from file
  fclose(fid);
end

But I am getting the following error: Please help me (I even used "textscan").

??? Error using ==> fscanf
Invalid file identifier.  Use fopen to generate a valid file identifier.

Error in ==> MultipleTextFileRead at 11
    A(k,:) = fscanf(fid, '%s', [1 inf]); % data from file

Kindly Help, Many thanks in advance.

Answer 1

[dpb]

my_files = dir('*.txt');
...
for k = 3:N_files

"(k start from 3 because, when I read the files, the first two are ".&,..)"

NO! When you do dir(*.txt') you've eliminated the directory entries and only returned the matching file names of the filename string passed. The directory entries only show up if you use an unqualified wild card that will match anything. So, having (wisely) not done that, iterate over 1:length(my_files), the files you specifically did want.

The lesson to be learned here is "investigate what you actually get, not what you THINK you're going to get". :)

Also, those files are output of some data collection system in a report format; to read them you'll have to write a specific routine to parse the particular pieces of data you wish. If it is only the last columnar data of a position and data value, then you can use the 'headerlines' option to skip the preceding information; otherwise if there is information within those that you need, you'll have to specifically handle how it is formatted to retrieve such.

ADDENDUM

Try something like...

d=dir('*.txt');  % all .txt files in working directory
N=length(d);     % how many did we find?
A=zeros(24,N);   % allocate for 24 points of data 1 column/file
for k=1:N
  [fid,msg]=fopen(d(k).name,'r');  % open for read permission
  if fid<0, error(['Failed fopen: ' d(k).name msg]),end % check opened it
  A(:,k)=cell2mat(textscan(fid, '%*d %f', ...
            'headerlines',54, 'collectoutput', 1));
  fid=fclose(fid);
end

If the dir is giving a non-NULL set of files but the fopen is still failing, the above will give you the error message as to why. I presumed that perhaps the files are protected as "read-only" whereas fopen by default gives you read/write access; hence I added the 'r' access indicator. You can test this in practice by removing it back to the default as had before.

For the textscan I copied the file into a code editor and observed the beginning of the first of the data at the end to find the numeric value "54" for a skip count; I also then wrapped the call inside cell2mat to return a regular double array (vector) instead of a cell array. I also then skipped the first column of xy position as seeming to be the same for all files; if you want it too, then reverting to a cell array for each file and a 2-column array for each may be more convenient data structure to refer to each group as an entity....that all depends on what the data are actually like in toto and what you want to do.

See

doc textscan % for all the gory details and examples on using it...

[Mekala balaji]

Sir,

I used:

for k = 1:length(my_files) %%1:N_files

But still I get the error:

??? Error using ==> fscanf
Invalid file identifier.  Use fopen to generate a valid file identifier.

Error in ==> MultipleTextFileRead at 11
    A(k,:) = fscanf(fid, '%s', [1 inf]); % data from file

By the way, Can you help me how to use header line to skip the some information, I am a newer to use matlab.

[dpb]

Have you tried from the command line and verified what your dir is actually returning? If you can't open the file after using the name returned from a directory search, it implies there's a problem somewhere...