Loop cycle to input similar txt files

Hello everybody, I'm trying to solve a problem to input data. I have 39 txt files that have similar names. The name has two variables and I'd like to import the data as a function that contains those two numbers.

So a general name of the file is "CV_d{m}_die{i}", i goes from 1 to 13 and m is equal to 100, 200 or 400.

I've done this

input_folder = 'folder'

files = dir(fullfile(input_folder, "*.txt"))

file_paths = fullfile({files.folder}, {files.name})

for i = 1:13
    for m = [100 200 400]
       Data_{m}_{i} = dlmread("CV_d{m}_die{i}.txt", ',', 4, 0)
    end
end

So my idea was to do this loop and get 39 Data_m_i. Like Data_100_1, Data_200_1, Data_400_1, and then i=2...

But I get an error that says that the file could not be opened because there's no such file or directory.

Answer 1

[Jan]

Data_{m}_{i} is your invention an no valid Matlab syntax. This is not the way programming works.

Matlab does not magically modifiy the string "CV_d{m}_die{i}.txt" to what you want. Why do you assume that the curly braces let Matlab inserts the numbers?

Use a command to create a specific string:

dlmread(sprintf('CV_d%d_die%d.txt', m, i), ',', 4, 0);

Why do you create a list of filenames, which is not used?

Data_1_100 is a bad choice for the name of a variable. Hiding information in the name of a variable makes it hard to access it later. Store such important information such, that you can use the later:

k = 0;
for i = 1:13
    for m = [100 200 400]
        file = fullfile(folder, sprintf('CV_d%d_die%d.txt', m, i));
        k = k + 1;
        Data(k).value = dlmread(file, ',', 4, 0);
        Data(k).m     = m;
        Data(k).i     = i;
    end
end

Hiding information in names of variables is a typical design error made by beginners. See this exhaustive explanation, why other methods should be preferred: TUTORIAL: Why Variables Should Not Be Named Dynamically (eval)

[Simone]

It does not work.

I've never said to be a programmer. If I were I wouldn't have needed help.

[Jan]

A purpose of this forum is to help you to improve your Matlab skills.

If you explain "It does not work" with details, we (you and me and the other readers) can modify the code until it solves your problem.

By the way, I'm not a programmer also. I've started participating in Matlab forums, because I've needed help. I've learned a thing or two and think it is worth to share it now.

[Simone]

I've imported the folder and it says that the file "" could not be opened because there's no such file in the directory, and in the folder there's the exact same file name

[Jan]

Please post a copy of the complete error message. Is the file really called "" ? And it is existing? Is the folder specified correctly as absolute path?

Insert a check for the existing of the file:

...
file = fullfile(folder, sprintf('CV_d%d_die%d.txt', m, i));
if ~isfile(file)
    error('File not found: %s', file);
end
...

[Simone]

Yes, the error says "Error using dlmread

The file 'folder-name\CV_d100_die4.txt'

could not be opened because: No such file or directory"

And looking at the folder, the file exist, I don't know if I explained myself. The file is saved in the folder when I look for it

I'm gonna paste the code also, so it's visible

input_folder = 'folder'

files = dir(fullfile(input_folder, "*.txt"))

file_paths = fullfile({files.folder}, {files.name})

k = 0;
for i = 1:14
    for m = [100 200 400]
        file = fullfile(input_folder, sprintf('CV_d%d_die%d.txt', m, i));
        k = k + 1;
        Data(k).value = dlmread(file, ',', 4, 0);
        Data(k).m     = m;
        Data(k).i     = i;
    end
end

[Jan]

@Simone: Remember, that Matlab is a deterministic program. If it tells you, that the file is not there, it is not there. If you look in the folder and do find this file, the only not magic explanation is, that you do not look in the same folder as Matlab does.

I've posted some code, which mentions the folder in the error message, if the file is missing. Unfortunately you did not insert it in your code.

Instead you include these lines:

files = dir(fullfile(input_folder, "*.txt"))
file_paths = fullfile({files.folder}, {files.name})

But for which purpose?

You post the message:

The file 'folder-name\CV_d100_die4.txt'

The most important part is hidden in this message: what is "folder-name"? Did you really call this folder "folder-name"?

In a former comment you have mentioned the error message: it says that the file "" could not be opened . Here the most important part is the "". Do you really try to open a file with an empty name or do you mask this part, because you think it is not important? Then please consider: This is the important point. It is challenging, if you remove exactly this information repeatedly from the posted message.

Please trust my suggestion and run this code:

input_folder = 'C:\Insert\TheRealName\OfYour\Folder\Here';
k = 0;
for i = 1:14
    for m = [100 200 400]
        file = fullfile(input_folder, sprintf('CV_d%d_die%d.txt', m, i));
        if ~isfile(file)
           error('File not found: %s', file);
        end
        k = k + 1;
        Data(k).value = dlmread(file, ',', 4, 0);
        Data(k).m     = m;
        Data(k).i     = i;
    end
end

Now run this code. If it stops with a message, Matlab tells you clearly, that the folder (mentioned in the error message) does not contain this file.

You explain "And looking at the folder, the file exist" - how do you look into which folder?

Maybe it matters, that you have mentioned the pattern "CV_d{m}_die{i}.txt" in the original question, but also: "Data_100_1, Data_200_1, Data_400_1". Well, do you mean the pattern "Data_%d_%d.txt" ?

Search with the correct pattern in the correct folder. If this is not successful, post the output of

files = dir(fullfile(input_folder, "*.txt"))
file_paths = fullfile({files.folder}, {files.name})

as MAT file here. Then maybe a file name contains Unicode characters.

[Simone]

Sorry I didn't understand that I had to remove those first lines, yes it perfectly works.

Doing so I get a table that has the imported data associated with each m and i value. But if I want to work with the second column of the data associated to m=100 and i=2 for example, how am I supposed to specify that particular data-file in the script?

Clicking to see the table of a particular imported data says the name Data with a number on parenthesis

[Jan]

"Sorry I didn't understand that I had to remove those first lines" - I do not understand. There is no need to remove the first lines, they are just useless - as long as the folder name is set correctly.

If you want the values for m=3 and i = 200:

m_list = [Data.m];
i_list = [Data.i];
value = Data(m_list == 3 & i_list == 200).value;

Now you can run a loop easily over all data.

Answer 2

[Image Analyst]

See the FAQ: https://matlab.fandom.com/wiki/FAQ#How_can_I_process_a_sequence_of_files?

Use the first snippet, not the second method like you're doing.

Avoid the problem of missing files by using dir() to get only files that actually exist.

% Specify the folder where the files live.
input_folder = 'C:\Users\yourUserName\Documents\My Pictures';
% Check to make sure that folder actually exists.  Warn user if it doesn't.
if ~isfolder(input_folder)
    errorMessage = sprintf('Error: The following folder does not exist:\n%s\nPlease specify a new folder.', myFolder);
    uiwait(warndlg(errorMessage));
    input_folder = uigetdir(); % Ask for a new one.
    if input_folder == 0
        % User clicked Cancel
        return;
    end
end
% Get a list of all files in the folder with the desired file name pattern.
filePattern = fullfile(input_folder, '*.txt'); % Change to whatever pattern you need.
theFiles = dir(filePattern);
for k = 1 : length(theFiles)
    baseFileName = theFiles(k).name;
    fullFileName = fullfile(theFiles(k).folder, baseFileName);
    fprintf(1, 'Now reading %s\n', fullFileName);
    
    % Now do whatever you want with this file name,
    % such as parsing it to get i and m.
    % Parse 'CV_dm_diei.txt'
    underlineLocations = strfind(baseFileName, '_');
    m = str2double(baseFileName(underlineLocations(1) + 2 : underlineLocations(2) - 1));
    dieIndex - strfind(baseFileName, '_die');
    i = str2double(baseFileName(dieIndex + 4: end - 4));
    
    % Read in the data (code below is not mine or from the FAQ).
    Data(k).value = dlmread(fullFileName, ',', 4, 0);
    Data(k).m     = m;
    Data(k).i     = i;
end