How do I get a structure to include all the data that I want?

folder_list = dir([data_dir filesep currentfolder(3).name]);

this is my code. I want folder_list to grab everything from 3 to 7. if I use a simple for loop from 3:7, it just gives me the data for 7.

[Jan]

If you post your code, we can find the probvlem and suggest a solution.

Prefer fullfile instead of a hardcoded concatenation:

dir(fullfile(data_dir, currentfolder(3).name));

Why do you start at 3? Do you assume that . and .. are guarateed to be the first two elements? This is not the case.

[Khush Patel]

I do not need the first two elements so I start at 3. More details: 3 is "brush teeth" with accelerometer data in regards to time. 4, 5, etc would be other activities like "running". I want the final table to include all activities. Right now, if I put this code in a loop it only includes the "7" which is walking.

[Khush Patel]

%%step 1 access and explore data
% Code provided.
clear
addpath(genpath(pwd));
data_dir = fullfile(pwd, 'Human_Activities', 'Selected_Data_from_HMP_Dataset');

% Code provided.
currentfolder=dir(data_dir);

%%step 2 preprocess data and extract features
folder_list = dir([data_dir filesep currentfolder(3).name]);
% Code Provided.
reference_table = table();
Label = table();

% Code Provided.
for ifolder = 1:length(folder_list)  
    % Code Provide.
    if folder_list(ifolder).bytes > 0
        fid = importdata(folder_list(ifolder).name);
        % Code Provided.
        
        %In the following code, we apply the extractAccFeatures function to 
        % generate a reference_table which stores the features from each 
        % accelerometer data in separate rows.
        reference_table = [reference_table; extractAccFeatures(fid)];
        % Code Provided.
        Label = [Label; {currentfolder(3).name}];
    end 
end


% Code Provided.
Label.Properties.VariableNames = {'Activities'};
reference_table = [reference_table, Label]

[Stephen23]

"I want folder_list to grab everything from 3 to 7."

That looks suspiciously like a buggy attempt to ignore the dot directory names:

https://www.mathworks.com/matlabcentral/answers/345841-unexplained-error-on-dir#answer_271627

https://www.mathworks.com/matlabcentral/answers/21150-function-call#comment_46042

https://www.mathworks.com/matlabcentral/answers/394807-create-text-file-containing-list-of-file-names#comment_556641

https://www.mathworks.com/matlabcentral/answers/288815-gui-for-loop-not-working-correctly#answer_224956

https://www.mathworks.com/matlabcentral/answers/484430-using-struct-dir-selpath-what-do-and-mean-as-struct-name#answer_395754

https://www.mathworks.com/matlabcentral/answers/40949-omitting-pointers-when-listing-folder-contents#answer_50568

https://www.mathworks.com/matlabcentral/answers/13978-list-of-files-sorting#answer_19278

As Walter Roberson wrote in that last link: "In short: if your code assumes that '.' and '..' are the first two entries in a directory, your code has a bug (even in MS Windows). If your code assumes that directory entries are returned in any sorted order, your code has a bug (in all OS.)"

[Jan]

@Khush Patel: Do not do this:

addpath(genpath(pwd));

Adding the folder, which contains data files, to Matlab's PATH has no advantage, but severe disadvantages. Matlab's PATH should include only the folders containing the M-files of the used functions. Use absolute path names for data files instead:

fid = importdata(fullfile(data_dir, folder_list(ifolder).name));

I do not see in your code the command matching the description "if I use a simple for loop from 3:7".

Answer 1

[Walter Roberson]

For example, to read all png files in all subfolders of data_dir, storing the input as a cell array of cell arrays broken up by folder.

dinfo = dir(data_dir);
dinfo(~[dinfo.isdir]) = [];  %remove non-folders
dinfo(ismember({dinfo.name}, {'.', '..'})) = [];  %remove . and ..
foldernames = fullfile({dinfo.folder}, {dinfo.name});

numfolders = length(foldernames);
folder_images = cell(numfolders, 1);
for D = 1 : numfolders
    thisfolder = foldernames{D};
    finfo = dir(thisfolder, '*.png');
    filenames = fullfile({finfo.folder}, {finfo.name});
    numfiles = length(filenames);
    these_images = cell(numfiles, 1);
    for F = 1 : numfiles
        thisfile = filenames{F};
        these_images{F} = imread(thisfile);
    end
    folder_images = these_images;
end

[Khush Patel]

Any hints to how I could apply it to this code?

%%step 1 access and explore data
% Code provided.
clear
addpath(genpath(pwd));
data_dir = fullfile(pwd, 'Human_Activities', 'Selected_Data_from_HMP_Dataset');
% Code provided.
currentfolder=dir(data_dir);
%%step 2 preprocess data and extract features
folder_list = dir([data_dir filesep currentfolder(3).name]);
% Code Provided.
reference_table = table();
Label = table();
% Code Provided.
for ifolder = 1:length(folder_list)  
    % Code Provide.
    if folder_list(ifolder).bytes > 0
        fid = importdata(folder_list(ifolder).name);
        % Code Provided.
        
        %In the following code, we apply the extractAccFeatures function to 
        % generate a reference_table which stores the features from each 
        % accelerometer data in separate rows.
        reference_table = [reference_table; extractAccFeatures(fid)];
        % Code Provided.
        Label = [Label; {currentfolder(3).name}];
    end 
end
% Code Provided.
Label.Properties.VariableNames = {'Activities'};
reference_table = [reference_table, Label]

[Walter Roberson]

Adjust ThingToIgnore and OtherThingToIgnore to be the names of the objects in the directory that you want to ignore. Do not count on them being in a particular relative position.

data_dir = fullfile(pwd, 'Human_Activities', 'Selected_Data_from_HMP_Dataset');

currentfolder = dir(data_dir);
currentfolder(~[currentfolder.isdir]) = [];  %remove non-folders
currentfolder(ismember({currentfolder.name}, {'.', '..', 'ThingToIgnore', 'OtherThingToIgnore'})) = []; %remove . and .. and two things not of interest

folder_list = fullfile( {currentfolder.folder}, {currentfolder.name} );
numfolders = length(folder_list);

reference_table = cell(numfolders, 1);
Labels = cell(numfolders, 1);
for D = 1 : numfolders
    thisfolder = folder_list{D};
    [~, folderlabel, ~] = fileparts(thisfolder);
    finfo = dir(thisfolder);
    finfo([finfo.isdir]) = [];  %remove folders
    filenames = fullfile({finfo.folder}, {finfo.name});
    numfiles = length(filenames);
    feats_cell = cell(numfiles, 1);
    feats_labels = cell(numfiles, 1);
    for F = 1 : numfiles
        thisfile = filenames{F};
        thisdata = importdata(thisfile);
        feats_cell{F} = extractAccFeatures(thisdata);
        feats_labels{F} = folderlabel;
    end
    reference_table{K} = vertcat(feats_cell{:});
    Labels{D} = feats_labels;
end

reference_table = vertcat(reference_table{:});
Labels = vertcat(Labels{:});

[Khush Patel]

I don't understand this. Sorry. Can you help me change my existing code to include everything from 3-7. Currently, when I change currentfolder(3) to currentfolder(5) it will grab data from my 5th folder. I know it's a simple fix but I can't figure it out. I want the currentfolder to go from 3 to 7.

[Image Analyst]

Why 3-7? Are you trying to avoid . and .. which is what this does:

currentfolder(~[currentfolder.isdir]) = [];  %remove non-folders

or are your first and second folder not . and ..?

[Walter Roberson]

No, I cannot do that. Your current code is incorrect in design.

You should never rely on the relative position of files within a folder. None of the file systems you are likely to ever encounter impose any internal ordering on names, and none of the operating systems supported by MATLAB define a fixed order to return names in response to queries of directory information.

The NTFS file system commonly used by Windows has some "typical" behaviour, but the typical behaviour is not strictly adhered to, and is different from what most people expect... and depends in part on the region setting of the user who created the file system. So copying between drives can result in different ordering of files.

You should always code in terms of accepting or rejecting files based upon names or other attributes, not on the basis of order of files returned by dir()