MATLAB Answers

Assign values of .mat files into matrix

3 views (last 30 days)
Tessa Kol
Tessa Kol on 19 Oct 2020
Edited: Tessa Kol on 21 Oct 2020
Dear all,
I have multiple .mat files (see attachment). I want to organize the MFR_mod variables within the those .mat files into one matrix. I feel like there is a smarter way then what I tried below. This will take me forever, since MFR_mod goes from 1 until 81.
for ii = 1:20
load(['silomodresults',num2str(ii),'.mat'])
end
MFR_tot = [MFR_mod1; MFR_mod2; MFR_mod3; MFR_mod4; MFR_mod5; MFR_mod6; MFR_mod7; MFR_mod8; MFR_mod9; MFR_mod10; MFR_mod11; MFR_mod12; MFR_mod13; MFR_mod14; MFR_mod15; MFR_mod16; MFR_mod17; MFR_mod18; MFR_mod19; MFR_mod20];

  0 Comments

Sign in to comment.

Accepted Answer

Stephen Cobeldick
Stephen Cobeldick on 19 Oct 2020
Edited: Stephen Cobeldick on 19 Oct 2020
D = 'path to the folder where the files are saved';
S = dir(fullfile(D,'silomodresults*.mat'));
C = {};
for k = 1:numel(S)
F = fullfile(D,S(k).name);
T = load(F,'-regexp','^MFR_mod\d+$');
N = fieldnames(T);
V = str2double(regexp(N,'\d+','once','match'));
C(V) = struct2cell(T); %#ok<SAGROW>
end
M = vertcat(C{:}); % optional
Giving:
>> size(M)
ans =
81 1
>> plot(M)

  9 Comments

Show 6 older comments
Tessa Kol
Tessa Kol on 20 Oct 2020
Based upon your code and hints I managed to organize the data into a single matrix (see code below). Maybe it can be even shorter/nicer, but for now this works perfectly.
TimeStepstable = [3.04, 6.5];
TimeStepsettle = [1.56];
V_silo = 0.66*0.60*0.053-0.25*0.053*(0.25*tand(45));
%% Objective function input data
for ii = 19:20
load(['silomoddata',num2str(ii),'.mat']);
N_run(ii) = numel(runData_struc);
% Pre-allocate one cell for each loop variable
particlesInSilo = cell(max(cellfun(@numel,runData_struc)), N_run(ii));
siloMass = particlesInSilo;
time = particlesInSilo;
k =1;
for i = 1:N_run(ii)
for j = 1:numel(runData_struc{i})
%Count the number of particles in the silo (silo outlet is located at z = 0.3 m)
particlesInSilo{j,i} = find(expData_struc{1,i}{1,j}(:,3)>=0.3);
% Multiply the number of particles with the volume of the particle
siloMass{j,i} = sum(expData_struc{1,i}{1,j}(cell2mat(particlesInSilo(j,i)),6));
% Corresponding time step
time{j,i} = runData_struc{1,i}{1,j}(:,2);
k = k + 1;
end
end
% Index the time step values closest to the desired time step where the mass flow is stable
for i = 1:N_run(ii)
for t = 1:numel(TimeStepstable)
[~, idxtime(t,i)] = min(abs(cell2mat(time(:,i))-TimeStepstable(t)));
end
end
% Calculate the mass flow rate (MFR) for the stable region
for i = 1:N_run(ii)
siloMass_stable{i} = cell2mat(siloMass(idxtime(1,i):idxtime(2,i),i));
time_stable{i} = cell2mat(time(idxtime(1,i):idxtime(2,i),i));
opts = fitoptions('Method', 'LinearLeastSquares', 'Robust', 'off');
[MFR_fit{i}, GoodnessOfFit] = fit(time_stable{i}(:,1),siloMass_stable{i}(:,1),'poly1',opts);
MFR_mod(i) = abs(MFR_fit{i}.p1);
end
% Bulk density
for i = 1:N_run(ii)
for s = 1:numel(TimeStepsettle)
[~, idxtimesettle(s,i)] = min(abs(cell2mat(time(:,i))-TimeStepsettle(s)));
end
end
for i = 1:N_run(ii)
siloMass_settle(i) = cell2mat(siloMass(idxtimesettle(1,i),i));
rho_bmod(i) = siloMass_settle(i)/V_silo;
end
if ii == 1
idx = [1];
save(['silomodresults',num2str(ii),'.mat'],'MFR_mod','rho_bmod','idx')
elseif ii == 2
idx = [2];
save(['silomodresults',num2str(ii),'.mat'],'MFR_mod','rho_bmod','idx')
elseif ii == 3
idx = [3];
save(['silomodresults',num2str(ii),'.mat'],'MFR_mod','rho_bmod','idx')
elseif ii == 4
idx = [4];
save(['silomodresults',num2str(ii),'.mat'],'MFR_mod','rho_bmod','idx')
elseif ii == 5
idx = [5,11,20,38,81];
save(['silomodresults',num2str(ii),'.mat'],'MFR_mod','rho_bmod','idx')
elseif ii == 6
idx = [12,15,31,55,65];
save(['silomodresults',num2str(ii),'.mat'],'MFR_mod','rho_bmod','idx')
elseif ii == 7
idx = [21,22,23,24,25];
save(['silomodresults',num2str(ii),'.mat'],'MFR_mod','rho_bmod','idx')
elseif ii == 8
idx = [30,32,33,34,35];
save(['silomodresults',num2str(ii),'.mat'],'MFR_mod','rho_bmod','idx')
elseif ii == 9
idx = [40,41,42,43,44];
save(['silomodresults',num2str(ii),'.mat'],'MFR_mod','rho_bmod','idx')
elseif ii == 10
idx = [13,14,16,17,18];
save(['silomodresults',num2str(ii),'.mat'],'MFR_mod','rho_bmod','idx')
elseif ii == 11
idx = [50,51,52,53,54];
save(['silomodresults',num2str(ii),'.mat'],'MFR_mod','rho_bmod','idx')
elseif ii == 12
idx = [60,61,62,63,64];
save(['silomodresults',num2str(ii),'.mat'],'MFR_mod','rho_bmod','idx')
elseif ii == 13
idx = [70,71,72,73,74];
save(['silomodresults',num2str(ii),'.mat'],'MFR_mod','rho_bmod','idx')
elseif ii == 14
idx = [19,26,27,28,29];
save(['silomodresults',num2str(ii),'.mat'],'MFR_mod','rho_bmod','idx')
elseif ii == 15
idx = [6,7,8,9,10];
save(['silomodresults',num2str(ii),'.mat'],'MFR_mod','rho_bmod','idx')
elseif ii == 16
idx = [36,37,38,39,45];
save(['silomodresults',num2str(ii),'.mat'],'MFR_mod','rho_bmod','idx')
elseif ii == 17
idx = [46,47,48,49,56];
save(['silomodresults',num2str(ii),'.mat'],'MFR_mod','rho_bmod','idx')
elseif ii == 18
idx = [57,58,59,66,67];
save(['silomodresults',num2str(ii),'.mat'],'MFR_mod','rho_bmod','idx')
elseif ii == 19
idx = [68,69,75,76];
save(['silomodresults',num2str(ii),'.mat'],'MFR_mod','rho_bmod','idx')
elseif ii == 20
idx = [77,78,79,80];
save(['silomodresults',num2str(ii),'.mat'],'MFR_mod','rho_bmod','idx')
end
end
%% Combining input data
MFR_modtot = zeros(1,81);
rho_bmodtot = zeros(1,81);
for ii = 1:20
S(ii) = load(['silomodresults',num2str(ii),'.mat']);
MFR_modtot(S(ii).idx) = S(ii).MFR_mod;
rho_bmodtot(S(ii).idx)= S(ii).rho_bmod;
end
Stephen Cobeldick
Stephen Cobeldick on 20 Oct 2020
You could define all of the indices before the loop:
idc = {1,2,3,4,[5,11,20,38,81],[12,15,31,55,65],...};
and then inside the loop you can replace that entire huge lbock of code
if i== ..
..
elseif i== ..
..
..
end
with just this:
idx = idc{ii};
fnm = sprintf('silomodresults%d.mat',ii);
save(fnm,'MFR_mod','rho_bmod','idx')
It is still a mystery to me, how those indices are defined.
Tessa Kol
Tessa Kol on 21 Oct 2020
Thank you for the tip!
The indices are based on poor folder structure management from my side.

Sign in to comment.

More Answers (2)

Ameer Hamza
Ameer Hamza on 19 Oct 2020
Edited: Ameer Hamza on 19 Oct 2020
One of the major problem is naming the variables like this x1, x2, ...: https://www.mathworks.com/matlabcentral/answers/304528-tutorial-why-variables-should-not-be-named-dynamically-eval. You can try something like this
MFR_tot = zeros(1, 20);
for ii = 1:20
data = load(['silomodresults',num2str(ii),'.mat']);
MFR_tot(ii) = data.(sprintf('MFR_mod%d', ii))
end

  1 Comment

Tessa Kol
Tessa Kol on 19 Oct 2020
I came across the issue with eval many times in different posts. You solution didn't work unfortunatly, because the .mat files have unorganized MFR_mod. Maybe if I start from the beginning there is a better solution to this.
So first I have created 20 .mat files. Each .mat file is about 1.5 GB. So it is not possible to send it in this post.
For each .mat file I run the same code manually. The below code is an example of running the code for only the silomoddata20.mat file.
%% Loading the Data
load([pwd,'\silomoddata20.mat'])
%% Parameters
% Number of simulation runs
N_run = numel(runData_struc);
TimeStepstable = [3.04, 6.5];
TimeStepsettle = [1.56];
%% Mass flow Rate
% Pre-allocate one cell for each loop variable
particlesInSilo = cell(max(cellfun(@numel,runData_struc)), N_run);
siloMass = particlesInSilo;
time = particlesInSilo;
k =1;
for i = 1:N_run
for j = 1:numel(runData_struc{i})
%Count the number of particles in the silo (silo outlet is located at z = 0.3 m)
particlesInSilo{j,i} = find(expData_struc{1,i}{1,j}(:,3)>=0.3);
% Multiply the number of particles with the volume of the particle
siloMass{j,i} = sum(expData_struc{1,i}{1,j}(cell2mat(particlesInSilo(j,i)),6));
% Corresponding time step
time{j,i} = runData_struc{1,i}{1,j}(:,2);
k = k + 1;
end
end
% Index the time step values closest to the desired time step where the mass flow is stable
for i = 1:N_run
for t = 1:numel(TimeStepstable)
[~, idxtime(t,i)] = min(abs(cell2mat(time(:,i))-TimeStepstable(t)));
end
end
% Calculate the mass flow rate (MFR) for the stable region
for i = 1:N_run
siloMass_stable{i} = cell2mat(siloMass(idxtime(1,i):idxtime(2,i),i));
time_stable{i} = cell2mat(time(idxtime(1,i):idxtime(2,i),i));
opts = fitoptions('Method', 'LinearLeastSquares', 'Robust', 'off');
[MFR_fit{i}, GoodnessOfFit] = fit(time_stable{i}(:,1),siloMass_stable{i}(:,1),'poly1',opts);
MFR_mod(i) = abs(MFR_fit{i}.p1);
end
%% save results
save silomodresults20.mat MFR_mod rho_bmod
I run the above code manually for every silomoddata*.mat file. Because I didn't know how to put this in a for loop.
If the whole code is in for loop I hope to get a 1x81 matrix of MFR_mod. The problem with that is the order.
silomoddata1.mat contains simulationrun 1
silomoddata2.mat contains simulationrun 2
silomoddata3.mat contains simulationrun 3
silomoddata4.mat contains simulationrun 4
silomoddata5.mat contains simulationrun 5, simulationrun 11, simulationrun 20, simulationrun 38 and simulationrun 81
silomoddata6.mat contains simulationrun 12, simulationrun 15, simulationrun 31, simulationrun 55 and simulationrun 65
silomoddata20.mat contains simulationrun 77, simulationrun 78, simulationrun 79, simulationrun 80
... etc.
When I calculated the MFR_mod of every silomoddata.mat file I can't just simply combine them into a matrix. The orde isn't right. That was what I am trying to achieve here, a matrix of MFR_mod in the correct order of 1 until 81.

Sign in to comment.


Mathieu NOE
Mathieu NOE on 19 Oct 2020
hello
seems all your mat file contains the same size of data
so the modification of you r code is fairly simple - if I understand what you want :
MFR_tot = [];
for ii = 1:20
data = load(['silomodresults',num2str(ii),'.mat']);
MFR_tot = [MFR_tot; data]; % concatenation
end

  3 Comments

Tessa Kol
Tessa Kol on 19 Oct 2020
Dear Mathieu,
The mat file do not have the same size of data. For example,
silomodresults5.mat contains: MFR_mod5, MFR_mod11, MFR_mod20, MFR_38, MFR_81 (5 values)
silomodresults6.mat contains: MFR_mod12, MFR_mod15, MFR_mod31, MFR_mod55, MFR_mod65 (5 values)
silomodresults20.mat contains: MFR_mod77, MFR_mod78, MFR_mod79, MFR_mod80 (4 values)
Also, I want them also in acending order. I tried to do this manually as explained in my post.
Mathieu NOE
Mathieu NOE on 19 Oct 2020
ok , now I understand
is there any possibility for you to generate dummy mat files , but much smaller. I'd like to try to solve it
I think the key thing is to extract the field (variables) names and then re order the whole thing based on the numeric content of the variable name
That shoudn't be too difficult for someone fluent in string / structure processing (I don't mean me only !)
I'll give it a try tomorrow if you can send me some dummy mat files in between
Tessa Kol
Tessa Kol on 19 Oct 2020
Dear Mathieu,
I compressed silomoddata5.mat silomoddata6.mat and silomoddata20.mat into a zip file. You can download this using the link below (about 4 GB).
Is this oke?

Sign in to comment.

Products


Release

R2020a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by