Read text data containing variables in rows and columns and to split them out paragraph wise in different variables.

I would like to read the text data below and seperate it in different parts such that (__PARAMETER__) variable will contain all the data below it, (__RESULTS__) variable would contain data below it and so on. Then would like to transpose the (__PARAMETER__) and the (__RESULTS__)column. Search the variables in and take the mean (average) values of only certain parameters (like box temperature, pressure and input temperature) and display them in a different variable.

__MEASUREMENT_INFO__

Date / Time: 07.01.2010, 17:14

Sample ID: Sample A / 100 %

Comment: 225 °c / 20 bar

__PARAMETER__

Box Temperature [°C]: 224 224.55 224.55 224.55 224.55 224.55 224.70 224.60 224.55 224.60 224.55 224.70

Box Pressure [bar]: 20.12 20.11 20.11 20.10 20.10 20.13 20.10 20.12 20.12 20.13 20.11 20.10

Input Pressure [bar]: 987.00 987.00 988.00 988.00 987.00 987.00 986.00 987.00 988.00 987.00 987.00 988.00

Input Duration [µsec]: 1200 0 0 0 0 0 0 0

Input Temperature [°C]: 107.20 107.30 107.20 107.10 107.10 107.10 107.10 107.10 107.10 107.10 107.00 107.10

Pressure Transmitter Temperature [°C]: 87.60 87.70 87.60 87.60 87.60 87.30 87.80 87.50 87.50 87.60 87.50 87.40

Samples Pressure 500 ms after Inj. [bar]: 372.00 345.00 344.00 345.00 350.00 340.00 351.00 331.00 347.00 344.00 349.00 350.00

Gas A, compressed air [%]: 100.0

Gas B, [%]: 0.0

__RESULTS__

Time Delay [msec]: 6.1943 6.2343 6.2264 6.3532 6.1839 6.3281 6.2283 6.3972 6.3166 6.2314 6.1887 6.3808

Mean [msec]: 6.2719

Standard Deviation [msec]: 0.0781

CN calc [-]: 0.00

CN Path: C:\ProgramData\

__DATA__

1 2 3 4 5 6 7 8 9 10 11 12 Resolution[µs]

-14.560 -14.272 -15.012 -14.904 -14.468 -14.953 -14.256 -15.350 -14.039 -14.360 -14.153 -14.670 4

-14.560 -14.272 -15.012 -14.904 -14.470 -14.953 -14.256 -15.350 -14.038 -14.360 -14.154 -14.669 4

-14.560 -14.270 -15.012 -14.904 -14.469 -14.954 -14.256 -15.350 -14.038 -14.360 -14.153 -14.668 4

-14.561 -14.270 -15.011 -14.905 -14.469 -14.954 -14.256 -15.352 -14.037 -14.360 -14.152 -14.670 4

-14.560 -14.270 -15.011 -14.904 -14.468 -14.953 -14.256 -15.352 -14.040 -14.360 -14.152 -14.670 4

-14.560 -14.271 -15.011 -14.905 -14.469 -14.954 -14.256 -15.352 -14.038 -14.361 -14.152 -14.670 4

-14.561 -14.271 -15.013 -14.905 -14.469 -14.955 -14.255 -15.352 -14.038 -14.360 -14.151 -14.669 4

-10.038 -9.831 -10.711 -10.848 -10.063 -10.873 -9.914 -10.708 -9.880 -10.118 -9.769 -10.536 40

-10.042 -9.835 -10.717 -10.853 -10.063 -10.879 -9.922 -10.712 -9.881 -10.120 -9.775 -10.540 40

__MSS__

MSS [mg/m3] Time[ms] Chamber Pressure[bar]

__CLD__

CLD [ppm] Time[ms] Chamber Pressure[bar]

Answer 1

[Mathieu NOE]

hello

this is not a 100% finished solution, but you can get a pretty good idea how to finalize it

you get the DATA, PARAMETERS and RESULTS (not everything , I left you some work :))

main code :

%%% main code %%% 
file_list = dir ('*txt');
M= length (file_list);
[DATA,PARAMETER,RESULTS] = retrieve_data(file_list(1).name);


function [DATA,PARAMETER,RESULTS] = retrieve_data(Filename)

fid = fopen(Filename);
tline = fgetl(fid);

% initialization
k = 0;
p = 0;
q = 0;
r = 0;
s = 0;
t = 0;

%%%%%%%% first loop : scan entire file to find pages (sections ) that must be evaluated %%%%%%%%%%%

while ischar(tline)
    
    k = k+1;    % loop over line index
    
   % retrieve line number  of __PARAMETER__
    if contains(tline,'__PARAMETER__')   
        k_parameter = k;
        p = p+1;
        Parameters_line_index(p,1) = k;  % 
    end
    
   
   % retrieve line number  of __RESULTS__
    if contains(tline,'__RESULTS__')   
        k_results= k;
        q = q+1;
        Results_line_index(q,1) = k;   % 
    end

   % retrieve line number  of __DATA__
    if contains(tline,'__DATA__')   
        k_data = k;
        r = r+1;
        Data_line_index(r,1) = k;   % 
    end
    
    % retrieve line number  of __MSS__
    if contains(tline,'__MSS__')   
        k_mss = k;
        s = s+1;
        MSS_line_index(r,1) = k;   % 
    end
    
     % retrieve line number  of ___CLD__
    if contains(tline,'__CLD__')   
        k_cld = k;
        t = t+1;
        CLD_line_index(r,1) = k;   % 
    end
    
tline = fgetl(fid);
    
end

fclose(fid);

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%% second loop  : data extraction all sections %%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
k = 0;
fid = fopen(Filename);
tline = fgetl(fid);

while ischar(tline)
    
    k = k+1;
    
    % job for __PARAMETER__ data retrieval
        %     __PARAMETER__
        % Box Temperature [°C]:		224	224.55
        % Box Pressure [bar]:		20.12	20.11
        % Input Pressure [bar]:		987.00	987.00
        % Input Duration [µsec]:		1200	0	
        % Input Temperature [°C]:		107.20	107.30
        % Pressure Transmitter Temperature [°C]:	87.60
        % Samples Pressure 500 ms after Inj. [bar]:	
        % Gas A, compressed air [%]:			
        % Gas B,  [%]:				0.0

   
        if (k == k_parameter+1) % Box Temperature [°C]:
            [m,n] = size(tline); string = tline(1:n);
             % find index of ":" mark
             ind_separator = findstr(string,':');
%              Name = string(1:ind_separator-1); data_string  = string(ind_separator+3:n);
             PARAMETER.Box_Temperature = regexp(string(ind_separator+3:n), '\t', 'split');

        elseif (k == k_parameter+2) % Box Pressure [bar]:
            [m,n] = size(tline); string = tline(1:n);
             % find index of ":" mark
             ind_separator = findstr(string,':');
%              Name = string(1:ind_separator-1); data_string  = string(ind_separator+3:n);
             PARAMETER.Box_Pressure = regexp(string(ind_separator+3:n), '\t', 'split');
             
         elseif (k == k_parameter+3) % Input Pressure [bar]:
            [m,n] = size(tline); string = tline(1:n);
             % find index of ":" mark
             ind_separator = findstr(string,':');
             % Name = string(1:ind_separator-1);  data_string  = string(ind_separator+3:n);
             PARAMETER.Input_Pressure =  regexp(string(ind_separator+3:n), '\t', 'split');

         elseif (k == k_parameter+4) % Input Duration [µsec]:
            [m,n] = size(tline); string = tline(1:n);
             % find index of ":" mark
             ind_separator = findstr(string,':');
             % Name = string(1:ind_separator-1);  data_string  = string(ind_separator+3:n);
             PARAMETER.Input_Duration =  regexp(string(ind_separator+3:n), '\t', 'split');

          elseif (k == k_parameter+5) % Input Temperature [°C]
            [m,n] = size(tline); string = tline(1:n);
             % find index of ":" mark
             ind_separator = findstr(string,':');
             % Name = string(1:ind_separator-1);  data_string  = string(ind_separator+3:n);
             PARAMETER.Input_Temperature =  regexp(string(ind_separator+3:n), '\t', 'split');
            
         elseif (k == k_parameter+6) % Pressure Transmitter Temperature [°C]
            [m,n] = size(tline); string = tline(1:n);
             % find index of ":" mark
             ind_separator = findstr(string,':');
             % Name = string(1:ind_separator-1);  data_string  = string(ind_separator+3:n);
             PARAMETER.Pressure_Transmitter_Temperature =  regexp(string(ind_separator+3:n), '\t', 'split');

         elseif (k == k_parameter+7) % Samples Pressure 500 ms after Inj. [bar]
            [m,n] = size(tline); string = tline(1:n);
             % find index of ":" mark
             ind_separator = findstr(string,':');
             % Name = string(1:ind_separator-1);  data_string  = string(ind_separator+3:n);
             PARAMETER.Samples_Pressure =  regexp(string(ind_separator+3:n), '\t', 'split');
             
        end
  
    %     % __RESULTS__
%     % Time Delay [msec]:			
%     % Mean [msec]:				
%     % Standard Deviation [msec]:		
%     % CN calc [-]:				
%     % CN Path:				

       if (k == k_results+1) % Time Delay [msec]:
            [m,n] = size(tline); string = tline(1:n);
             % find index of ":" mark
             ind_separator = findstr(string,':');
%              Name = string(1:ind_separator-1); data_string  = string(ind_separator+3:n);
             RESULTS.Box_Temperature = regexp(string(ind_separator+3:n), '\t', 'split');
       end


tline = fgetl(fid);
    
end

fclose(fid);


% %%% DATA %%%%% can be read in one shot
DATA = dlmread(Filename,'\t',[k_data+1 0 k_mss-4 12]); % reads only data vector

end
%     

[Amanullah Khan]

Thank you so much. Will try to understand and use it for my work. Thanks again.

[Mathieu NOE]

hello

so how are you doing ? need further assistance ? (got your PM in my mail box)

[Amanullah Khan]

Hello,

After reading the lines I would like to take the average of all the parameters in the __PARAMETERS_ and__RESULT__ tabs. But there are some values which is taken as zero and those values have to be deleted. After doing this I will have to read a multiple files from input from which I have to extract the same data. and form it in a tab to 3d graphs. I have attached the same program I did in Python. I would like to something exactly the same. Please could you have a look and suggest me.

Thanks

Aman

Answer 2

[Mathieu NOE]

hello

so finally upgraded the code for multiple files processing and doing the min/mean/max computation on several variables

hope this helps you

[Amanullah Khan]

Hello Mr. Mathieu,

I have attached the modification. But what will the possible change in the retrieve_data function file. In the Parameters tab, The Chamber Pressure sometimes comes on the 4 tabs and not on (k==k_parameter +2) as seen on line 74 in retrieve_data.m. I have sent a pm as well.

Regards,

Amanullah

[Mathieu NOE]

Hello Aman

So I have modified the two m functions

In the main function, I have created a list of the parameters you want to process. Only the data of the listed parameters will be processed (mean value) and retrieved.

So the result of the 2 txt files gives at the end :

Surface_table =

2×3 table

Chamber Temperature [°C]: Chamber Pressure [bar]: Ignition Delay [msec]:

_________________________ _______________________ ______________________

{[20.1125]} {[624.5833]} {[6.2719]}

{[35.2658]} {[599.5833]} {[3.0763]}

All the best

[Amanullah Khan]

This is so perfect! Thank you so much Mr Mathieu.

[Mathieu NOE]

Glad it works and helps you !

[Peter Perkins]

Almost perfect.

Surface_table has three variables, all of which are cell arrays that contain only scalar numeric values. That is almost surely not what you want. It's hard to tell from the code where that is happening, but I suspect it has to do with using dlmread - stay away from that if possible, use readtable instead. Barring that, convert those table variables to double using

t.X = vertcat(t.X{:});

[Amanullah Khan]

Dear Mr Mathiei,

Could you let me know what could be the possible error and how to remove it for multiple graphs?

I have attached the Plot_from_Tables matlab file and the input datas.

Regards,

Aman

[Mathieu NOE]

hello Aman

I wished I could help you , but I don't have the Curve Fitting Toolbox.

I cannot run your code

'prepareSurfaceData' requires Curve Fitting Toolbox.

Error in Plot_from_Tables (line 11)

[xData, yData, zData] = prepareSurfaceData(Pressure,Temperature,IDT);

[Amanullah Khan]

Hello Mr. Mathieu,

Ok. Thanks. I guess even I got the tool from the apps in Matlab. First it was not running.

Regards,

Aman