Combining data columns from many text files into one text file of a specified order

Hello,

I have a bunch (hundreds) of AFM indentation curves in individual text files and I want to be able to take just two of the columns from the data file and then organize them into one text file. All the file names are sequential.

Thank you so much for your help.

Here is an example of the data. I lterally only need the first two columns in the data file.

[Point Spectroscopy Data]

Approach Rate=0.4993 µm/sec

Retreat Rate=0.4993 µm/sec

Z Start=-0.1452 µm

Z End=0.1000 µm

Cycle Hold Time=0.000 sec

½ Cycle Hold Time=0.000 sec

&Z Piezo Linerizer On=yes

Trigger on:TM Deflection >==1.000000 V

Deflection Sensitivity=29.0000 nm/V

Spring Constant=0.0917 N/m

[Series layout]

Points Number=694

Series Number=1

Trace Z Position, µm TM Deflection, V Retrace Z Position, µm TM Deflection, V

[Data]

-0.145200 -0.393064 0.020908 1.002931

-0.144955 -0.375664 0.020664 1.030710

-0.144716 -0.371390 0.020424 1.005678

-0.144476 -0.383296 0.020184 0.974846

-0.144236 -0.381159 0.019944 0.967214

-0.143996 -0.384211 0.019705 0.936992

-0.143757 -0.385738 0.019465 0.951035

-0.143517 -0.380853 0.019225 0.926613

-0.143278 -0.388485 0.018986 0.941877

-0.143038 -0.379022 0.018746 0.893034

-0.142798 -0.396117 0.018506 0.856401

-0.142559 -0.388180 0.018267 0.870444

-0.142319 -0.378717 0.018027 0.874412

-0.142079 -0.366201 0.017787 0.867696

-0.141839 -0.396422 0.017548 0.856401

-0.141600 -0.394285 0.017308 0.837170

-0.141360 -0.380243 0.017068 0.853043

-0.141120 -0.374443 0.016828 0.841138

-0.140881 -0.373527 0.016589 0.796874

-0.140641 -0.360400 0.016349 0.766652

-0.140401 -0.368337 0.016109 0.761158

-0.140162 -0.376885 0.015870 0.699799

-0.139922 -0.372916 0.015630 0.722999

[Stephen23]

"All the file names are sequential."

Depending on how they are named, you might not import the files in the order you expect.

Are the files numbered? If so, do they use sufficient leading zeros?

Answer 1

[Mathieu NOE]

hello

try this

I created some txt files with names : Point Spectroscopy Data 01.txt to Point Spectroscopy Data 03.txt to test the code.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% solution using the structure returned by DIR:
P = cd;
S = dir(fullfile(P, 'Point Spectroscopy Data*.txt'));
NHEADERLINES = 16; 
out = [];

for k = 1:numel(S)
%     S(k).folder
    S(k).name
    F = fullfile(S(k).folder, S(k).name);
    
    % output  : vertical concatenation
    tmp = readmatrix(F,"NumHeaderLines", NHEADERLINES); 
    out = [out; tmp];
  
end

[Orangutan]

Thank you.

[Mathieu NOE]

My pleasure !

[Orangutan]

One issue I should have mentioned is that they are not all the same size. How can we modify it to export individual arrays of the data from each file? Thank you

[Mathieu NOE]

hello

you can simply export each data array as a cell

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% solution using the structure returned by DIR:
P = cd;
S = dir(fullfile(P, 'Point Spectroscopy Data*.txt'));
NHEADERLINES = 16; 
out = [];
for k = 1:numel(S)
%     S(k).folder
    S(k).name
    F = fullfile(S(k).folder, S(k).name);
    
    % output  : vertical concatenation
    out{k} = readmatrix(F,"NumHeaderLines", NHEADERLINES); 
  
end