textscan different number of floating digits

Hello Matlab Community,

I have a dataset with 'Loggers_3340_837584_20.5m_TXT' format.

For example:

'Loggers_3340_838653_20.55m_TXT

'Loggers_3340_200584_30.0m_TXT

'Loggers_3340_636400_23.5m_TXT

'Loggers_3340_268584_12.5m_TXT

Bold parts are changing in each file.

I am using below script to read files in loop. My problem is changing decimal digits in depth (xx.xm or xx.xxm) . I put only the reading part of the script to get your opinions.

If I use '%g\n', the files ending with one digit zero are read as 30m instead of 30.0m.

But if I use '%0.1f' or '%0.2f', the output is whether two or one decimal digits which does not work in my case.

DOm{iii}=readloggerstxt(['Loggers_','3340','-',num2str(idm3(iii),'%6d'),'_',num2str(idm4(iii),'%g\n'),'m.TXT'])

What can be an alternative way to read varying decimal parts?

Thank you in advance!

[Stephen23]

You should use READTABLE() to get the correct variable types, e.g.:

fnm = 'Loggers_3340-106208_33.7m.txt';
obj =  detectImportOptions(fnm, 'filetype','text', 'delimiter',',', ...
    'VariableNamingRule','preserve', 'VariableNamesLine',8, 'VariableUnitsLine',9);
obj = setvartype(obj, 'Unix Timestamp','uint64');
tbl = readtable(fnm, obj)

tbl = 14×8 table
    Unix Timestamp      UTC_Date_&_Time      Mitteleuropäische Zeit    Battery    Temperature    Dissolved Oxygen    Dissolved Oxygen Saturation      Q  
    ______________    ___________________    ______________________    _______    ___________    ________________    ___________________________    _____

      1632844800      2021-09-28 16:00:00     2021-09-28 18:00:00       3.56         7.939             11.01                   98.236               0.993
      1632848400      2021-09-28 17:00:00     2021-09-28 19:00:00       3.55         5.235            11.655                    97.16               0.992
      1632852000      2021-09-28 18:00:00     2021-09-28 20:00:00       3.55          4.12            11.947                   96.747               0.992
      1632855600      2021-09-28 19:00:00     2021-09-28 21:00:00       3.55         3.363            12.171                   96.605               0.991
      1632859200      2021-09-28 20:00:00     2021-09-28 22:00:00       3.55         2.678            12.384                   96.506                0.99
      1632862800      2021-09-28 21:00:00     2021-09-28 23:00:00       3.54           2.3            12.513                   96.517                0.99
      1632866400      2021-09-28 22:00:00     2021-09-29 00:00:00       3.55         1.813            12.664                   96.391                0.99
      1632870000      2021-09-28 23:00:00     2021-09-29 01:00:00       3.54         1.621            12.716                   96.277               0.989
      1632873600      2021-09-29 00:00:00     2021-09-29 02:00:00       3.54         1.525            12.734                   96.159               0.989
      1632877200      2021-09-29 01:00:00     2021-09-29 03:00:00       3.54         1.396            12.797                   96.291               0.989
      1632880800      2021-09-29 02:00:00     2021-09-29 04:00:00       3.54         1.041            12.909                   96.181               0.989
      1632884400      2021-09-29 03:00:00     2021-09-29 05:00:00       3.54        -0.142             13.22                   95.272               0.987
      1632888000      2021-09-29 04:00:00     2021-09-29 06:00:00       3.54        -0.131            13.266                   95.633               0.987
      1632891600      2021-09-29 05:00:00     2021-09-29 07:00:00       3.54         0.145            13.215                   96.015               0.987

Answer 1

[Stephen23]

"...but I need to order them according to the last part which is 30.0m 31.65m 33.5m"

So why not just sort the filenames? That is very easy to do:

P = '.'; % absolute or relative path to where the files are
S = dir(fullfile(P,'Loggers*.txt'));
{S.name} % wrong order

ans = 1×3 cell array
    {'Loggers_3340-106208_33.7m.txt'}    {'Loggers_3340-267560_30.0m.txt'}    {'Loggers_3340-270176_31.65m.txt'}

C = regexp({S.name},'\d+\.?\d*(?=m)','once','match')

C = 1×3 cell array
    {'33.7'}    {'30.0'}    {'31.65'}

[~,X] = sort(str2double(C));
S = S(X);
{S.name} % right order

ans = 1×3 cell array
    {'Loggers_3340-267560_30.0m.txt'}    {'Loggers_3340-270176_31.65m.txt'}    {'Loggers_3340-106208_33.7m.txt'}

[Cakil]

Thank you, it is working perfectly. I stuck on one solution but this one is much more easier.

Answer 2

[Image Analyst]

Did you try using dir?

fileList = dir('Loggers*.txt');
allFileNames = {fileList.name}; % Get all filenames into a cell array.
for iii = 1 : numel(allFileNames)
    % Read results into the two arrays.
    DOm{iii} = readloggerstxt(allFileNames{iii});
    readloggerstxtDOm{iii} = readloggerstxt(allFileNames{iii});
    % Now, here comes the calculations
end

[Cakil]

Thank you Image Analyst.

Using DIR is working to read all my files but I aim to collect them according to a specific order. So I wanted to use below script to collect my data according to the order I want.

idm3=[267560 270176 106208];
idm4=[30.0 31.65 33.7];
for iii=1:3
    DOm{iii}=readloggerstxtDOm{iii}=readloggerstxt(['Loggers_','3340','-',num2str(idm3(iii),'%6d'),'_',num2str(idm4(iii),'%g\n'),'m.TXT'])
   %here comes the calculations
end

The numbers on the datasets are random numbers coming from each logger but I need to order them according to the last part which is 30.0m 31.65m 33.5m etc. So thought creating idm3 and idm4 will help me to read them accordingly. This is why I used:

DOm{iii}=readloggerstxtDOm{iii}=readloggerstxt(['Loggers_','3340','-',num2str(idm3(iii),'%6d'),'_',num2str(idm4(iii),'%g\n'),'m.TXT'])

It is working well for every file except the one ends with 30.0m. Because it is read as 30m when I used '%g\n'.

[Image Analyst]

I am totally baffled. I just can't believe that works. For example, this simplified version throws an error:

readloggerstxtDOm = cell(1);
DOm = cell(1)
iii = 1;
% Call it with two equals signs.
DOm{iii}=readloggerstxtDOm{iii}=readloggerstxt('blah')
% Define a simplified function
function output = readloggerstxt(str)
output = 1;

Not sure what you want but I would have done one of these below. However it looks like you say Stephen's solution works.

idm3=[267560 270176 106208];
idm4=[30.0 31.65 33.7];
% Process the files in this specific, special order.  idm3 and idm4 are synced up.
for k = 1: numel(idm3)
    % Construct filename.  Might want %6.6d if you want leading zeros to appear
    filename = sprintf('Loggers_3340-%6d_%gm.TXT', idm3(k), idm4(k));
    fprintf('Processing %s\n', filename);
    if isfile(filename)
        % Process it.  Can't believe the line below will work though.
        % 		DOm{k}=readloggerstxtDOm{k}=readloggerstxt(])
        %here comes the calculations
    else
        fprintf('    but %s does not exist!\n', filename);
    end
end

fprintf('\n=================================================\n')
% Or maybe you want all combinations like this:
% Process the files in this specific, special order.
for k1 = 1: numel(idm3)
    for k2 = 1: numel(idm4)
        % Construct filename.  Might want %6.6d if you want leading zeros to appear
        filename = sprintf('Loggers_3340-%6d_%gm.TXT', idm3(k1), idm4(k2));
        fprintf('Processing %s\n', filename);
        if isfile(filename)
            % Process it.  Can't believe the line below will work though.
            % 		DOm{k}=readloggerstxtDOm{k}=readloggerstxt(])
            %here comes the calculations
        else
            fprintf('    but %s does not exist!\n', filename);
        end
    end
end

[Cakil]

Thank you Image Analyst. I think this can be an alternative solution. But the problem with the dataset ends 30.0m still continues.

I have tried the above script and I can collect the dataset according to the order I want. However first dataset (ends with 30.0m) is empty because of the digit after zero. Once I change the name of dataset as .xxxxxxx_30m it is working. (I can not change the file name but changes just to see the result).

[Image Analyst]

Then you can define them as strings instead of numbers and there won't be any problem.

idm3={'267560', '270176', '106208'};
idm4={'30.0', '31.65', '33.7'};
% Process the files in this specific, special order.  idm3 and idm4 are synced up.
for k = 1: numel(idm3)
    % Construct filename.  Might want %6.6d if you want leading zeros to appear
    filename = sprintf('Loggers_3340-%s_%sm.TXT', idm3{k}, idm4{k});
    fprintf('Processing %s\n', filename);
    if isfile(filename)
        % Process it.  Can't believe the line below will work though.
        % 		DOm{k}=readloggerstxtDOm{k}=readloggerstxt(])
        %here comes the calculations
    else
        fprintf('    but %s does not exist!\n', filename);
    end
end

Results:

Processing Loggers_3340-267560_30.0m.TXT

Processing Loggers_3340-270176_31.65m.TXT

Processing Loggers_3340-106208_33.7m.TXT

[Cakil]

The double equal part is mistakenly written. Sorry for the confusion. It was supposed be ;

DOm{k}=readloggerstxt(['Loggers_','3340','-',num2str(idm3(k),'%6d'),'_',num2str(idm4(k),'%g\n'),'m.TXT'])

I will be removing the num2str part as I will be using the strings.

I understand the logic and it is supposed to work but I guess i need to change my reader file as well. Becuase I am receiving the error below when I swicthed to strings. Maybe I should combine it with DIR().

%Error in readloggerstxt (line 6)
%fid=fopen(fname,'r');

Thank you for your support.

[Image Analyst]

OK, good. I always think it's easier to read if you use sprintf() to construct the filename instead of the bracket and num2str way.

With both the way you made the filename, and the way I did, both ways sent a string into readloggerstxt() so if it's not working, you'll have to figure out why readloggerstxt() does not like a string.

And glad you finally understand what I was saying about the double = giving a syntax error.

You can only "Accept" one answer but you can "Vote" for as many as you want. If I helped you, can you click the thumbs up Vote icon. Accepting or Voting for an answer will award the answerer with "reputation points" for their efforts in helping you. They'd appreciate it. Thanks in advance. 🙂