Cell array Filtering when using readlines

Hello, after I have read an array from a piece of equipment

writeline(s,command);  pause(0.02);
data='N/A';
nb=s.NumBytesAvailable
data={}; ct=0; 
while nb>0
    ct=ct+1;
    d = readline(s)
    data{ct,1}=d;
end
data

I get this:

data =
884×1 cell array
{["       190    190.03    190.06    190.09    190.12    190.15    190.18    190.21"]}
{["    190.24    190.27     190.3    190.33    190.36      -999      -999      -999"]}
{["      -999      -999      -999      -999      -999      -999      -999      -999"]}
{["      -999      -999      -999      -999      -999      -999      -999      -999"]}
{["      -999      -999      -999      -999      -999      -999      -999      -999"]}
...
{["    190.24    190.27     190.3    190.33    190.36      -999      -999      -999"]}
{["      -999      -999      -999      -999      -999      -999"                    ]}
...
{["-1.38866E-009      -999      -999      -999      -999      -999      -999"       ]}
{["      -999      -999      -999"                                                  ]}
{[":"

I want to remove any line that doesn't have just -999 as all the values, so I tried modifing to this:

writeline(s,command);  pause(0.02);
data='N/A';
nb=s.NumBytesAvailable
data={}; ct=0; idx=0;      
while nb>0
    ct=ct+1;
    d = readline(s)               
    if d~=["      -999      -999      -999      -999      -999      -999      -999      -999"]    
       idx=idx+1;                
       data{idx,1}=d;
    end     
    nb=s.NumBytesAvailable

end  
data

and this nearly gives me what I want

data =
  11×1 cell array
    {["       190    190.03    190.06    190.09    190.12    190.15    190.18    190.21"]}
    {["    190.24    190.27     190.3    190.33    190.36      -999      -999      -999"]}
    {["      -999      -999      -999      -999      -999      -999"                    ]}
    {[" -0.999999        -1        -1        -1  -1.00001  -1.00001  -1.00001  -1.00001"]}
    {["  -1.00002  -1.00002  -1.00002  -1.00003  -1.00003      -999      -999      -999"]}
    {["      -999      -999      -999      -999      -999      -999"                    ]}
    {["-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009"  ]}
    {["-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009"  ]}
    {["-1.38866E-009      -999      -999      -999      -999      -999      -999"       ]}
    {["      -999      -999      -999"                                                  ]}
    {[":"                                                                               ]}

How can I also get rid of these line entries:

{["      -999      -999      -999      -999      -999      -999"                    ]}
{["      -999      -999      -999"                                                  ]}
{[":"                                                                               ]}

Thanks for any help

Answer 1

[Stephen23]

Do not store lots of scalar strings in a cell array, doing so makes processing them harder:

https://www.mathworks.com/help/matlab/matlab_prog/frequently-asked-questions-about-string-arrays.html#mw_ea39fa70-6d47-4204-80b4-9abb58bbce77

S = ["       190    190.03    190.06    190.09    190.12    190.15    190.18    190.21"; "    190.24    190.27     190.3    190.33    190.36      -999      -999      -999"; "      -999      -999      -999      -999      -999      -999"; " -0.999999        -1        -1        -1  -1.00001  -1.00001  -1.00001  -1.00001"; "  -1.00002  -1.00002  -1.00002  -1.00003  -1.00003      -999      -999      -999"; "      -999      -999      -999      -999      -999      -999"; "-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009"; "-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009"; "-1.38866E-009      -999      -999      -999      -999      -999      -999"; "      -999      -999      -999"; ":"]

S = 11x1 string array
    "       190    190.03    190.06    190.09    190.12    190.15    190.18    190.21"
    "    190.24    190.27     190.3    190.33    190.36      -999      -999      -999"
    "      -999      -999      -999      -999      -999      -999"
    " -0.999999        -1        -1        -1  -1.00001  -1.00001  -1.00001  -1.00001"
    "  -1.00002  -1.00002  -1.00002  -1.00003  -1.00003      -999      -999      -999"
    "      -999      -999      -999      -999      -999      -999"
    "-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009"
    "-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009"
    "-1.38866E-009      -999      -999      -999      -999      -999      -999"
    "      -999      -999      -999"
    ":"

X = cellfun(@isempty,regexp(S,'^(\s+-999)+|:$','once'));
Z = S(X)

Z = 7x1 string array
    "       190    190.03    190.06    190.09    190.12    190.15    190.18    190.21"
    "    190.24    190.27     190.3    190.33    190.36      -999      -999      -999"
    " -0.999999        -1        -1        -1  -1.00001  -1.00001  -1.00001  -1.00001"
    "  -1.00002  -1.00002  -1.00002  -1.00003  -1.00003      -999      -999      -999"
    "-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009"
    "-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009"
    "-1.38866E-009      -999      -999      -999      -999      -999      -999"

[Jason]

Thankyou Stephen, could you explain each of the regexp parts please?

I've also just realised that your S is different to my 'd'. I only get one line at a time

[Stephen23]

"could you explain each of the regexp parts please?"

'^(\s+-999)+|:$'
%^               match start of text
% (       )+     group match one or more times
%  \s+           match one more more whitespace characters
%     -999       match literal "-999"
%           |    or
%            :   match literal colon
%             $  match end of text

[Stephen23]

"I've also just realised that your S is different to my 'd'. I only get one line at a time"

I assumed that you wanted to filter the entire array after the loop. You can also apply the REGEXP to one single line of text, in which case you might want to use e.g. IF rather than indexing:

d = "       190    190.03    190.06    190.09    190.12    190.15    190.18    190.21";
isempty(regexp(d,'^(\s+-999)+|:$','once'))

ans = logical
   1

d = "      -999      -999      -999      -999      -999      -999";
isempty(regexp(d,'^(\s+-999)+|:$','once'))

ans = logical
   0

[Jason]

Yes thats done it, thankyou!

Answer 2

[Star Strider]

Try something like this —

% data =
% 884×1 cell array
data = { ...
{["       190    190.03    190.06    190.09    190.12    190.15    190.18    190.21"]}
{["    190.24    190.27     190.3    190.33    190.36      -999      -999      -999"]}
{["      -999      -999      -999      -999      -999      -999"                    ]}
{[" -0.999999        -1        -1        -1  -1.00001  -1.00001  -1.00001  -1.00001"]}
{["  -1.00002  -1.00002  -1.00002  -1.00003  -1.00003      -999      -999      -999"]}
{["      -999      -999      -999      -999      -999      -999"                    ]}
{["-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009"  ]}
{["-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009"  ]}
{["-1.38866E-009      -999      -999      -999      -999      -999      -999"       ]}
{["      -999      -999      -999"                                                  ]}
{[":"                                                                               ]}};

% Q = cellfun(@(x) x, data, Unif=0)                                 % See What 'cellfun' Returns
% Q{1}
% Q{end}

db = cellfun(@(x)sscanf(x{:},"%f"), data, Unif=0);                  % Use 'sscanf' To Convert The String Values To Numeric
Lvc = cellfun(@(x) ~all(x == -999), db, Unif=0);                    % Test For 'all' Values To Be -999
Lv = cell2mat(Lvc);                                                 % Convert Cell Logical Array To Logical Array

data{Lv,:}                                                          % Return Edited Cell Array

ans = 1x1 cell array
    {["       190    190.03    190.06    190.09    190.12    190.15    190.18    190.21"]}

ans = 1x1 cell array
    {["    190.24    190.27     190.3    190.33    190.36      -999      -999      -999"]}

ans = 1x1 cell array
    {[" -0.999999        -1        -1        -1  -1.00001  -1.00001  -1.00001  -1.00001"]}

ans = 1x1 cell array
    {["  -1.00002  -1.00002  -1.00002  -1.00003  -1.00003      -999      -999      -999"]}

ans = 1x1 cell array
    {["-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009"]}

ans = 1x1 cell array
    {["-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009-1.38866E-009"]}

ans = 1x1 cell array
    {["-1.38866E-009      -999      -999      -999      -999      -999      -999"]}

.

[Jason]

Thankyou

[Star Strider]

My pleasure!