fast delete of rows in a table

hi

I've got a table with 120k rows, and I need to delete some rows if there is a specific condition.

I wrote this:

for i=1:(size(db,1)-1)
     if db.INTERVENTION(i)==1 && isequal(db.SETTLEMENTDATE(i-1),db.SETTLEMENTDATE(i)) && isequal(db.DUID(i-1),db.DUID(i))
            db(i-1,:)=[];
     end
end

This code takes a long time to execute.

Is there a faster way?

thanks!

Answer 1

[Image Analyst]

Try this:

numRows = size(db,1)-1;
rowsToDelete = false(numRows, 1);
for k = 2 : numRows  % Has to start at 2, right?  Since you're using k-1.  Don't use i, the imaginary variable, as a loop index.
     if db.INTERVENTION(k)==1 && isequal(db.SETTLEMENTDATE(k-1),db.SETTLEMENTDATE(k)) && isequal(db.DUID(k-1),db.DUID(k))
            rowsToDelete(k) = true;
     end
end
db(rowsToDelete, :) = []; % Remove these rows.
% Or equivalently
%db = db(~rowsToDelete, :); % Extract everything BUT the rows to delete.

[Francesco Giuseppe Fornari]

Much faster, thanks a lot!

(For future readers: note that I was needing to delete the k-1 rows ;)

[Image Analyst]

I think the reason being is that when you removed a single row from the table, it had to rebuild the table each time. If you give it a vector, it has to rebuild the table only once.

[Francesco Giuseppe Fornari]

yes, it makes a lot of sense ;)

Answer 2

[darova]

Write indices and delete outside the for loop

ind = logical(db*0);
for i=1:(size(db,1)-1)
     if db.INTERVENTION(i)==1 && isequal(db.SETTLEMENTDATE(i-1),db.SETTLEMENTDATE(i)) && isequal(db.DUID(i-1),db.DUID(i))
            ind(i,:) = true;
     end
end
db(ind) = [];

[Francesco Giuseppe Fornari]

seems ok, but I don't get the inizialization of ind. Line

ind = logical(db*0);

gives "Undefined operator '*' for input arguments of type 'table'"

Did I got wrong?

Thanks anyway!

[darova]

See Image Analyst answer below. It's correct