Avoid redundant computation in parfor

Hi all, I have used parfor for parallel computation. But there appears too redundant computation in the parfor. That is, parfor loops perform iteration for each 'ii' when using the command 'parfor ii = 1:n'. For example, the following iteration:

n = numel(seq);
parfor ii = 1:n
  b(ii,:) = seq(indices);
end

But what I want is: if an element in 'seq' vector (says jjth element in 'seq') has been assigned to the matrix 'b', I do not let the code perform the computation for this element again, i.e. when the ii equals to jj. This can be done when using for loop, as:

n = numel(seq);
for ii = 1:n 
  if numel(find(b==seq(ii))) == 0
      b(ii,:) = seq(indices);
  end
end

However, an error occurs as "The PARFOR loop cannot run due to the way variable 'b' is used." when I modified the code as the above.

Because my code is very complex, and can waste lots of time if iterating on every element in 'seq'.

How can I avoid this?

Best Regars.

Jorey.

Answer 1

[Walter Roberson]

You could use spmd and labbroadcast to tell each of the labs which elements have been assigned.

When you say

if an element in 'seq' vector (says jjth element in 'seq') has been assigned to the matrix 'b',

then do you want to go by value (as you do in your "for" loop), or do you want to go by index ? Or is it known that the values in seq are unique?

If you want to go by index or if the values are unique, you can improve performance by using a logical array: e.g.,

if ~any(seqdone(indices))
  b(ii,:) = seq(indices);
  seqdone(indices) = true;
end

except that you need to adjust this for whatever difference there is between "ii" and "indices", and you need to consider what you want done if some of seq(indices) have been processed already but not all of them have.

[Jorey]

Hi Roberson,

Thanks for your reply.

When I try the way you suggested, the code is modified as follows:

n = numel(seq);
assigned_tag = false(n,1);
parfor ii = 1:n
  if ~assigned_tag(ii)
    b(ii,:) = seq(indices);
    assigned_tag(indices) = true;
  end
end

Then two errors occurred for parfor: # Valid indices for 'assigned_tag' are restricted in PARFOR loops. # The PARFOR loop cannot run due to the way variable 'assigned_tag' is used.

These errors still occur when I annotated the initialization of the variable 'assigned_tag'.

So, how can I fixed these errors?

PS: What I want is, if the jjth element in 'seq' has been assigend to the matrix 'b' in previous loops, then the loop just continues to next ii (i.e. jj+1) without performing any computation when ii == jj.

Best Regards.

[Walter Roberson]

What is the relationship between "ii" and "indices" ? And do you possibly want to "break" once a location is found to assign to, or do you want to do that same assignment for all ii that have not already been assigned ?

[Jorey]

Hi Roberson, Thanks for your reply. 'ii' is the index of the 'seq'. So if any of the element in 'seq' has been assigned (indicated in the 'indices'), ignore the computation for this element during the loops. I think this statement is similar with the latter meaning in your reply.

Best Regards.