parfor loop with continue gives incorrect results

Consider the following code:

N = 1000;

failed = false( 1, N );
values = cell( 1, N );

n_failed = 0;

parfor idx = 1:N
  try
    if ( rand() > 0.7 )
      n_failed = n_failed + 1;
      error( '' );
    end
  catch err
    failed(idx) = true;
    continue;
  end

    values{idx} = rand( 1e3, 1 );
end

fprintf( 'N failed 1: %d\n\n', sum(failed) );
fprintf( 'N failed 2: %d\n', n_failed );

If I run this on my machine (macOS 10.12, r2017a), `sum(failed)` is 0, while `n_failed` is, as expected, ~300. What am I missing here? I don't see anything in the documentation about `continue` not being supported in a parfor loop?

[Matt J]

while `n_failed` is, as expected, ~300

How can you have any expectations at all about sum(failed) or n_failed, when their values are driven by the randomization in

if ( rand() > 0.7 )

[Walter Roberson]

Statistically approximately 300 would be most common.

Question: have you tried nnz() to cross check sum()?

[Steve Chang]

Using nnz instead of sum gives the same result. Also, regarding the randomization -- removing the if statement gives the same behavior (i.e., sum(failed) is 0, while n_failed is 1000).

I guess there's a bug in how continue is handled in a try / catch context, or else it is not meant to be used in this way in a parfor loop.

[Matt J]

The problem doesn't seem to have anything to do with 'continue'. Even when the continue is commented out, I still obtain sum(failed)=0. It might have more to do with try...catch.

Answer 1

[Matt J]

Don't pass an empty string '' to error() if you want the catch block to be triggered. An empty string apparently does not result in an error being thrown.

In other words, this works fine:

N = 1000;
failed = false( 1, N );
n_failed = 0;
parfor i = 1:N
  try
    if ( rand() > 0.7 )
      n_failed = n_failed + 1;
      error('an error');
    end
  catch 
    failed(i) = true;  
    continue
  end

end
fprintf( 'N failed 1: %d\n\n', sum(failed) );
fprintf( 'N failed 2: %d\n', n_failed );

[Steve Chang]

The code above works, but I'm experiencing a different error with my original code. With the randomization, the assignment to `values` gives an error: 'In an assignment A(:) = B, the number of elements in A and B must be the same.' Removing the if statement gives an undefined function or variable error for `values`.

[Matt J]

Bizarre. Well, I don't understand the problem, but I do find that removing the continue fixes it. I also find that moving the assignment to values up several lines also fixes it:

   N = 1000;
  failed = false( 1, N );
  values = cell( 1, N );
  n_failed = 0;

parfor idx = 1:N

     values{idx} = rand( 1e3, 1 );   

    try
      if ( rand() > 0.7 )
        n_failed = n_failed + 1;
        error( 'A' );
      end
    catch err
      failed(idx) = true;
      continue;
    end

end
fprintf( 'N failed 1: %d\n\n', sum(failed) );
fprintf( 'N failed 2: %d\n', n_failed );

[Steven Lord]

Don't pass an empty string '' to error() if you want the catch block to be triggered. An empty string apparently does not result in an error being thrown.

That is correct. The last item in the Tips section on the documentation page for the error function states that if all the inputs are empty, error does not throw an error.

Historically, error was often used with functions like nargchk that would return '' if the number of inputs was in the allowed range and an error message if not. Now, as the warning box at the top of the documentation page states, you should use narginchk (and/or nargoutchk) instead. Those will directly throw the error, so you don't need to wrap them in error calls.

The code above works, but I'm experiencing a different error with my original code.

That suggests that your original code is doing something different than the code you posted in Answers, and the differences between those two sections of code is relevant. Post your original code, or simplify the original code down to the minimal working example that reproduces the exact error you're seeing, and we may be able to offer guidance relevant to that original code.

[Matt J]

That suggests that your original code is doing something different than the code you posted in Answers

@Steven Lord, The posted code does reproduce the problem, when a non-empty string is passed to error().

N = 1000;
failed = false( 1, N );
values = cell( 1, N );
n_failed = 0;

parfor idx = 1:N

    try
      if ( rand() > 0.7 )
        n_failed = n_failed + 1;
        error( 'A' );
      end
    catch err
      failed(idx) = true;
      continue;
    end

     values{idx} = rand( 1e3, 1 );   

end
fprintf( 'N failed 1: %d\n\n', sum(failed) );
fprintf( 'N failed 2: %d\n', n_failed );

In R2018a, the above gives,

Error using test>(parfor consume)
Unable to perform assignment because the left and right sides have a different number
of elements.

Error in test (line 9)
parfor idx = 1:N

[Steve Chang]

Yes -- or consider a simplified example:

function example()

N = 1;
values = cell( N, 1 );

parfor i = 1:N
  try
    assert( false );
  catch err
    warning( err.message );
    continue;
  end

   values{i} = 1;
end
end

If I call this function, I get an error: Output argument "values" (and maybe others) not assigned during call to "example>F%1". If I extract the function body and run it at the command prompt I get an undefined function or variable error for "values".

It seems like the issue is specifically related to continue being placed in a try/catch block; replacing the `assert( false )` with a `continue` statement, and leaving the catch block empty, results in the same behavior.

[Walter Roberson]

continue is going to proceed to the next parfor iteration -- without having executed the assignment to values{i} that is after the continue.

[Steve Chang]

The assignment code can't be reached, but I don't think the function should throw an error, unless I'm missing something. `values` should remain a cell array of empty arrays after the loop.

[Edric Ellis]

There's an existing problem in the parfor machinery that causes the values assignment to fail. The problem relates to the combination of try / catch and the assignment to values. You can trick the parfor machinery into operating correctly by changing how it analyses values. The following should work:

N = 1000;
failed = false( 1, N );
values = cell( 1, N );
n_failed = 0;
parfor idx = 1:N
    % Dummy reference to "values(idx)":
    if false
        values(idx);
    end
    try
        if ( rand() > 0.7 )
            n_failed = n_failed + 1;
            assert(false);
        end
    catch err
        failed(idx) = true;
        continue;
    end
    values{idx} = rand( 1e3, 1 );
end
fprintf( 'N failed 1: %d\n\n', sum(failed) );
fprintf( 'N failed 2: %d\n', n_failed );

[Steve Chang]

Good to know -- thanks for the response!