nested for inside parfor, writing to shared variable

I want to do:

parfor ...

count;

for ...

if ...

count += 1

Do I need to worry about a race condition in this situation?

Answer 1

[Edric Ellis]

parfor knows how to handle "reduction" variables like this correctly. (Behind the scenes, each worker process accumulates a partial summation for "count", and then at the end of the loop execution, the partial summations are sent back to the client where the final result is computed.) So, the following loop works as expected:

count = 0;
parfor i = 1:N
    for j = 1:N
        if i < j
            count = count + 1;
        end
    end
end
count

[chris g]

I saw a similar response in another post, I need the count variable local to the iteration though (inside the parfor, outside the nested for). Depending on how the loop is parallel-ized, this may create a race condition. I don't know how the work is broken up amongst the workers to make a determinization whether or not it would create a race condition

Can it handle this

parfor i = 1:N
    count = 0;
    for j = 1:N
        if % there is an error
            count = count + 1;
        end
        if count > 3
            % yell at user
        end
    end
end

[Edric Ellis]

Well, did you try? parfor loop iterations are executed in a non-deterministic order, so in cases where the loop terminates with an error, you might see some non-deterministic behaviour. For example, this loop:

parfor i = 1:3
    if i == 1
        error('Error 1');
    elseif i == 2
        error('Error 2');
    else
        error('Another error')
    end
end

could result in any of the errors being thrown to the user. However, parfor puts constraints on the way that you can assign results which means that non-error cases are generally deterministic.

[chris g]

That would not work. I take it, since you keep suggesting other methods, each worker does not handle an iteration of the parfor loop, and there will be a race condition?

Bonus question:

When I turn my for loop into a parfor, it seems to make the workspace static even before it even gets to the loop. Does that seem right or am I missing something?

"attempt to add ___ to a static workspace ... Error ... "

[Edric Ellis]

You said "That would not work" - I'm not sure which "that" you are referring to. It would be really helpful if you could post some code that you think has a race condition. None of the code you have posted so far does so far as I can tell.

parfor puts language restrictions in place that essentially prevent race conditions. Loop iterations are required to be independent - this is enforced by language analysis of the result variables. If you really want to, you can make order-dependent parfor loops by hiding state in persistent variables in functions that you call from within parfor. Also, as I explained, error cases might leak some minor non-determinism.

I don't know if this helps: each iteration of a parfor loop is performed by a single worker. In fact, the parfor loop is divided up into "subranges" consisting of several iterations of the loop. Normally this is handled automatically by the parfor machinery, but in rare cases it can be useful to give extra direction to parfor about how you'd like things to be divided up. This is documented here.

[chris g]

Perfect, thank you!

I apologize, I must have poorly phrase the original question. I was trying to ask if the pseudocode I posted had a race condition in it. I couldn't find any information how the workload was actually divided up among the workers, so I couldn't tell if there was a race condition. I probably should have just asked that in the first place.