Parfor loop hangs in R2021b

I have a parfor loop that runs fine in R2021a. The same exact parfor loop in R2021b hangs and I have to CTRL + C to stop it. When I do that, I get the following error:

So the getCompleteIntervals function seems to be the one hanging. Any ideas?

Thanks,

Alex.

[Raymond Norris]

Can you post the parfor code? Are you running a local process pool? threaded pool? Have you tried it with R2022a?

[Alexandru Bitca]

I can post the parfor code, unfortunately it makes little sense on its own as it's part of a much larger bit of software that I've developed. This parfor loop lives inside a function.

F1 = parfevalOnAll(@setupTempdir, 1); % launch process pool and setup temp folders for model code generation
wait(F1)
currDir = fetchOutputs(F1);

permutations = arrangePerm(loop1Value,loop2Value,loop3Value); % generate permutations

parfor ii = 1:length(permutations)
    
    ts = testcase; % testcase here is an input 
    load_system(ts.simulinkModel) % call to load a simulink model
    ts.updateModel % call to update simulink model with data
    
    pwd
    Simulink.fileGenControl('getconfig') % handle to code gen folder
    
    ts.Parameterset = ts.Parameterset.updateParameterValue(loop1Name,permutations{ii}(1)); % function to update variable in model
    if ~isempty(loop2Name)
        ts.Parameterset = ts.Parameterset.updateParameterValue(loop2Name,permutations{ii}(2));
    end
    if ~isempty(loop3Name)
        ts.Parameterset = ts.Parameterset.updateParameterValue(loop3Name,permutations{ii}(3));
    end
    
    ts = ts.run; % run testcase call
    
    switch length(metricNames)
        case 1
            metric1(ii) = extractMetrics(ts,metricNames); % extract run metrics
        case 2
            [metric1(ii),metric2(ii)] = extractMetrics(ts,metricNames);
        case 3
            [metric1(ii),metric2(ii),metric3(ii)] = extractMetrics(ts,metricNames);
        case 4
            [metric1(ii),metric2(ii),metric3(ii),metric4(ii)] = extractMetrics(ts,metricNames);
    end
end

It is running in a local pool which is created before the parfor loop using all of the physical cores of the machine. My machine has 6 physical cores and 32GB RAM.

Unfortunately I do not have access to R2022a.

As mentioned, it works just fine in R2021a. Has the way memory is allocated for the pool changed from R2021a to R2021b?

[Edric Ellis]

From what you've shown here, it's not straightforward to tell what changed to cause the problem. The function getCompleteIntervals is what your desktop MATLAB does while it is waiting for the workers to complete their work, so the fact that it is waiting there is completely expected.

Unfortunately, problems like this can be tricky to diagnose. You could try sprinkling disp statements through the body of your parfor loop. Or, you could contact MathWorks support who can help you gather diagnostic logging information which should help get to the bottom of this.

Answer 1

[Mitch Lautigar]

parfor is a for loop that runs in parallel to your base process. Because of that, there are a bunch of things the issue could be. I've tried to list them down below in the forms of my comments.

1. Parallel Processes: MATLAB is running your parfor loop in a partitioned section of the code. If your parfor loop is taking too long, it is likely to be very memory intensive. If possible, I recommend running your code in a regular for loop with the inputs you expect, and then change the for loop to be a parfor loop.
2. The error you are seeing is completely logical. When you stop the code, MATLAB is stating that it was currently waiting in distributed_execution, and that it was also in runSweep on line 38. I recommend putting a break point before the parfor loop, manually setting "ii", and then copying your code into the command window one line at a time to see if you can find the line it's hanging up on. More than likely, since you are using structures/classes (I noticed the p.function name) you have some point in your parfor loop that is a very memory intensive command and figuring out what this line is will solve your problem (or at the very least, give you a starting point).

Hope this helps! If you post more of your code, i'm sure we (MATLAB forums) can help more.