Issues with parfor, strsplit and MATLAB path

Hi

(MATLAB 9.2.0.556344 (R2017a) on Windows 10)

I am testing out the parfor function, and I run into some unexpected behaviour:

1) It seems like the order in MATLAB path changes during parfor

2) The MATLAB function strsplit seems to not work as expected in parfor

(Using parfor in the examples below is of course meaningless. But the strsplit is only a small part of a larger code which is supposed to run in parallel).

Example of #1:

I use the function library of Peter J Acklam, which had a strsplit before MATLAB introduced this function. His function takes the separator as first argument and string as second (reversed compared to MATLAB's strsplit). I am using other of his functions, so I have him added to my MATLAB path, but with lower priority than MATLAB functions. When I run this code:

a = 'hello_world';
disp('Strsplit before parfor:');
disp(strsplit(a, '_'));
disp(' ');
disp('Strsplit during parfor:');
parfor (n = 1 : 1, 4)
    disp(strsplit('_', a));
end
disp(' ');
disp('Strsplit after parfor:');
disp(strsplit(a, '_'));

I get the following output

Strsplit before parfor:
    'hello'    'world'

Strsplit during parfor:
** Using strsplit to Peter J. Acklam **
    'hello'    'world'

Strsplit after parfor:
    'hello'    'world'

Note that I added a comment inside Peter's strsplit ( Using strsplit to Peter J. Acklam **), and I am also forced to change the order of input arguments to make the code run.

Example of #2:

To avoid #1 from happening for strsplit, I can just rename Peter's strsplit to e.g. xstrsplit. But when I do that and update my code, MATLAB's strsplit seems to fail in parfor. After renaming, I run this code:

a = 'hello_world';
disp('Strsplit before parfor:');
disp(strsplit(a, '_'));
disp(' ');
disp('Strsplit during parfor:');
parfor (n = 1 : 1, 4)
    disp(strsplit(a, '_'));
end
disp(' ');
disp('Strsplit after parfor:');
disp(strsplit(a, '_'));

and I get this output

Strsplit before parfor:
    'hello'    'world'

Strsplit during parfor:
    'hello_world'

Strsplit after parfor:
    'hello'    'world'

Any input on what I am doing wrong is welcome.

Answer 1

[Erling Hugo]

Contacting MathWorks Technical Support provided the following insight:

1) There is currently a bug with the Parallel Toolbox when it comes to the path, which the developers are working on fixing for a future release.

2) Even though I renamed the 3rd party strsplit and my output indicated that it was running MATLAB's strsplit, there seemed to be some type of cache effect. Calling restoredefaultpath and then running the same code resulted in a correct splitting inside the parfor in example 2.

Answer 2

[Edric Ellis]

You can see which version of strsplit the workers are seeing by executing:

fetchOutputs(parfeval(@which, 1, 'strsplit'))

It seems as though the path synchronisation for parallel pools doesn't work out quite right if you have a function that is shadowed by a function that is part of MATLAB. One workaround is to add this to your path after opening the parallel pool (using addpath ... -end).

[Erling Hugo]

Thank you for the reply, though I am not able to solve the problems with this.

The fetchOutputs function you suggest confirms my findings regarding which strsplit function is used.

I am not sure I follow you on the suggested workaround. I have the following in my startup.m file (already):

    addpath(genpath(fullfile('..', 'com', 'rptas')), '-end');
    addpath(genpath(fullfile('..', 'com', 'UiB')), '-end');
    addpath(genpath(fullfile('..', 'com')), '-end');
    addpath(fullfile(pwd,'gfx'));

If I create the following test function

myPool = parpool();
fetchOutputs(parfeval(@which, 1, 'strsplit'))
startup();
fetchOutputs(parfeval(@which, 1, 'strsplit'))
which strsplit

It gives the same output from the two fetchOutputs calls, which is different from the one from the which strsplit call. But maybe this is not what you meant?

Also, even though working around the path issue in problem 1 (illustrated in example 1), problem 2 (illustrated in example 2), would still remain: strsplit seems to fail splitting strings while in parfor.