Transparency violation error when using 'parfor'

I am trying to take advantage of multiple processor with the following codes:

mfile_name = run_sims_func();
parfor i=1:length(mfile_name)
    run(mfile_name{i});
end

I understand everything should be known or defined to Matlab to use 'parfor', but I don't know why the cell array of script names is considered undefined or unknown.

I have a function 'run_sims_func()' which creates a script in the same directory of 'run_sims_main.m' based on some other information. The cell array of the script names is returned to the main function as shown below:

mfile_name = run_sims_func();

For example,

mfile_name = {'eric.m';'hong.m'};

Then, I want to run them simultaneously with 'parfor' command as attempted below:

parfor i=1:length(mfile_name)
    run(mfile_name{i});
end

Then I get the subject error. Surely enough, if I just use 'for', then it works just fine.

Why is this and how can I get it work?

Your help will save me ^_^

Thank you in advance,

Eric

Answer 1

[Walter Roberson]

scripts are essentially not compatible with parfor if the scripts create any variables or use any values from the workspace that the parfor is executing in.

If you can, convert the scripts to functions and pass any necessary values into the functions, and instead of using run() use a cell array of function handles.

[Eric Hong]

Thank you. That makes sense. One of the things I found that was causing this problem was 'plot' command. Once I removed it, it worked.

I wonder why? Is 'plot' command also creating a variable? Maybe an object handle is the problem?

Also, I haven't used the function handle. How would you do that with the provided code?

[Walter Roberson]

function h = run_sims_func(SomeParameter)
  funcname = 'Something_appropriate';
  filename = [funcname '.m'];
  fid = fopen(filename, 'wt');
  fprintf(fid, '%s\n', 'Some content');
  fclose(fid);
  h = {str2func(funcname)};  %return cell array of handles
end

and

mfile_handles = run_sims_func(someparameter);

parfor K = 1 : length(mfile_handles)
  mfile_handles{K}();   %execute it
end

[Eric Hong]

Thank you again. At first, the script ran and showed some hope as it said, "Analyzing and transferring files to the workers ...done."

However, quickly after, it crashed with the following error message.

An UndefinedFunction error was thrown on the workers for 'Test_2.m'.  This might be because the file
containing 'Test_2.m' is not accessible on the workers.  Use addAttachedFiles(pool, files) to specify the
required files to be attached.  See the documentation for 'parallel.Pool/addAttachedFiles' for more
details. Caused by: Undefined function or variable 'Test_2.m'.

What does it mean?

Let me know.

Thank you again,

Eric

[Walter Roberson]

function [h, f] = run_sims_func(SomeParameter)
  funcname = 'Something_appropriate';
  filename = [funcname '.m'];
  fid = fopen(filename, 'wt');
  fprintf(fid, '%s\n', 'Some content');
  fclose(fid);
  h = {str2func(funcname)};  %return cell array of handles
  f = {filename};            %return cell array of file names
end

and

[mfile_handles, mfile_names] = run_sims_func(someparameter);

poolobj = gcp;
addAttachedFiles(poolobj, mfile_names);

parfor K = 1 : length(mfile_handles)
  mfile_handles{K}();   %execute it
end

[Eric Hong]

This didn't solve the issue. However, I believe it has everything to do with how my functions are written and I now know how to use the function handle method. I will have to clean up my functions first. Then, I'm sure this method will work.

Thank you,

Eric

[Walter Roberson]

Are you possibly writing them somewhere on your path instead of into your current directory?