MATLAB Answers

1

Parallel Computing Toolbox: Underperformance AND Cannot cancel or destroy a job that was not created by this Local cluster.

Asked by Matlab2010 on 18 Nov 2014
Latest activity Edited by Sean de Wolski
on 18 Dec 2014
I am running Win Server 2012 R2 with 2014A with PCT. I have two questions.
Question 1:
I keep getting the below warning,
Warning: Cannot cancel or destroy a job that was not created by this Local cluster.
> In Local.cancelOrDestroyJob at 34
In Local.Local>Local.hDestroyJob at 237
In CJSJobMethods>CJSJobMethods.destroyOneJob at 70
In CJSCommunicatingJob>CJSCommunicatingJob.destroyOneJob at 94
In Job.Job>Job.delete at 1037
In my_mFile at 39
Starting parallel pool (parpool) using the 'local' profile ... connected to 32 workers.
My code looks like,
myCluster = parcluster('local');
delete(myCluster.Jobs);
delete(gcp);
parpool('local',32);
A = length(myFiles);
parfor a = 1: A
%do stuff
end
The reason I included "delete(myCluster.Jobs);" is that without it I get another warning, as below. I should say I only have one MATLAB IDE running at anyone time on my machine.
Warning: Found 1 pre-existing communicating job(s) created by pool that are running. You can use 'delete(myCluster.Jobs)' to remove
all jobs created with profile local. To create 'myCluster' use 'myCluster = parcluster('local')'.
What code do I need to change/include to make sure I do not get any warning?
Question 2:
I have 32 vCPU (vCPU = half a physical core, ie half of a shared core, or one hyperthread), (Intel Xeon E5-2680 v2) on my machine. The task I am trying to run is very demanding (each task takes c. 3mins to run). Hence I want to make sure the machine is maxed out.
In the MATLAB IDE I go to HOME, parallel, manage cluster profiles, edit and change "Number of Workers to start on your local machine" to 32.
I start the job. Task manager shows the total CPU at 40-50%. Going into resource monitor shows the individual matlab.exe taking up 0.2% CPU on average.
Is this right? Normally when I use PCT (on a normal/low spec machine) and don't set myCluster, parpool(local,32) etc) I get CPU useage of 100%.
When I remove the two lines:
myCluster = parcluster('local');
delete(myCluster.Jobs);
and then run the job, I see CPU now averages at >90%. No idea why...
thanks

  2 Comments

I think that warning appears due to PARFOR loops unsuccessfully closed in other passed sessions. But you can rename (or deleted ?) configuration file that saves history of pass parfor evaluations (in my GNU/LINUX> .matlab/local_cluster_jobs/R2014a/matlab_metadata.mat)
oups!
Renaming file gives an error in new parpool sessions, so in my case I save original configuration file with some prefix (orig-matlab_metadata.mat), and when I have this warning, I copy this file as "matlab_metadata.mat" to avoid this problem.
...That's still as a not technical fix, I know, but it's works.

Sign in to comment.

Tags

1 Answer

Answer by Sean de Wolski
on 18 Dec 2014
Edited by Sean de Wolski
on 18 Dec 2014

The only time I have seen that is also when something failed in a previous MATLAB session (hang/out of memory/etc.)
You should be able to clear them out in the job monitor.

  0 Comments

Sign in to comment.