MATLAB Answers

Parfor number of workers, CPUs, how are they related? What are the limits?

3 views (last 30 days)
Hi,
Using parfor specifying the number of parallel workers, how does it relate to the number of CPUs?
Where M specifies the Maximum number of workers running in parallel, as both a cap, and a limit.
It may seem like a very obvious question and answer: is the number of workers equal to the CPUs available? That is, on a quadcore computer, the maximum number of workers would be 4?
A separate question, I've been advised I can set some settings using this written in somewhere before the parfor coding:
tempdir = getenv('TMPDIR')
c = parcluster
c.JobStorageLocation = tempdir
c.NumWorkers=8
parpool(c.NumWorkers);
Would this specify the pool to 8 workers? In which case the code below for parfor would cap at 8 even if M is specifed as 10? What would happen if M was 4? Would it be capped at 4 or 8?
parfor (i=1:10,10)
...
end
parfor (i=1:10,4)
...
end
And where should theTMPDIR in tempdir = getenv('TMPDIR') be located?
Thanks a lot!

Accepted Answer

Raymond Norris
Raymond Norris on 30 Jun 2021
A quadcore computer should max at 4 workers. We don't suggest throttling it to include hyperthreads. With that said, let's say you have 4 additional HT and you start a pool of 8 workers, perhaps you get even a 10% increase (for example), then maybe it's worth it. You just shouldn't expect an 8x improvement.
Setting the JobStorageLocation is only a suggestion if you are running MATLAB jobs on an HPC cluster and in some sort of rapid batch mode. For example, you write a PBS job script to run MATLAB code that uses a local pool. And then you submit a slew of those at once. In that case, setting the JSL can be helpful (due to a race condition).
The number of workers used is the min(M,size-of-pool). In your example, that would be 8 and 4 respectfully.
  3 Comments

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by