How to utilize all resources assigned by slurm on HPC

I am trying to run a matlab script on a HPC (hyak, University of Washington), the only parallel component is a single parpool loop. Jobs are submitted through slurm, and we are given the option in our slurm script to use a checkpoint partition that allows the jobs to run on multiple nodes.

When I submit my matlab script job, I can see that slurm has assigned multiple nodes. However, the matlab session only shows a single node available. I am using commands `myCluster = parcluster(‘local’)' or `myCluster = parallel.cluster.Slurm' followed by `parpool(myCluster)',

IT has stated repeatedly that "we are not matlab experts" and have been unable to assist.

Any help would be appreciated.

Answer 1

[Parag]

Hi @Larry Pierce,

When submitting MATLAB jobs on an HPC cluster like Hyak using SLURM, it's important to distinguish between MATLAB’s Parallel Computing Toolbox (PCT) and MATLAB Parallel Server (MPS). PCT allows parallel execution only within a single node, even if SLURM allocates multiple nodes. To run parallel tasks across multiple nodes, MATLAB Parallel Server must be installed and configured with a SLURM-integrated cluster profile. If MPS is not available, MATLAB will default to using only the cores of the node where the job starts. In such cases, jobs should be submitted with ‘--nodes=1’ and ‘--cpus-per-task=N’ to fully utilize a single node. For multi-node execution, the “parcluster” call must reference a properly configured SLURM cluster profile, and workers must be launched using “batch” or “createJob”, not just “parpool”.

1. Local Parallel Pool (Single Node Only) - Uses all available cores on a single node via the “local” cluster profile with “parpool”.

myCluster = parcluster('local'); 
parpool(myCluster, myCluster.NumWorkers);  % Use all available local cores 
% Example parallel loop 
parfor i = 1:10 
    fprintf('Running iteration %d on worker %d\n', i, getCurrentTask().ID); 
end 

2. Multi-Node parallelism - Executes code across multiple nodes using MATLAB Parallel Server with batch and a custom ‘SLURM’ cluster profile.

myCluster = parcluster('SlurmProfile');  % Custom profile for Slurm 
j = batch(myCluster, @myFunction, 1, {}, 'Pool', 32);  % Pool size = total workers 
wait(j); 
result = fetchOutputs(j);

Please refer to the following MathWorks documentation for:

• For Getting Started with Parallel Computing Toolbox: https://www.mathworks.com/help/parallel-computing/getting-started-with-parallel-computing-toolbox.html
• Install and Configure MATLAB Parallel Server for SLURM: https://www.mathworks.com/help//releases/R2021a/matlab-parallel-server/install-and-configure-matlab-parallel-server-for-slurm-1.html

Hope this resolves the query!