# How to efficiently allocate memory using a parfor loop

8 views (last 30 days)
tiwwexx on 28 Jun 2022
Commented: tiwwexx on 30 Jun 2022
Hello all, I have a quick optimization question.
I'm doing calculations on some very large point cloud data. The calculation I'm doing is
for n=1:size(E_mat,1)
Q_matrix(n,:,:) = sigmaE(n)/2/mass_density(n)*squeeze(E_mat(n,:,:))'*squeeze(E_mat(n,:,:));
end
where size(E_mat) ~70000000,3,24. This code should be super parallelizable but when I use parfor I get a memory issue. I have access to a good compute server with 40 cores and 512Gb of RAM. The current for loop utilizes about 300Gb of RAM but only 1.2% CPU. I'm pretty new to high performance computing but I'm pretty sure the for loop is running single threaded due to the low CPU usage. Is there a simple way to fix this?
Thanks so much for the help!!
tiwwexx on 29 Jun 2022
Thanks for the explaination!

Jan on 29 Jun 2022
Edited: Jan on 29 Jun 2022
ET = permute(E_mat, [2,3,1]);
Q = zeros(size(ET));
parfor n = 1:size(E_mat, 3)
Q(:,:,n) = sigmaE(n) / 2 / mass_density(n) * ET(:, :, n)' * ET(:, :, n);
% Or maybe this is faster:
% tmp = ET(:, :, n);
% Q(:,:,n) = sigmaE(n) / 2 / mass_density(n) * tmp' * tmp;
end
I'm curious: What do you observe?
Du you really mean ctranspose or is ET real? Then .' would be the transposition.
ET = permute(E_mat, [2,3,1]);
Q = pagetimes(ET, 'transpose', ET, 'none');
tiwwexx on 30 Jun 2022
That was a by product of my GPU running out of memory, I had to split up the array into a few parts to fit it on the gpu.

R2021b

### Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!