How to efficiently allocate memory using a parfor loop

8 views (last 30 days)
Hello all, I have a quick optimization question.
I'm doing calculations on some very large point cloud data. The calculation I'm doing is
for n=1:size(E_mat,1)
Q_matrix(n,:,:) = sigmaE(n)/2/mass_density(n)*squeeze(E_mat(n,:,:))'*squeeze(E_mat(n,:,:));
end
where size(E_mat) ~70000000,3,24. This code should be super parallelizable but when I use parfor I get a memory issue. I have access to a good compute server with 40 cores and 512Gb of RAM. The current for loop utilizes about 300Gb of RAM but only 1.2% CPU. I'm pretty new to high performance computing but I'm pretty sure the for loop is running single threaded due to the low CPU usage. Is there a simple way to fix this?
Thanks so much for the help!!
  4 Comments

Sign in to comment.

Accepted Answer

Jan
Jan on 29 Jun 2022
Edited: Jan on 29 Jun 2022
ET = permute(E_mat, [2,3,1]);
Q = zeros(size(ET));
parfor n = 1:size(E_mat, 3)
Q(:,:,n) = sigmaE(n) / 2 / mass_density(n) * ET(:, :, n)' * ET(:, :, n);
% Or maybe this is faster:
% tmp = ET(:, :, n);
% Q(:,:,n) = sigmaE(n) / 2 / mass_density(n) * tmp' * tmp;
end
I'm curious: What do you observe?
Du you really mean ctranspose or is ET real? Then .' would be the transposition.
What about using pagemtimes ?
ET = permute(E_mat, [2,3,1]);
Q = pagetimes(ET, 'transpose', ET, 'none');
  5 Comments
tiwwexx
tiwwexx on 30 Jun 2022
That was a by product of my GPU running out of memory, I had to split up the array into a few parts to fit it on the gpu.

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by