Share two variables between workers

Hi all,

I'm trying to share information between workers. I want to share a counter of the loop (parfor) and the elapsed time of the iteration (both of them allow me to see the progress of the script).

I've managed to use the parallel.pool.DataQueue along with afterEach in order to share the counter between workers but I can't seem to make it work with the elapsed time.

My code:

D = parallel.pool.DataQueue;
afterEach(D, @nUpdateWaitbar);
counter = 0;
parfor ...
    time = tic;
    %compute something..  
    elapsedTime = toc(time);
    send(D,counter);
end

function nUpdateWaitbar(~)
    counter = counter + 1;
    %do something with counter - works fine.
    %do somthing with elapsedTime - doesn't work.
end

I've tried to use parallel.pool.PollableDataQueue to share the elapsedTime:

D = parallel.pool.DataQueue;
shareTime = parallel.pool.PollableDataQueue;
afterEach(D, @nUpdateWaitbar);
counter = 0;
parfor ...
    time = tic;
    %compute something..  
    elapsedTime = toc(time);
    send(shareTime,elapsedTime);
    send(D,counter);
end

function nUpdateWaitbar(~)
    elapsedTime = poll(shareTime);
    counter = counter + 1;
    %do something with counter - works fine.
    %do somthing with elapsedTime - doesn't work.
end

But unfortunately it didn't work.

Any ideas?

Thanks in advance!

Answer 1

[Edric Ellis]

I think this should work... I tried a slight modification of your code to make it executable:

function repro
D = parallel.pool.DataQueue;
shareTime = parallel.pool.PollableDataQueue;
afterEach(D, @nUpdateWaitbar);
counter = 0;
parfor idx = 1:10
    time = tic;
    pause(idx/4);
    elapsedTime = toc(time);
    send(shareTime,elapsedTime);
    send(D,counter);
end

function nUpdateWaitbar(~)
elapsedTime = poll(shareTime);
counter = counter + 1;
disp([counter, elapsedTime]);
end
end

And got the following output:

>> repro
    1.0000    0.2506
    2.0000    0.5008
    3.0000    0.7511
    4.0000    1.0013
    5.0000    1.2515
    6.0000    1.5018
    7.0000    1.7520
    8.0000    2.0022
    9.0000    2.5006
   10.0000    2.2525

(This was using R2019a)

[or ohev shalom]

Thanks! I've figured it out now thanks to you.

My main problem is that I received weird times. Forgot to take into account the number of workers for the remaining time calculation >.<