Some thoughts/comments
- I thought that fetchNext should be able to extract values from parfeval as they are produced, but it appears to only extract them once all iterations of the first loop are complete. When I put a breakpoint at the 'fetchNext' line of the code below, it always shows all of the FevalFuture items complete before the breakpoint is reached.
I suspect your code is finishing too quickly (just calling rand) and that by the time you get to the fetch you're done. Try the following. You ought to see most futures are queued.
p = gcp();
tic
for idx = 1:100
f(idx) = parfeval(@pause,0,10);
end
for idx = 1:100
completedIdx = fetchNext(f);
end
toc
- If I include a "disp('par')" and "disp('fetch')" inside each loop then all the "disp('par')" are printed out first and then all the "disp('fetch')" are printed out afterwards.
This part is true. The for-loops are running in serial, so MATLAB can't fetch even the first future until it's submitted the last one in the first for loop.
- Also, on this code using fetchNext is actually slower than fetchOutputs. The fetchNext version takes 73 seconds, whereas the fetchOutputs version at the bottom only takes 31 seconds.
That's true as well, but fetchNext and fetchOutputs serve different purposes. fetchNext allows you to get a result at a time. If you've found what you're looking for, you could cancel your remaining tasks, which can save signicant time. fetchOutputs will fetch all values, which might not be what you want to do. Take for example the following:
for idx = 1:10000
f(idx) = parfeval(@pause,1,10);
end
tic
for idx = 1:1
[completedIdx,value] = fetchNext(f);
end
toc
tic
for idx = 1:1000
f(idx) = parfeval(@pause,1,10);
end
tic
R = fetchOutputs(f);
toc
This is an extreme example, but if you found your answer on the first pass, it'd be 125x faster to call fetchNext then to call fetchOutputs.
