Correct fetchNext index after clearing FevalFuture object
4 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
I am using parfeval to run code asynchronously on workers. The running time varies a lot between workers depending on some parameters that in the following code I represent with 'm'.
When I collect the results with fetchNext, I then process them with idx, in particular retrieving m(idx). At the same time, I need to clear the last retrieved FevalFuture object to save memory.
In Parfeval - Memory consumption piling up - Clear output data? it is suggest to simply assign [ ] to the FevalFuture object. However, it seems to me that doing this mix up the indices idx of the objects, which I need to further process the data. See for instance this mve
N = 10;
m = 1:10;
for idx = N:-1:1
f(idx) = parfeval(@(m,N) m*rand(N), 1, m(idx), N);
end
total = 0;
for idx = 1:N
[idx, data] = fetchNext(f);
idx
f(idx) = []; % This line here
% process data with idx, m(idx) ..
end
If you comment the line indicated the code will work fine, but if I clear f(idx) the just-used idx could be reused and I would process the data with the wrong m(idx).
How is it possible to get the correct index idx with which f(idx) was called in the fist loop?
Thanks in advantage
0 comentarios
Respuestas (1)
Edric Ellis
el 15 de Oct. de 2019
One way you can deal with this is by making the same modification to m as you proceed. Something like this perhaps:
N = 10;
m = 1:10;
for idx = N:-1:1
f(idx) = parfeval(@(m,N) m*rand(N), 1, m(idx), N);
end
for idx = 1:N
[fidx, data] = fetchNext(f);
% Get the value out of "m" corresponding to the completed future
mval = m(fidx);
% Delete corresponding elements of both "f" and "m"
f(fidx) = [];
m(fidx) = [];
% Work with the particular value of "m" and the result
disp([mval, sum(data(:))]);
end
0 comentarios
Ver también
Categorías
Más información sobre Asynchronous Parallel Programming en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!