Is it possible to call parfeval from within a parfor loop
14 visualizaciones (últimos 30 días)
Mostrar comentarios más antiguos
Jon
el 2 de Feb. de 2023
Respondida: Walter Roberson
el 3 de Feb. de 2023
I have a time consuming analysis task I'm trying to speed up, on a beefy workstation with lots of cores.
Currently, code constists of an outer parfor loop, which let's say is loading an input file, then calling several functions which each do some independant processing, and output a file.
Obviously, the easiest approach is just to parfor the outer loop -- but since there are often only a small # of input files to analyze (say, 3), this doesn't make good use of the cores.
Alternatively, I could parfor the time consuming functions, but since there are small # of these functions, it also ends up using a small # of cores. (The functions could be non-communicating, fire and forget, since they can just output files, results are not used within the loop ).
Ideally, it seems like I'd want to be able to have an outer parfor, and then have each parfor kick off some separate tasks or parfeval's for the work going on inside the loop. However, in Matlab 2021, "Workers cannot submit jobs to a local cluster." , and "Workers cannot execute parfeval or parfevalOnAll."
Is there any way to run parfeval from within parfor? Or any sort of equivalent construct using the lower level job/task controls?
0 comentarios
Respuesta aceptada
Walter Roberson
el 3 de Feb. de 2023
No, that is not possible.
What you can do:
Create a backgroundPool
loop parfeval() reading the files, recording the futures.
afterEach a function that fetches the file content and parfeval() a series of function handles on the file content, one for each of the independent tasks to be done for each file.
You would end up with a bunch of parfeval() futures that might be working on any combination of task and file; as soon as one finishes the next would start, so you should keep your CPUs busy.
0 comentarios
Más respuestas (0)
Ver también
Categorías
Más información sobre Asynchronous Parallel Programming en Help Center y File Exchange.
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!