Threading: Load data while processing some other data - possible somehow?

Hello,

I have a tool that loads images from a large set of images into memory and then applies some processing. Both parts (loading and processing) take some time. But one part (loading) is mainly using the hard disk, while the other part (processing) is mainly using the CPU. Is there a way to do this in parallel to save time? E.g. while image_1 is processed, image_2 is already loaded into memory? Is this only possible with the parallel computing toolbox or is there a different approach possible?

Thank you!!

Answer 1

[Walter Roberson]

Parallel Computing Toolbox is the only supported approach when loading files from disk. In theory there are potential unsupported approaches involving Java threads or mex files or calling C++ libraries.

One challenge when using parfor or spmd is that they load into other processes and you have to transfer data between processes. However a small number of releases ago an additional facility was added to use threads for parfor; I do not know what the implications are for transfer of data.