MATLAB Answers

Why doesn't a parfor loop run properly with a custom object with saveobj and loadobj methods?

18 views (last 30 days)
Kouichi C. Nakamura
Kouichi C. Nakamura on 3 Nov 2017
I'm having an issue with a parfor loop that uses custom objects. The loop runs properly if it was for loop instead of parfor. However, with parfor it issues warning as follows. I think this is strange in that it's not an error despite an index exceeding matrix dimensions. The output of the parfor loop contained empty objects, which later resulted in an error outside of the parfor loop.
Warning: While loading an object of
class 'MarkerChan':
Index exceeds matrix dimensions.
> In parallel.internal.pool.deserialize (line 29)
In distcomp.remoteparfor/getCompleteIntervals (line 257)
In parallel_function>distributed_execution (line 823)
In parallel_function (line 590)
In main_GABA_util>local_prepareC (line 571)
In main_GABA_util.prep (line 48)
Warning: While loading an object of
class 'Record':
Index exceeds matrix dimensions.
> In parallel.internal.pool.deserialize (line 29)
In distcomp.remoteparfor/getCompleteIntervals (line 257)
In parallel_function>distributed_execution (line 823)
In parallel_function (line 590)
In main_GABA_util>local_prepareC (line 571)
In main_GABA_util.prep (line 48)
The same parfor loop runs properly when saveobj and loadobj methods of the custom class are commented out. Related to this finding, I found the following description in the Documentation.
> If you are passing objects into or out of a parfor-loop, the objects must properly facilitate being saved and loaded. https://uk.mathworks.com/help/distcomp/objects-and-handles-in-parfor-loops.html
As far as I know, the saveobj and loadobj methods have been running properly; when tested, the original object and a saved and loaded object were identical.
methods
function s = saveobj(obj)
s.ChanInfo_ = obj.ChanInfo_;
s.Data_ = obj.Data_;
s.Length_ = obj.Length_;
s.MarkerCodes = obj.MarkerCodes;
s.MarkerFilter = obj.MarkerFilter;
s.MarkerName = obj.MarkerName;
s.TextMark = obj.TextMark;
end
end
methods (Static)
function obj = loadobj(s)
obj = MarkerChan;
obj.ChanInfo_ = s.ChanInfo_;
obj.Data_ = s.Data_;
obj.Length_ = s.Length_;
obj.MarkerCodes = s.MarkerCodes;
obj.MarkerFilter = s.MarkerFilter;
obj.MarkerName = s.MarkerName;
obj.TextMark = s.TextMark;
end
end
Can I anyone tell me what's the real problem here?
Thanks, Kouichi

  0 Comments

Sign in to comment.

Answers (2)

Edric Ellis
Edric Ellis on 6 Nov 2017
You can use the undocumented flag 'debug' to a parfor loop to diagnose this sort of problem (normally the loadobj function is invoked on the workers, so you cannot debug into it). For example
parfor (idx = 1:10, 'debug')
% do stuff
end
This forces the parfor machinery to use the serialization mechanism, but runs everything on the client. This enables you to set breakpoints in your loadobj implementation. (Note there's a slight weirdness in using the 'debug' flag - it gets ignored if you run too few iterations in your loop).

  1 Comment

Kouichi C. Nakamura
Kouichi C. Nakamura on 9 Nov 2017
Wow, thank you for the hidden tips. I was very close to give up this issue, but I will investigate further.

Sign in to comment.


Walter Roberson
Walter Roberson on 3 Nov 2017
When I look at https://www.mathworks.com/help/matlab/matlab_oop/understanding-the-save-and-load-process.html I would read it as indicating that loadobj is only called if there were errors in the load process, and in that case what gets passed to loadobj is a struct with the fields that were retrieved from the file. In such a case, the fields might be incomplete -- you should not count on any of them existing (because loadobj is the mechanism for dealing with upgrades and changes to classes.)
If no fields were retrieved from the file, then it is not immediately obvious whether s would be an empty array, or if it would be an empty struct with no fields, or if it would be a non-empty struct with no fields.
The error message you are seeing hints that s might be an empty struct, leading to s.ChanInfo_ possibly being interpreted as s(1).ChanInfo_ at a time when s(1) does not exist.

  5 Comments

Show 2 older comments
Walter Roberson
Walter Roberson on 4 Nov 2017
Right, because if I interpret correctly, loadobj() is called only if something failed in the process.
You are not explicitly loading saved custom objects, but they are being transferred into the workers for the purpose of your calculations, and the way that is handled is for the client to serialize the objects and send them to the worker and the worker deserializes them, effectively loading them.
I suggest in your loadobj check for empty s. You already need to test for fields potentially missing (I figure). I have not yet seen what should be done if the object is just not salvageable.
Edric Ellis
Edric Ellis on 6 Nov 2017
loadobj is always called when loading an object. It's unfortunate that the documentation implies otherwise, but I think it's explicit in the loadobj reference page.

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by