Share PollableDataQueue between Matlab sessions

I'm trying to send data between two sessions using data queue. Here is how I tried to do that:

1. First Matlab instance creates PollableDataQueue and saves it in matfile
2. Second instance reads it and sends data object to the first one

However such approach results in an error when I send a message:

Struct contents reference from a non-struct array object.
Error in parallel.internal.pool.AbstractDataQueue/send (line 135)
                if obj.Session.isSessionRunning
Error in QueueManager/send_request (line 33)
            request_queue.send(task);
Error in TaskSender/get_latest_max (line 15)
            obj.queue_manager.send_request(task);

Is there a way to achieve such communication between Matlab sessions?

[Tomasz Wyrowinski]

I also get a warning when loading queue from matfile:

Warning: While loading an object of class 'parallel.pool.PollableDataQueue':
To send and receive messages there must be a pool.

Answer 1

[Thomas Falch]

DataQueue and PollableDataQueue are meant for sending data between the client and worker, or between workers in the same parallel pool. In other words, it is meant for sending data between MATLAB sessions that run at the same time.

It can not be used to send data from one MATLAB session to another you run later. To do this, you can simply save the data itself to a .mat file directly, and then load this .mat file in a MATLAB session you open later.

Answer 2

[Raymond Norris]

@Tomasz Wyrowinski are there other reasons to run a second MATLAB session? Or is it just to "offload" work while the first MATLAB session is running? If so in R2021b we introduced backgroundPool for running code in the background pool, using parfeval. There you could use a dataqueue. Would this suffice?

[Tomasz Wyrowinski]

@Raymond Norris @Thomas Falch If I remember correctly, the project I had worked on required me to build a software service capable of interacting with some continuously running calculations in MATLAB. To support that functionality, I tried to start an additional MATLAB session that would use a data queue to send configuration updates for the calculations and also receive some data in response. If I'm not mistaken, I ended up building the service in Python, starting another MATLAB engine instance, and somehow establishing a connection between the engines—most likely by following the documentation here.