Storing data and Matlab Production Server

Hi there,

I'm currently looking into the possibility of running MATLAB functions from a webserver, so I looked into MATLAB Production Server. What I basically want it to do is two things.

- Periodically (every say 15 minutes), it runs a function that simulates data and does calculations to it (takes about 10 minutes in total) and produces a +- 200 MB file.

- Upon user request, MATLAB functions should be called that act on the simulated data (function takes less than a second).

.

The first function takes too long to be called on user request. Therefore, period calls to the function will make sure that data is always available to the user.

A possibility is to store the data to an SQL database, but I think the storing and retrieving of the data would take too long. I am wondering if it would be possible to either:

- Store the data in a .mat file locally on the MPS, so that upon the second function call, the data can be loaded, and it can be acted on.

- Constantly keep the data in memory (using persistent or global variables for example). Then the first function can constantly loop, and at certain time points redo the simulations, and update the data.

.

I've searched a lot on the MATLAB website, but I could not find a clear answer to this question. Any help is appreciated!

Best regards,

Robert

Answer 1

[Titus Edelhofer]

Hi Robert,

first of all, MATLAB Production Server usually assumes the algorithms to be "state-less", which means, that subsequent calls are totally independent. When you run your MPS with more than one worker (you can use up to 24 workers, i.e., MATLAB instances), then this becomes even more clear, because you can't assume that the second call will go to the same worker than the call before.

Nevertheless it's a valid use case, and there is a chapter in the documentation that talks about this. The one line that outlines this is the following:

Ensure Access to Data Caches

If your function relies on cached replies to previous requests, for instance, ensure your deployed system and application has access to that cache outside of the MATLAB environment.

So, as you already thought of, you need to store the data outside of MPS. In your use case I would suggest a .mat file as well. To be on the safe side I would suggest the following procedure:

• Before saving the data in the "long" computation, you remove the dataAvailable.txt. Then call save. Then add the dataAvailable.txt again.
• Your fast computation, when loading the data, does the following:

while ~exist('dataAvailable.txt', 'file')
pause(1);
end
load('nameOfFile.mat');

You could do the same the other way around: add a file "someoneReads.txt". While this exists, the "long" process should not write the file.

This way you can prevent someone trying to read while it is written.

Titus

[Robert Vullings]

Thank you for the reply Titus. It is much clearer for me now, especially with the workers. Keeping the files in memory is not an option then, but I will look into storing things locally in a mat file.

Good suggestion on the text-file creation BTW!