Memory leak when interfacing between Matlab and python

Hi. I have found a problem when I try to make communication between Matlab and python (using optimisation in matlab). I have built a model in Matlab and tried to use Scipy to optimise the parameters in the model. Therefore, I wrote an objective function in python and use "matlab.engine" to interface between the two. However, I have found that the simulation time in each iteration increases when I call Matlab engine.

With the code below, the simulation time keeps increasing after finishing eacg interation.

eng = matlab.engine.start_matlab()

def opt_pf(x):

y = eng.matlab_model(excel_path, x, nargout=0)

return y

gc.collect()

### Run optimisation

from scipy.optimize import minimize

res = minimize(opt_pf, x0, method='SLSQP', constraints=[eq_cons], options={'ftol': 1e-09, 'disp': True}, bounds=bounds)

I think that the optimizer canno't clear the cache from matlab.engine in each iteration.

[Rakshit Kothari]

I am observing the same issue with MATLAB R2022b with Python running on Ubuntu. Any help would be appreciated! Likewise, I'll keep this thread posted if I find a solution or workaround.

[Walter]

We are as well. Same matlab version.

[Rakshit Kothari]

For what it's worth, I find that initializing MATLAB via Python as such `eng = matlab.engine.start_matlab('-nodisplay -singleCompThread -nojvm')` helps reduce MATLAB's memory consumption but the leak still exists. Hopefully you can finish your job with the saved memory and avoid waiting for an actual solution.

[Rakshit Kothari]

I can also confirm that this issue does not occur in R2021b.

[hyuntae.kim]

I am using a python open module from matlab as a blockproc function.

MATLAB version is R2021b R2022b.

1. I confirmed that the MATLAB process memory and analysis time increased as image analysis was repeated using blockproc().

2. Delete and reload the Python environment with the code below

>> terminate(pyenv); py. list;

3. I checked the task manager.

4. In the MATLAB R2022b version, the memory increased compared to when the Python open module was first called.

5. I have to quit and restart MATLAB R2022b to get the memory back.

[涵]

1. I have the same problem when using R2022b python API in Ubuntu20.02.
2. I solved this problem by save the actual parameter which is fed to eng.my_function(actual parameter) in .mat format, and then load them in my_functionI.m.
3. I'm pretty sure the problem is caused by trying to transmit parameter from matlab python API(eng.function(actual parameter)) to matlab engine. So try to avoide transmit parameter or transmit it like 2.
4. Additionally, if you just transmit a short string, there are also no memory leaks.

[Shisui Uchiha]

I have this problem with matlab R2023a in Ubuntu22.04 too...

[Shisui Uchiha]

https://bastibe.de/2015-10-29-matlab-engine-leaks.html

[Øyvind]

I have experienced similar issues (Matlab R2023b on Windows).

For me, it seems like the problem is mostly if running in ExecutionMode="OutOfProcess", while "InProcess" is better.

With OutOfProcess execution mode, I see that there is a process called matlabpyhost which ends up using tons of memory (as seen in Task Manager).

So to do this, use something like:

pyenv(ExecutionMode="OutOfProcess")

(Obviously, the InProcess vs OutOfProcess also affects other things, see https://se.mathworks.com/help/matlab/ref/pyenv.html )

Answer 1

[Donato]

I am facing the same problem, we really need some support here.

Answer 2

[Clark]

Me too! HUGE leaks... Sometimes swelling to 40 GB, when expecting maybe only 2GB

Answer 3

[Neil]

I encountered the same issue in MATLAB2023b. The attachments in the following old bug report looks helpful, but it is for a much much older version of MATLAB.

https://www.mathworks.com/support/bugreports/details/1319969