Overhead when using optimization: parfor loops or UseParallel 'always' for fmincon?

I am currently running an optimization problem using fmincon, and I have access to a cluster. I am unsure about the best way to parallelize my code, and seems I have two options.

First, a bit more detail: the objective function that I feed into fmincon performs some numerical integration, and is thus rather costly to run. In fact, the objective performs this numerical integration for each of about 600 observations. It seems that I might achieve a significant speed increase if I run these loops in parallel using parfor.

However, all the documentation I can currently find recommends using the UseParallel 'always' options to parallelize the computation of the numerical gradient used in the optimization. Currently, my parameter space is relatively small (8-dimensions), but future iterations of the problem will have a 20-30 dimensional parameter space.

It seems that the first option (parfor) actually requires a lot of overhead to run, and this really bogs down performance. The UseParallel option seems to offer marginal improvements over the non-parallelize version of the code, but I want more. Is the overhead parfor just prohibitively costly?

My code takes the form:

[x ...] = fmincon(@zambia_SL,initial,constraints,options,passed parameters)

...

function ML = zambial_SL(initial,passed parameters)

    (par)for i=1:obs %obs = 600 or so

         costly independent executions here

    end

ML = g(computed vectors from above)

Answer 1

[Alan Weiss]

It is difficult to know which method will give you more speedup. You could try both methods on your current, smaller problem--it sounds to me as if you might already have done so.

There are some techniques you can use that might help. If you are passing a large set of data in the "passed parameters", this can slow parallel computing significantly, because of the wasteful communication of these parameters every iteration. You could do better to load the data into every worker, then access the local version and avoid communication overhead.

There are also some techniques related to optimizing simulations or ODEs that might be relevant for your problem. These suggestions are pretty generic, but maybe you will find something of interest.

Sorry if I didn't give you any new information. It is generically a tough problem to run optimizations in parallel efficiently.

Good luck,

Alan Weiss

MATLAB mathematical toolbox documentation