Is cellfun multithreaded or does it do the operation one cell at a time?

Does the cellfun function perform the function on all the cells simultaneously, using multiple cores if available, or does it just do one cell at a time? It seems like the process takes much longer than it should, and I don't think it's using multiple cores.

Is there some way to make cellfun multithreaded? I have the Parallel computing toolbox, although I'm not using it in this code. I'd hate to have to write everything in parfor loops just to get parallel processing.

[Jan]

You hate to parallelize the program? Why? It makes programs faster. I like it.

[Walter Roberson]

Switching to parfor() is, I've read, slower even if there is no pool open.

[Sean de Wolski]

That is true Walter.

[Shinya]

Using parfor just has a larger overhead. If the benefit of parallelization can overcome it, it will still be useful. Please see my answer below. In that implementation, my function that uses parfor is faster if the function spends roughly 100ms.

Answer 1

[Walter Roberson]

cellfun(@disp,num2cell(1:10))

gives a perfect 1 through 10 listing for me. Mind you I don't have a pool opened. It's a simple enough test to make though.

I would not expect cellfun to be multithreaded at the current time.

[Kenneth Eaton]

Although, the documentation for CELLFUN does say this: "The order in which cellfun computes elements of A is not specified and should not be relied upon." So I guess the behavior you see is not guaranteed.

[Matt Tearle]

IANA developer, so this is wild speculation, but I think Walter's observation is guaranteed, because nothing in disp could change any elements of the cell. I suspect that cellfun may take the cell elements in some non-monotonic order behind the scenes, but then compile the results (again, behind the scenes). With something like parfor, OTOH, you see things in the order they actually happened.

Answer 2

[Matt Tearle]

Currently cellfun is not multithreaded.

[Royi Avital]

Moreover it is slower than a loop along the array.

Answer 3

[Shinya]

As others noted, 'cellfun' is single threaded, but you can write a parallel version of 'cellfun' relatively easily.

function result = cellfunp(func, c, varargin)
% Parallel version of cellfun that uses parfor inside

p = inputParser;
addParameter(p, 'UniformOutput', 1, @isscalar);
parse(p, varargin{:});


result = cell(size(c));

parfor i = 1:numel(c)
    result{i} = func(c{i});
end

if p.Results.UniformOutput % uniform
    result = cell2mat(result);
end

This method has relatively large overhead (~0.1s on my computer), but will be useful if 'func' is time consuming and the benefit overcomes this overhead. Make sure that func is a function handle.

[Walter Roberson]

You will, however, have a problem if the function handle does not return any value.

[Andrea]

Hi @Shinya I'm interested in the method you propose, but due to my relative naivety with matlab, I would need a simple worked out example to implement your solution in my script. Can you provide a simple example to explain the use of variable "c"? Thank you!

[Walter Roberson]

c = arrayfun(@(varargin) randi([-9 9]), 1:20, 'uniform', 0);  %just SOME cell

result = cellfunp(@(X) X.^2 - X.^3, c)

Starting parallel pool (parpool) using the 'Processes' profile ...
Connected to the parallel pool (number of workers: 2).

result = 1×20

-180 -48 0 576 810 -180 392 810 -294 0 -294 -48 392 2 810 810 0 -4 36 -648

function result = cellfunp(func, c, varargin)
    % Parallel version of cellfun that uses parfor inside
    
    p = inputParser;
    addParameter(p, 'UniformOutput', 1, @isscalar);
    parse(p, varargin{:});
    
    
    result = cell(size(c));
    
    parfor i = 1:numel(c)
        result{i} = func(c{i});
    end
    
    if p.Results.UniformOutput % uniform
        result = cell2mat(result);
    end
end

[Andrea]

Thanks! Really appreciate the help.