coder.perfCompare

Compare execution times of MATLAB code and code generated using multiple configuration objects

Since R2024b

Syntax

t = coder.perfCompare(fcnName,numOutputs,runtimeArgs)

t = coder.perfCompare(fcnName,numOutputs,runtimeArgs,coderConfigs)

t = coder.perfCompare(___,Name=Value)

Description

t = coder.perfCompare(fcnName,numOutputs,runtimeArgs) compares the execution times of the MATLAB^® functions specified by fcnName with the generated MEX code. Use runtimeArgs to specify the run-time input arguments used to compare the performance, and numOutputs to specify the number of output arguments.

• By default, coder.perfCompare uses a MEX configuration object with IntegrityChecks and ResponsivenessChecks properties set to false.

• coder.perfCompare compares the performance for multiple runs and returns the median of execution times.

• coder.perfCompare function uses internal heuristics to determine the number of runs.

• coder.perfCompare excludes the timing overhead incurred by the data transfer between MATLAB and generated code execution.

t = coder.perfCompare(fcnName,numOutputs,runtimeArgs,coderConfigs) compares the execution times of MATLAB code and the code generated using configuration objects specified by coderConfigs. For build types lib and dll, generated code is executed using software-in-the-loop (SIL) or processor-in-the-loop (PIL) simulation, which requires an Embedded Coder^® license.

t = coder.perfCompare(___,Name=Value) specifies additional options using name-value arguments.

Examples

Compare Execution Times of MATLAB and Generated MEX Functions

Compare the execution times for the MATLAB® function integralExpLn and its generated MEX function.

Define the MATLAB® function integralExpLn.

function out = integralExpLn(xmin, xmax)
    f = @(x)x.^5.*exp(-x).*sin(x);
    out = integral(f, xmin, xmax, 'RelTol',1e-10);
end

Measure and compare the execution times of integralExpLn and its generated MEX function.

t = coder.perfCompare("integralExpLn",1,{0,Inf})

==== Running (integralExpLn, MATLAB) ====
- Running MATLAB script.
TimingResult with 775 Runtime Sample(s)

Statistical Overview:
   mean = 6.46e-04 s    max = 3.77e-02 s     sd = 1.42e-03 s
 median = 4.77e-04 s    min = 4.12e-04 s   90th = 8.28e-04 s

==== Running (integralExpLn, Codegen Mex) ====
- Generating code and building MEX.
- Running MEX.
TimingResult with 8485 Runtime Sample(s)

Statistical Overview:
   mean = 5.89e-05 s    max = 3.19e-03 s     sd = 5.31e-05 s
 median = 5.68e-05 s    min = 5.45e-05 s   90th = 5.82e-05 s

t=1×2 table
                        MATLAB                             Codegen Mex                      
                     _____________    ______________________________________________________

                     Runtime (sec)    Runtime (sec)    Speedup: MATLAB / Codegen Mex (times)
                     _____________    _____________    _____________________________________
                                                                                            
    integralExpLn      0.000477        5.6758e-05                     8.4041                

Compare Execution Times of MATLAB Function and Two Generated MEX Functions

Compare the execution time of the MATLAB® function linearInterp with the execution times of two MEX functions generated using separate code configuration objects.

Define the MATLAB function linearInterp.

function out = linearInterp(x,v,xq)
    out = interp1(x,v,xq,"linear");
end

Define input arguments and code generation configurations.

x = 0:pi/1000:2*pi;
v = sin(x);
xq = 0:pi/2000:2*pi;

cfgFastMex = coder.config('mex');
cfgFastMex.IntegrityChecks = false;
cfgFastMex.ResponsivenessChecks = false;

cfgSIMDMex = coder.config('mex');
cfgSIMDMex.IntegrityChecks = false;
cfgSIMDMex.ResponsivenessChecks = false;
cfgSIMDMex.SIMDAcceleration = 'Full';

coder.perfCompare("linearInterp",1,{x,v,xq},{cfgFastMex,cfgSIMDMex})

==== Running (linearInterp, MATLAB) ====
- Running MATLAB script.
TimingResult with 2878 Runtime Sample(s)

Statistical Overview:
   mean = 1.74e-04 s    max = 1.37e-02 s     sd = 3.91e-04 s
 median = 1.08e-04 s    min = 8.60e-05 s   90th = 3.32e-04 s

==== Running (linearInterp, Coder Config 1) ====
- Generating code and building MEX.
- Running MEX.
TimingResult with 3233 Runtime Sample(s)

Statistical Overview:
   mean = 1.55e-04 s    max = 7.70e-03 s     sd = 2.32e-04 s
 median = 1.24e-04 s    min = 1.16e-04 s   90th = 1.50e-04 s

==== Running (linearInterp, Coder Config 2) ====
- Generating code and building MEX.
- Running MEX.
TimingResult with 3090 Runtime Sample(s)

Statistical Overview:
   mean = 1.62e-04 s    max = 5.84e-03 s     sd = 2.70e-04 s
 median = 1.31e-04 s    min = 1.26e-04 s   90th = 1.44e-04 s

                       MATLAB                             Coder Config 1                                               Coder Config 2                      
                    _____________    _________________________________________________________    _________________________________________________________
                    Runtime (sec)    Runtime (sec)    Speedup: MATLAB / Coder Config 1 (times)    Runtime (sec)    Speedup: MATLAB / Coder Config 2 (times)
                    _____________    _____________    ________________________________________    _____________    ________________________________________
    linearInterp      0.000108        0.00012422                      0.86943                      0.00013105                      0.82409                 

Compare Execution Times of MEX Functions Using Different Code Generation Configurations

Compare the execution times of the MEX functions generated from the MATLAB® functions sum1, sum2, and sum3, each with two different code generation configurations.

Define MATLAB® functions sum1, sum2, and sum3.

function out = sum1(in)
    out = sum(in,2);
end

function out = sum2(in)
    [nOut, nSum] = size(in);
    out = coder.nullcopy(zeros(nOut,1));
    for ii = 1: nOut
        outLocal = 0;
        for jj = 1:nSum
            outLocal = outLocal + in(ii,jj);
        end
    end
    out(ii) = outLocal;
end

function out = sum3(in)
    [nOut, nSum] = size(in);
    out = zeros(nOut,1);
    for ii = 1:nSum
        for jj = 1:nOut
            out(jj) = out(jj) + in(jj, ii);
        end
    end
end

Define code generation configurations.

cfg1 = coder.config('mex');
cfg1.IntegrityChecks = false;
cfg1.ResponsivenessChecks = false;

cfg2 = coder.config('mex');
cfg2.IntegrityChecks = false;
cfg2.ResponsivenessChecks = false;
cfg2.EnableAutoParallelization = true;

Define inputs, and measure and compare execution times for the six generated MEX functions (two for each sum function).

 
rng(42);
x = rand(1e4, 1e4);
 
rSum = coder.perfCompare({'sum1','sum2','sum3'},1,{x},{cfg1,cfg2}, ...
                ConfigNames={'Default','Autopar'},CompareWithMATLAB=false)

==== Running (sum1, Default) ====
- Generating code and building MEX.
- Running MEX.
TimingResult with 10 Runtime Sample(s)

Statistical Overview:
   mean = 9.71e-02 s    max = 1.03e-01 s     sd = 3.33e-03 s
 median = 9.67e-02 s    min = 9.29e-02 s   90th = 1.02e-01 s

==== Running (sum1, Autopar) ====
- Generating code and building MEX.
- Running MEX.
TimingResult with 10 Runtime Sample(s)

Statistical Overview:
   mean = 1.02e-01 s    max = 1.05e-01 s     sd = 2.44e-03 s
 median = 1.03e-01 s    min = 9.85e-02 s   90th = 1.05e-01 s

==== Running (sum2, Default) ====
- Generating code and building MEX.
- Running MEX.
TimingResult with 10 Runtime Sample(s)

Statistical Overview:
   mean = 1.91e-01 s    max = 2.12e-01 s     sd = 1.08e-02 s
 median = 1.87e-01 s    min = 1.77e-01 s   90th = 2.07e-01 s

==== Running (sum2, Autopar) ====
- Generating code and building MEX.
- Running MEX.
TimingResult with 10 Runtime Sample(s)

Statistical Overview:
   mean = 2.25e-01 s    max = 2.66e-01 s     sd = 2.66e-02 s
 median = 2.23e-01 s    min = 1.86e-01 s   90th = 2.64e-01 s

==== Running (sum3, Default) ====
- Generating code and building MEX.
- Running MEX.
TimingResult with 10 Runtime Sample(s)

Statistical Overview:
   mean = 9.97e-02 s    max = 1.07e-01 s     sd = 3.89e-03 s
 median = 9.89e-02 s    min = 9.38e-02 s   90th = 1.06e-01 s

==== Running (sum3, Autopar) ====
- Generating code and building MEX.
- Running MEX.
TimingResult with 10 Runtime Sample(s)

Statistical Overview:
   mean = 9.80e-02 s    max = 1.01e-01 s     sd = 2.52e-03 s
 median = 9.81e-02 s    min = 9.45e-02 s   90th = 1.01e-01 s

rSum=3×2 table
               Default                             Autopar                      
            _____________    ___________________________________________________

            Runtime (sec)    Runtime (sec)    Speedup: Default / Autopar (times)
            _____________    _____________    __________________________________
                                                                                
    sum1      0.096709          0.10266                    0.94201              
    sum2       0.18715          0.22341                    0.83768              
    sum3       0.09892         0.098096                     1.0084              

==== Running (sum1, Default) ====
Perf Compare INFO: Generating code and building MEX.
Perf Compare INFO: Running MEX.
TimingResult with 10 Runtime Sample(s)

Statistical Overview:
   mean = 6.53e-02 s    max = 6.70e-02 s     sd = 1.00e-03 s
 median = 6.54e-02 s    min = 6.40e-02 s   90th = 6.67e-02 s

==== Running (sum1, Autopar) ====
Perf Compare INFO: Generating code and building MEX.
Perf Compare INFO: Running MEX.
TimingResult with 12 Runtime Sample(s)

Statistical Overview:
   mean = 4.46e-02 s    max = 4.93e-02 s     sd = 2.70e-03 s
 median = 4.36e-02 s    min = 4.15e-02 s   90th = 4.84e-02 s

==== Running (sum2, Default) ====
Perf Compare INFO: Generating code and building MEX.
Perf Compare INFO: Running MEX.
TimingResult with 10 Runtime Sample(s)

Statistical Overview:
   mean = 5.40e-01 s    max = 5.56e-01 s     sd = 9.38e-03 s
 median = 5.40e-01 s    min = 5.27e-01 s   90th = 5.53e-01 s

==== Running (sum2, Autopar) ====
Perf Compare INFO: Generating code and building MEX.
Perf Compare INFO: Running MEX.
TimingResult with 10 Runtime Sample(s)

Statistical Overview:
   mean = 8.43e-02 s    max = 9.70e-02 s     sd = 6.77e-03 s
 median = 8.40e-02 s    min = 7.44e-02 s   90th = 9.49e-02 s

==== Running (sum3, Default) ====
Perf Compare INFO: Generating code and building MEX.
Perf Compare INFO: Running MEX.
TimingResult with 12 Runtime Sample(s)

Statistical Overview:
   mean = 4.36e-02 s    max = 4.55e-02 s     sd = 1.08e-03 s
 median = 4.37e-02 s    min = 4.14e-02 s   90th = 4.50e-02 s

==== Running (sum3, Autopar) ====
Perf Compare INFO: Generating code and building MEX.
Perf Compare INFO: Running MEX.
TimingResult with 12 Runtime Sample(s)

Statistical Overview:
   mean = 4.26e-02 s    max = 4.47e-02 s     sd = 1.16e-03 s
 median = 4.23e-02 s    min = 4.09e-02 s   90th = 4.42e-02 s


rSum =

  3×2 table

               Default                     Autopar               
            _____________    ____________________________________

            Runtime (sec)    Runtime (sec)    Speedup wrt Default
            _____________    _____________    ___________________
                                                                 
    sum1       0.06535         0.043552             1.5005       
    sum2        0.5395         0.083972             6.4248       
    sum3      0.043654         0.042331             1.0313       

Input Arguments

fcnName — Names of entry-point functions

character vector | cell array of character vectors or strings | string array

Names of entry-point functions for code generation and MATLAB execution, specified as a character vector, cell array of character vector or strings, or string array. When fcnName specifies multiple entry-point functions, they must all have the same number of input and output arguments, and the same input argument data types.

numOutputs — Number of outputs

nonnegative integer

Number of outputs the functions return, specified as a nonnegative integer.

runtimeArgs — Run-time arguments

cell array

Run-time input arguments to fcnName, specified as a cell array.

coderConfigs — Code generation configuration objects

cell array of coder.CodeConfig objects | cell array of coder.MexCodeConfig objects | cell array of coder.EmbeddedCodeConfig objects

Code generation configurations objects. To create these objects, use coder.config.

coder.config("exe") is not supported with coder.perfCompare.

Name-Value Arguments

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Example: t = coder.perfCompare("sum1",1,{x},{cfg1},ConfigNames={"Mex"},CompareWithMATLAB=false);

ConfigNames — Names of output table columns

cell array of character vectors | cell array of strings

Names of the output table columns, specified as cell array of character vectors or cell array of strings.

Example: t = coder.perfCompare({'sum1','sum2'},1,{x},{cfg1,cfg2},ConfigNames={"config1","config2"});

CompareWithMATLAB — Option to compare execution time of generated code with MATLAB

true (default) | false

Option to compare execution time of generated code with MATLAB, specified as one of the values in this table.

Value	Description
true(default)	Compare the execution time of the MATLAB function with the different versions of the generated code.
false	Compare the execution times of the different versions of the generated code with each other.

Example: t = coder.perfCompare('sum1',1,{x},CompareWithMATLAB=false);

Output Arguments

t — Execution times

table

Execution times in seconds, returned as a table. The table also shows execution time speedup of each function relative to the first column. If you specify a single configuration and multiple functions, coder.perfCompare calculates the speedup relative to the run time of the first function. The table shows the medians of run times for each entry.

Tips

• To achieve consistent results, make sure no other compute-intensive processes are running on the machine where the MATLAB or the generated function is timed.

Version History

Introduced in R2024b