Profiler Paradox

In an attempt to rotate multiple 3x3 matrices I use two approaches. The first is by running a for loop in which I rotate each job separately. The other approach is where setup a sparse block diagonal matrix and rotate once. In order to find out which is most time-efficient I place tic-toc's around each function and I also use Profiler.

Like this:

tic
function1
toc

tic 
function2
toc

The paradox: When using Profiler the function2 is faster, when not using Profiler the function1 is faster. Significantly different (I run several times to get average)

I would expect that using Profiler would yield an overall extended runtime, because it measures everything going on, but I certainly wouldn't expect the two functions switch place in speed.

Answer 1

[Oleg Komarov]

The difference is due to the JIT accelerator that kicks in when using the profiler. Same is if you save the script which calculates the running times and call it from cmd window.

Some useful links:

• undocumented-profiler-options
• accel_matlab.pdf

Oleg

Answer 2

[Walter Roberson]

Some releases have inexplicable longer times for a later call when profiling. See a specific case and some test results here

Answer 3

[Derek O'Connor]

In certain cases the profiler is worse than useless -- it is misleading. As you will have noticed, the profiler adds a lot of overhead time. This is noise (not wanted) and it can swamp the execution times of the functions you are interested in.

For me, the only reliable numbers that the profiler produces are the statement counts.

See pages 16 and 17 of my notes which discusses an example similar to yours.

http://www.scribd.com/doc/47228910/RPGLab-A-Matlab-package-for-Random-Permutation-Generation

Regards,

Derek O'Connor

Answer 4

[Matt Tearle]

Profiler isn't really the right tool for (absolute) timing a chunk of code - as has been noted, it adds overhead, plus you have to worry about whether JITing is happening or not, etc etc etc. Use the profiler to find bottlenecks/inefficiencies within a piece of code. Use tic and toc (averaged over multiple runs - +10 Cleve points for knowing to do that!) to compare two functions.