Profiling built-in functions

Built-in functions (e.g., isempty, length, find) are not monitored or displayed in the profiling summary by default. They are also not linkable to a detailed profiling report in the detailed report of any user function. So, if we have a code line such as the following, we cannot know how much time is spent in length, find as opposed to being spent in userFunction:

numResults = length(find(userFunction()));

However, we can tell the profile to monitor and display built-in functions using the undocumented optional -detail builtin switch of the profile function:

profile on –detail builtin
profile('on','-detail','builtin')   % an equivalent alternative

Alternately, we can use the undocumented built-in function callstats directly (as I already explained, the profile function is simply a convenient wrapper function to callstats):

callstats('level', 2);  % 1=mmex, 2=builtin

The built-in functions will now be reported in the profiling summary and detailed reports, and will be linkable within the code lines of the detailed report.
Note: old MATLAB releases also supported the ‘operator’ detail level, which provided profiling information about built-in operators such as +. However, this level of detail is no longer supported in the latest MATLAB releases.
The -detail switch can also be used with other profile switches. For example:

>> profile('on','-detail','builtin','-timestamp'); surf(peaks); profile('off')

To return to the standard profiling level (‘mmex’), run one of the following:

profile on –detail mmex
profile('on','-detail','mmex')   % an equivalent alternative
callstats('level', 1);           % an equivalent alternative

Profiling overhead removal

The -remove_overhead switch checks the profiler’s own overhead and knows to remove this overhead from all relevant timing values. This is normally off by default, and returns profData.overhead=0. When turned on, it stores the computed profiling overhead in profData.overhead:

profile -remove_overhead on
profile('-remove_overhead','on')        % equivalent alternative
callstats('remove_sample_overhead',1);  % equivalent alternative

We need to make a small fix to the profview function (%matlabroot%/toolbox/matlab/codetools/profview.m), otherwise the profiling report will croak on an error. The fix is simple: replace the following code segment (lines #351-355 in the profview.m file of R2012a)

if profileInfo.Overhead==0
    s{end+1} = sprintf(['<p>', getString(message('MATLAB:profiler:SelfTime1st')) ' ']);
else
    s{end+1} = sprintf(['<p>', getString(message('MATLAB:profiler:SelfTime2nd', profileInfo.Overhead))]);
end

with this (changed line highlighted – note the extra num2str):

if profileInfo.Overhead==0
    s{end+1} = sprintf(['<p>', getString(message('MATLAB:profiler:SelfTime1st')) ' ']);
else
    s{end+1} = sprintf(['<p>', getString(message('MATLAB:profiler:SelfTime2nd', num2str(profileInfo.Overhead)))]);
end

Turning the overhead switch on causes the message at the bottom of the profiling summary report to change, from something that says:

Self time is the time spent in a function excluding the time spent in its child functions. Self time also includes overhead resulting from the process of profiling.

To this (the numeric value is the computed overhead, also reported in profData.overhead):

Self time is the time spent in a function excluding both the time spent in its child functions and most of the overhead resulting from the process of profiling.
In the present run, self time excludes 0.80601 seconds of profiling overhead. The amount of remaining overhead reflected in self time cannot be determined.

A similar customization needs to be done to fix an error arising from the use of the undocumented profile -timer none option. I don’t really see great value in running a Profiler without timing information, so I’ll skip this part here. If anyone has an idea what this could be used for, I’d be happy to hear.

[As-yet] unsolved mysteries

To conclude my series on undocumented profiling options, a couple of unsolved mysteries:

hardware profiling

The profview.m function has several references to hardware performance counter data, which, if available as fields in profData.FunctionTable, are displayed as separate columns to the left of the time column field in the code section, and also enabled for the sorting and highlighting drop-downs in the detailed profiling report. Separate monitored hardware events are stored in separate fields named profData.FunctionTable.HWevent (see the getHwFields() sub-function on lines 1557-1564). The mystery: how can we tell the Profiler to start hardware monitoring in a way that would add these HW fields to the collected profData.FunctionTable? Lines 1652-1653 give a hint:

hw_events = callstats('hw_events');
match = strcmpi(['hw_' str],hw_events);

Unfortunately, at least on my system, callstats('hw_events') returns an empty cell array:

>> hw_events = callstats('hw_events')
hw_events =
   Empty cell array: 1-by-0

So if anyone has an idea how to use these HW events, I’d be happy to hear – please leave a comment below.

parallel profiling

Another mystery relates to the question of how to profile parallel Matlab code. The underlying Java object of the Profiler tool (com.mathworks.mde.profiler.Profiler.getInstance) contains the following tell-tale methods:

I would have expected the parallel-enabled Profiler to be started using com.mathworks.mde.profiler.Profiler.getInstanceWithParallelOpts.invoke(). This does indeed start the Profiler, but I can’t see any difference between its output and that of the normal Profiler, even when I run parallel (parfor) code. It definitely does not look like output from the Parallel Computing Toolbox (PCT)’s mpiprofile function.
Perhaps these methods are used by mpiprofile internally? It could make sense for both profile and mpiprofile to share the same underlying profiler tool. On the other hand, if this were indeed the case then why would there be a reason for a separate mpiprofile function? Wouldn’t the standard profile function be sufficient?
Anyway, if anyone has an idea how to use these methods in the standard profiler, perhaps to profile parallel applications without the need for pmode, I’d be happy to hear – please leave a comment below.

Conclusion

The Matlab Profiler has quite a few important features, which for some unknown reason have not been made public. Matlab’s Profiler has remained stagnant since Matlab 7 was released last decade. In fact, some functionality, like the JIT information feature, was actually removed. Hopefully, future Matlab releases will improve the Profiler, by making these undocumented features officially supported, as well as by adding other important functionality.
This concludes my series on undocumented profiling features in Matlab. If anyone knows any other Profiler tricks, please share them in a comment below.

The post Undocumented Profiler options part 4 appeared first on Undocumented Matlab.

Simple call history

One of the features that the programmatic profiling interface provides, and which is not represented in the Profiler GUI, is a report of the exact order in which different functions were called during the profiling session. This profiling history can be turned on using the –history switch. There is also an optional switch of –historysize, which enables us to modify the history size from a default maximum of 1 million function entry and exit items. Here is a sample usage of this history feature:

>> profile('on','-history'); surf(peaks); profile('off')
>> profData = profile('info');
>> history = profData.FunctionHistory
history =
  Columns 1 through 11
     0     0     0     1     0     1     0     1     1     1     0
    19     1    17    17    17    17    18    18     1    19     5
  ...

The history data is actually a numeric matrix, where the first row contains the values 0 (=function entry) or 1 (=function exit), and the second row is the corresponding index into profData.FunctionTable, indicating the called function. We can easily convert this matrix into human-readable form using the following code snippet:

offset = cumsum(1-2*history(1,:)) - 1;  % calling depth
entryIdx = history(1,:)==1;     % history items of function entries
funcIdx = history(2,entryIdx);  % indexes of relevant functions
funcNames = {profData.FunctionTable(funcIdx).FunctionName};
for idx = 1: length(funcNames);
   disp([repmat(' ',1,offset(idx)) funcNames{idx}]);
end

which generates the following calling list in the MATLAB Command Window:

isempty
 isempty
  transpose
 meshgrid
  peaks
 nargchk
  error
 ishg2parent
...

Unfortunately, the history information does not by default contain specific timing of each function entry/exit, but we can still make good use of it by looking at the sequence in which the functions were called from each other.

Detailed call history, with timing information

In order to retrieve actual history timing information, we can run profile with the undocumented/unsupported –timestamp switch, which stores the CPU clock next to the history information. The reported history matrix now has 4 rows rather than 2, where the extra rows represents the timestamp of each function entry/exit:

>> profile('on','-timestamp'); surf(peaks); profile('off')
>> profData = profile('info');
>> profData.FunctionHistory(:,1:3)
ans =
                0                0                1
                1                2                2
       1347473710       1347473710       1347473710
           453000           453000           468000

In this report, the 3rd row represents the timestamp in seconds, while the 4th row represents the fractional portion of the timestamp, in microseconds. In the example above, the first timestamp item corresponds to 1347473710.453 seconds.
The seconds value appears to be related to the number of seconds since midnight Jan 1, 1970 (so-called Epoch), which is a standard time representation in computing systems. However, the actual value appears to be off by slightly over a day from the expected value (which is retrievable via getTime(java.util.Date)) for some unknown reason. Since we are only interested in relative, rather than absolute times when profiling, this minor difference does not affect us.
The timeline of the profiling session can be visualized as follows:

histData = profData.FunctionHistory;
startTime     = histData(3,1) + histData(4,1)/1e6;
relativeTimes = histData(3,:) + histData(4,:)/1e6 - startTime;
plot(relativeTimes);

Invocation  #1 of func():  0.500 secs
Invocation  #2 of func():  0.013 secs
Invocation  #3 of func():  0.011 secs
   ...
Invocation #10 of func():  0.012 secs
_____________________________________
Total invocation time:     0.600 secs

In this simulation, we would not have known that the 0.6 secs invocation time of func() is not really evenly distributed across all 10 invocations, and this would lead us to incorrect conclusions. For example, we could spend time unnecessarily on tuning the steady-state run-time performance of the function, whereas we should really have concentrated on only the first invocation. By looking at the actual timestamps we could see the large run-time used by the first invocation and this information can possibly be used to tune this first invocation and significantly reduce the overall time taken by the function.
Note: instead of using profile -timestamp, we could also have used the undocumented built-in function callstats, which is the underlying profiling engine (the profile function is simply a convenient wrapper function to callstats – take a look within profile.m):

callstats('history',2);  % 0 = -nohistory, 1 = -history, 2 = -timestamp

History collection has an overhead, so if you don’t need it then you should turn it off:

profile -nohistory
profile('-nohistory')    % equivalent alternative
callstats('history',0);  % equivalent alternative

Addendum (June 16th, 2014): I have created a utility (profile_history, available on the Matlab File Exchange) that parses all this profile data and presents it in an interactive GUI. See this article for details.

Function call timeline profiling (click for full-size image)

The post Undocumented Profiler options part 3 appeared first on Undocumented Matlab.

profile -memory on;
profile('-memory','on');  % an alternative

To turn on JIT information, run this (again, only once is necessary, prior to profile report):

setpref('profiler','showJitLines',1);

You will then see additional JIT and memory (allocated, freed and peak) information displayed in the profile report, as well as the options to sort by allocated, freed and peak memory:

showJitLines = getpref('profiler','showJitLines',false);

…and profile.m (lines 163-165 on R2007b), for the memory option:

if memory ~= -1
    callstats('memory', memory);
end

Note that there appears to be two undocumented additional memory-related options in profile.m (lines 311-312):

options = {'detail', 'timer', 'history', 'nohistory', 'historysize', ...
           'timestamp', 'memory', 'callmemory', 'nomemory' };

However, ‘-nomemory’ appears to simply turn the memory stats collection off, and ‘-callmemory’ is not recognized because of a bug in line 349, which looks for ‘callnomemory’…:

    case 'callnomemory'   % should be 'callmemory'
           memory = 2;

When this bug is fixed, we see that we get only partial memory information, so the ‘-callmemory’ option is really not useful – use ‘-memory’ instead.
Addendum (Jan 31, 2011): JIT information has been removed in Matlab 7.12 (R2011a). I assume that this was done so that programmers will not attempt to depend on JITC functionality in their code (see Michelle Hirsch’s comment below). It’s a good thing that the memory options remain, since these are quite useful in profiling memory-related bottlenecks.

The post Undocumented profiler options appeared first on Undocumented Matlab.