Today I will show another example, this time speeding up the mvksdensity (multi-variate kernel probability density estimate) function, part of the Statistics toolbox since R2016a. You will need Matlab R2016a or newer with the Stats Toolbox to recreate my results, but the general methodology and conclusions hold well for numerous other builtin Matlab functions that may be slowing down your Matlab program. In my specific problem, this function was used to compute the probability density-function (PDF) over a 1024×1024 data mesh.

The builtin mvksdensity function took 76 seconds to run on my machine; I got this down to 13 seconds, a 6x speedup, without compromising accuracy. Here’s how I did this:

Preparing the work files

While we could in theory modify Matlab’s installed m-files if we have administrator privileges, doing this is not a good idea for several reasons. Instead, we should copy and rename the relevant internal files to our work folder, and only modify our local copies.

To see where the builtin files are located, we can use the which function:

>> which('mvksdensity')
C:\Program Files\Matlab\R2020a\toolbox\stats\stats\mvksdensity.m

In our case, we copy \toolbox\stats\stats\mvksdensity.m as mvksdensity_.m to our work folder, replace the function name at the top of the file from mvksdensity to mvksdensity_, and modify our code to call mvksdensity_ rather than mvksdensity.

If we run our code, we get an error telling us that Matlab can’t find the statkscompute function (in line #107 of our mvksdensity_.m). So we find statkscompute.m in the \toolbox\stats\stats\private\ folder, copy it as statkscompute_.m to our work folder, rename its function name (at the top of the file) to statkscompute_, and modify our mvksdensity_.m to call statkscompute_ rather than statkscompute:

[fout,xout,u] = statkscompute_(ftype,xi,xispecified,npoints,u,L,U,weight,cutoff,...

We now repeat the process over and over, until we have all copied all the necessary internal files for the program to run. In our case, it tuns out that in addition to mvksdensity.m and statkscompute.m, we also need to copy statkskernelinfo.m.

Finally, we check that the numeric results using the copied files are exactly the same as from the builtin method, just to be on the safe side that we have not left out some forgotten internal file.

Now that we have copied these 3 files, in practice all our attentions will be focused on the dokernel sub-function inside statkscompute_.m, since the profiling report (below) indicates that this is where all of the run-time is spent.

Identifying the hotspots

Now we run the code through the Matlab Profiler, using the “Run and Time” button in the Matlab Editor, or profile on/report in the Matlab console (Command Window). The results show that 99.8% of mvksdensity‘s time was spent in the internal dokernel function, 75% of which was spent in self-time (meaning code lines within dokernel):

Let’s drill into dokernel and see where the problems are:

Evaluating the normal kernel distribution

We can immediately see from the profiling results that a single line (#386) in statkscompute_.m is responsible for nearly 40% of the total run-time:

fk = feval(kernel,z);

In this case, kernel is a function handle to the normal-distribution function in \stats\private\statkskernelinfo>normal, which is evaluated 1,488,094 times. Using feval incurs an overhead, as can be seen by the difference in run-times: line #386 takes 29.55 secs, whereas the normal function evaluations only take 18.53 secs. In fact, if you drill into the normal function in the profiling report, you’ll see that the actual code line that computes the normal distribution only takes 8-9 seconds – all the rest (~20 secs, or ~30% of the total) is totally redundant function-call overhead. Let’s try to remove this overhead by calling the kernel function directly:

fk = kernel(z);

Now that we have a local copy of statkscompute_.m, we can safely modify the dokernel sub-function, specifically line #386 as explained above. It turns out that just bypassing the feval call and using the function-handle directly does not improve the run-time (decrease the function-call overhead) significantly, at least on recent Matlab releases (it has a greater effect on old Matlab releases, but that’s a side-issue).

We now recognize that the program only evaluates the normal-distribution kernel, which is the default kernel. So let’s handle this special case by inlining the kernel’s one-line code (from statkskernelinfo_.m) directly (note how we move the condition outside of the loop, so that it doesn’t get recomputed 1 million times):

...
isKernelNormal = strcmp(char(kernel),'normal');  % line #357 
for i = 1:m
    Idx = true(n,1);
    cdfIdx = true(n,1);
    cdfIdx_allBelow = true(n,1);
    for j = 1:d
        dist = txi(i,j) - ty(:,j);
        currentIdx = abs(dist) <= halfwidth(j);
        Idx = currentIdx & Idx; % pdf boundary
        if iscdf
            currentCdfIdx = dist >= -halfwidth(j);
            cdfIdx = currentCdfIdx & cdfIdx; %cdf boundary1, equal or below the query point in all dimension
            currentCdfIdx_below = dist - halfwidth(j) > 0;                   
            cdfIdx_allBelow = currentCdfIdx_below & cdfIdx_allBelow; %cdf boundary2, below the pdf lower boundary in all dimension
        end
    end
    if ~iscdf
        nearby = index(Idx);
    else
        nearby = index((Idx|cdfIdx)&(~cdfIdx_allBelow));
    end
    if ~isempty(nearby)
        ftemp = ones(length(nearby),1);
        for k =1:d
            z = (txi(i,k) - ty(nearby,k))./u(k);
            if reflectionPDF
                zleft  = (txi(i,k) + ty(nearby,k)-2*L(k))./u(k);
                zright = (txi(i,k) + ty(nearby,k)-2*U(k))./u(k);
                fk = kernel(z) + kernel(zleft) + kernel(zright);  % old: =feval()+...
            elseif isKernelNormal
                fk = exp(-0.5 * (z.*z)) ./ sqrt(2*pi);
            else
                fk = kernel(z);  %old: =feval(kernel,z);
            end
            if needUntransform(k)
                fk = untransform_f(fk,L(k),U(k),xi(i,k));
            end
            ftemp = ftemp.*fk;
        end
        f(i) = weight(nearby) * ftemp;
    end
    if iscdf && any(cdfIdx_allBelow)
        f(i) = f(i) + sum(weight(cdfIdx_allBelow));
    end
end
...

This reduced the kernel evaluation run-time from ~30 secs down to 8-9 secs. Not only did we remove the direct function-call overhead, but also the overheads associated with calling a sub-function in a different m-file. The total run-time is now down to 45-55 seconds (expect some fluctuations from run to run). Not a bad start.

Main loop – bottom part

Now let’s take a fresh look at the profiling report, and focus separately on the bottom and top parts of the main loop, which you can see above. We start with the bottom part, since we already messed with it in our fix to the kernel evaluation:

The first thing we note is that there’s an inner loop that runs d=2 times (d is set in line #127 of mvksdensity_.m – it is the input mesh’s dimensionality, and also the number of columns in the txi data matrix). We can easily vectorize this inner loop, but we take care to do this only for the special case of d==2 and when some other special conditions occur.

In addition, we hoist outside of the main loop anything that we can (such as the constant exponential power, and the weight multiplication when it is constant [which is typical]), so that they are only computed once instead of 1 million times:

...
isKernelNormal = strcmp(char(kernel),'normal');
anyNeedTransform = any(needUntransform);
uniqueWeights = unique(weight);
isSingleWeight = ~iscdf && numel(uniqueWeights)==1;
isSpecialCase1 = isKernelNormal && ~reflectionPDF && ~anyNeedTransform && d==2;
expFactor = -0.5 ./ (u.*u)';
TWO_PI = 2*pi;
for i = 1:m
    ...
    if ~isempty(nearby)
        if isSpecialCase1
            z = txi(i,:) - ty(nearby,:);
            ftemp = exp((z.*z) * expFactor);
        else
            ftemp = 1;  % no need for the slow ones()
            for k = 1:d
                z = (txi(i,k) - ty(nearby,k)) ./ u(k);
                if reflectionPDF
                    zleft  = (txi(i,k) + ty(nearby,k)-2*L(k)) ./ u(k);
                    zright = (txi(i,k) + ty(nearby,k)-2*U(k)) ./ u(k);
                    fk = kernel(z) + kernel(zleft) + kernel(zright);  % old: =feval()+...
                elseif isKernelNormal
                    fk = exp(-0.5 * (z.*z)) ./ sqrt(TWO_PI);
                else
                    fk = kernel(z);  % old: =feval(kernel,z)
                end
                if needUntransform(k)
                    fk = untransform_f(fk,L(k),U(k),xi(i,k));
                end
                ftemp = ftemp.*fk;
            end
            ftemp = ftemp * TWO_PI;
        end
        if isSingleWeight
            f(i) = sum(ftemp);
        else
            f(i) = weight(nearby) * ftemp;
        end
    end
    if iscdf && any(cdfIdx_allBelow)
        f(i) = f(i) + sum(weight(cdfIdx_allBelow));
    end
end
if isSingleWeight
    f = f * uniqueWeights;
end
if isKernelNormal && ~reflectionPDF
    f = f ./ TWO_PI;
end
...

This brings the run-time down to 31-32 secs. Not bad at all, but we can still do much better:

Main loop – top part

Now let’s take a look at the profiling report’s top part of the main loop:

Again we note is that there’s an inner loop that runs d=2 times, which we can again easily vectorize. In addition, we note the unnecessary repeated initializations of the true(n,1) vector, which can easily be hoisted outside the loop:

...
TRUE_N = true(n,1);
isSpecialCase2 = ~iscdf && d==2;
for i = 1:m
    if isSpecialCase2
        dist = txi(i,:) - ty;
        currentIdx = abs(dist) <= halfwidth;
        currentIdx = currentIdx(:,1) & currentIdx(:,2);
        nearby = index(currentIdx);
    else
        Idx = TRUE_N;
        cdfIdx = TRUE_N;
        cdfIdx_allBelow = TRUE_N;
        for j = 1:d
            dist = txi(i,j) - ty(:,j);
            currentIdx = abs(dist) <= halfwidth(j);
            Idx = currentIdx & Idx; % pdf boundary
            if iscdf
                currentCdfIdx = dist >= -halfwidth(j);
                cdfIdx = currentCdfIdx & cdfIdx; % cdf boundary1, equal or below the query point in all dimension
                currentCdfIdx_below = dist - halfwidth(j) > 0;
                cdfIdx_allBelow = currentCdfIdx_below & cdfIdx_allBelow; %cdf boundary2, below the pdf lower boundary in all dimension
            end
        end
        if ~iscdf
            nearby = index(Idx);
        else
            nearby = index((Idx|cdfIdx)&(~cdfIdx_allBelow));
        end
    end
    if ~isempty(nearby)
        ...

This brings the run-time down to 24 seconds.

We next note that instead of using numeric indexes to compute the nearby vector, we could use faster logical indexes:

...
%index = (1:n)';  % this is no longer needed
TRUE_N = true(n,1);
isSpecialCase2 = ~iscdf && d==2;
for i = 1:m
    if isSpecialCase2
        dist = txi(i,:) - ty;
        currentIdx = abs(dist) <= halfwidth;
        nearby = currentIdx(:,1) & currentIdx(:,2);
    else
        Idx = TRUE_N;
        cdfIdx = TRUE_N;
        cdfIdx_allBelow = TRUE_N;
        for j = 1:d
            dist = txi(i,j) - ty(:,j);
            currentIdx = abs(dist) <= halfwidth(j);
            Idx = currentIdx & Idx; % pdf boundary
            if iscdf
                currentCdfIdx = dist >= -halfwidth(j);
                cdfIdx = currentCdfIdx & cdfIdx; % cdf boundary1, equal or below the query point in all dimension
                currentCdfIdx_below = dist - halfwidth(j) > 0;
                cdfIdx_allBelow = currentCdfIdx_below & cdfIdx_allBelow; %cdf boundary2, below the pdf lower boundary in all dimension
            end
        end
        if ~iscdf
            nearby = Idx;  % not index(Idx)
        else
            nearby = (Idx|cdfIdx) & ~cdfIdx_allBelow;  % no index()
        end
    end
    if any(nearby)
        ...

This brings the run-time down to 20 seconds.

We now note that the main loop runs m=1,048,576 (=1024×1024) times over all rows of txi. This is expected, since the loop runs over all the elements of a 1024×1024 mesh grid, which are reshaped as a 1,048,576-element column array at some earlier point in the processing, resulting in a m-by-d matrix (1,048,576-by-2 in our specific case). This information helps us, because we know that there are only 1024 unique values in each of the two columns of txi. Therefore, instead of computing the “closeness” metric (which leads to the nearby vector) for all 1,048,576 x 2 values of txi, we calculate separate vectors for each of the 1024 unique values in each of its 2 columns, and then merge the results inside the loop:

...
isSpecialCase2 = ~iscdf && d==2;
if isSpecialCase2
    [unique1Vals, ~, unique1Idx] = unique(txi(:,1));
    [unique2Vals, ~, unique2Idx] = unique(txi(:,2));
    dist1 = unique1Vals' - ty(:,1);
    dist2 = unique2Vals' - ty(:,2);
    currentIdx1 = abs(dist1) <= halfwidth(1);
    currentIdx2 = abs(dist2) <= halfwidth(2);
end
for i = 1:m
    if isSpecialCase2
        idx1 = unique1Idx(i);
        idx2 = unique2Idx(i);
        nearby = currentIdx1(:,idx1) & currentIdx2(:,idx2);
    else
        ...

This brings the run-time down to 13 seconds, a total speedup of almost ~6x compared to the original version. Not bad at all.

For reference, here's a profiling summary of the dokernel function again, showing the updated performance hotspots:

The 2 vectorized code lines in the bottom part of the main loop now account for 72% of the remaining run-time:

    ...
    if ~isempty(nearby)
        if isSpecialCase1
            z = txi(i,:) - ty(nearby,:);
            ftemp = exp((z.*z) * expFactor);
        else
            ...

If I had the inclination, speeding up these two code lines would be the next logical step, but I stop at this point. Interested readers could pick up this challenge and post a solution in the comments section below. I haven't tried it myself, so perhaps there's no easy way to improve this. Then again, perhaps the answer is just around the corner - if you don't try, you'll never know...

Data density/resolution

So far, all the optimization I made have not affected code accuracy, generality or resolution. This is always the best approach if you have some spare coding time on your hands.

In some cases, we might have a deep understanding of our domain problem to be able to sacrifice a bit of accuracy in return for run-time speedup. In our case, we identify the main loop over 1024x1024 elements as the deciding factor in the run-time. If we reduce the grid-size by 50% in each dimension (i.e. 512x512), the run-time decreases by an additional factor of almost 4, down to ~3.5 seconds, which is what we would have expected since the main loop size has decreased 4 times in size. While this reduces the results resolution/accuracy, we got a 4x speedup in a fraction of the time that it took to make all the coding changes above.

Different situations may require different approaches: in some cases we cannot sacrifice accuracy/resolution, and must spend time to improve the algorithm implementation; in other cases coding time is at a premium and we can sacrifice accuracy/resolution; and in other cases still, we could use a combination of both approaches.

Conclusions

Matlab is composed of thousands of internal functions. Each and every one of these functions was meticulously developed and tested by engineers, who are after all only human. Whereas supreme emphasis is always placed with Matlab functions on their accuracy, run-time performance often takes a back-seat. Make no mistake about this: code accuracy is almost always more important than speed, so I’m not complaining about the current state of affairs.

But when we run into a specific run-time problem in our Matlab program, we should not despair if we see that built-in functions cause slowdown. We can try to avoid calling those functions (for example, by reducing the number of invocations, or decreasing the data resolution, or limiting the target accuracy, etc.), or we could optimize these functions in our own local copy, as I have shown today. There are multiple techniques that we could employ to improve the run time. Just use the profiler and keep an open mind about alternative speed-up mechanisms, and you’d be half-way there. For ideas about the multitude of different speedup techniques that you could use in Matlab, see my book Accelerating Matlab Performance.

Let me know if you’d like me to assist with your Matlab project, either developing it from scratch or improving your existing code, or just training you in how to improve your Matlab code’s run-time/robustness/usability/appearance.

In the meantime, Happy Easter/Passover everyone, and stay healthy!

The post Speeding-up builtin Matlab functions – part 3 appeared first on Undocumented Matlab.

Profiling

As I explained last week, the first step in speeding up any function is to profile its current run-time behavior using Matlab’s builtin Profiler tool, which can either be started from the Matlab Editor toolstrip (“Run and Time”) or via the profile function.
The profile report for the client’s function showed that it had two separate performance hotspots:

1. Code that checks the drawdown format (optional 2nd input argument) against a set of allowed formats
2. Main code section that iteratively loops over the data-series values to compute the maximal drawdown

In order top optimize the code, I copied %matlabroot%/toolbox/finance/finance/maxdrawdown.m to a local folder on the Matlab path, renaming the file (and the function) maxdrawdn, in order to avoid conflict with the built-in version.

Optimizing input args pre-processing

The main problem with the pre-processing of the optional format input argument is the string comparisons, which are being done even when the default format is used (which is by far the most common case). String comparison are often more expensive than numerical computations. Each comparison by itself is very short, but when maxdrawdown is run in a loop (as it often is), the run-time adds up.
Here’s a snippet of the original code:

if nargin < 2 || isempty(Format)
    Format = 'return';
end
if ~ischar(Format) || size(Format,1) ~= 1
    error(message('finance:maxdrawdown:InvalidFormatType'));
end
choice = find(strncmpi(Format,{'return','arithmetic','geometric'},length(Format)));
if isempty(choice)
    error(message('finance:maxdrawdown:InvalidFormatValue'));
end

An equivalent code, which avoids any string processing in the common default case, is faster, simpler and more readable:

if nargin < 2 || isempty(Format)
    choice = 1;
elseif ~ischar(Format) || size(Format,1) ~= 1
    error(message('finance:maxdrawdown:InvalidFormatType'));
else
    choice = find(strncmpi(Format,{'return','arithmetic','geometric'},length(Format)));
    if isempty(choice)
        error(message('finance:maxdrawdown:InvalidFormatValue'));
    end
end

The general rule is that whenever you have a common case, you should check it first, avoiding unnecessary processing downstream. Moreover, for improved run-time performance (although not necessarily maintainability), it is generally preferable to work with numbers rather than strings (choice rather than Format, in our case).

Vectorizing the main loop

The main processing loop uses a very simple yet inefficient iterative loop. I assume that the code was originally developed this way in order to assist debugging and to ensure correctness, and that once it was ready nobody took the time to also optimize it for speed. It looks something like this:

MaxDD = zeros(1,N);
MaxDDIndex = ones(2,N);
...
if choice == 1   % 'return' format
    MaxData = Data(1,:);
    MaxIndex = ones(1,N);
    for i = 1:T
        MaxData = max(MaxData, Data(i,:));
        q = MaxData == Data(i,:);
        MaxIndex(1,q) = i;
        DD = (MaxData - Data(i,:)) ./ MaxData;
        if any(DD > MaxDD)
            p = DD > MaxDD;
            MaxDD(p) = DD(p);
            MaxDDIndex(1,p) = MaxIndex(p);
            MaxDDIndex(2,p) = i;
        end
    end
else             % 'arithmetic' or 'geometric' formats
    ...

This loop can relatively easily be vectorized, making the code much faster, and arguably also simpler, more readable, and more maintainable:

if choice == 3
    Data = log(Data);
end
MaxDDIndex = ones(2,N);
MaxData = cummax(Data);
MaxIndexes = find(MaxData==Data);
DD = MaxData - Data;
if choice == 1	% 'return' format
    DD = DD ./ MaxData;
end
MaxDD = cummax(DD);
MaxIndex2 = find(MaxDD==DD,1,'last');
MaxIndex1 = MaxIndexes(find(MaxIndexes<=MaxIndex2,1,'last'));
MaxDDIndex(1,:) = MaxIndex1;
MaxDDIndex(2,:) = MaxIndex2;
MaxDD = MaxDD(end,:);

Let's make a short run-time and accuracy check - we can see that we achieved a 31-fold speedup (YMMV), and received the exact same results:

>> data = rand(1,1e7);
>> tic, [MaxDD1, MaxDDIndex1] = maxdrawdown(data); toc  % builtin Matlab function
Elapsed time is 7.847140 seconds.
>> tic, [MaxDD2, MaxDDIndex2] = maxdrawdn(data); toc  % our optimized version
Elapsed time is 0.253130 seconds.
>> speedup = round(7.847140 / 0.253130)
speedup =
    31
>> isequal(MaxDD1,MaxDD2) && isequal(MaxDDIndex1,MaxDDIndex2)  % check accuracy
ans =
  logical
   1

Disclaimer: The code above seems to work (quite well in fact) for a 1D data vector. You'd need to modify it a bit to handle 2D data - the returned maximal drawdown are still computed correctly but not the returned indices, due to their computation using the find function. This modification is left as an exercise for the reader...

Related functions

Very similar code could be used for the corresponding maxdrawup function. Although this function is used much less often than maxdrawdown, it is in fact widely used and very similar to maxdrawdown, so it is surprising that it is missing in the Financial Toolbox. Here is the corresponding code snippet:

% Code snippet for maxdrawup (similar to maxdrawdn)
MaxDUIndex = ones(2,N);
MinData = cummin(Data);
MinIndexes = find(MinData==Data);
DU = Data - MinData;
if choice == 1	% 'return' format
    DU = DU ./ MinData;
end
MaxDU = cummax(DU);
MaxIndex = find(MaxDD==DD,1,'last');
MinIndex = MinIndexes(find(MinIndexes<=MaxIndex,1,'last'));
MaxDUIndex(1,:) = MinIndex;
MaxDUIndex(2,:) = MaxIndex;
MaxDU = MaxDU(end,:);

Similar vectorization could be applied to the emaxdrawdown function. This too is left as an exercise for the reader...

Conclusions

Matlab is composed of thousands of internal functions. Each and every one of these functions was meticulously developed and tested by engineers, who are after all only human. Whereas supreme emphasis is always placed with Matlab functions on their accuracy, run-time performance sometimes takes a back-seat. Make no mistake about this: code accuracy is almost always more important than speed (an exception are cases where some accuracy may be sacrificed for improved run-time performance). So I'm not complaining about the current state of affairs.
But when we run into a specific run-time problem in our Matlab program, we should not despair if we see that built-in functions cause slowdown. We can try to avoid calling those functions (for example, by reducing the number of invocations, or limiting the target accuracy, etc.), or optimize these functions in our own local copy, as I've shown last week and today. There are multiple techniques that we could employ to improve the run time. Just use the profiler and keep an open mind about alternative speed-up mechanisms, and you'd be half-way there.
Let me know if you'd like me to assist with your Matlab project, either developing it from scratch or improving your existing code, or just training you in how to improve your Matlab code's run-time/robustness/usability/appearance. I will be visiting Boston and New York in early June and would be happy to meet you in person to discuss your specific needs.

The post Speeding-up builtin Matlab functions – part 2 appeared first on Undocumented Matlab.

Profiling

The first step in speeding up any function is to profile its current run-time behavior using Matlab’s builtin Profiler tool, which can either be started from the Matlab Editor toolstrip (“Run and Time”) or via the profile function.
The profile report for the client’s function showed that 99% of the time was spent in the Statistics Toolbox’s fitdist function. Drilling into this function in the profiling report, we see onion-like functions that processed input parameters, ensured data validation etc. The core processing is done inside a class that is unique to each required distribution (e.g., prob.StableDistribution, prob.BetaDistribution etc.) that is invoked within fitdist using an feval call, based on the distribution name that was specified by the user.
In our specific case, the external onion layers of sanity checks were unnecessary and could be avoided. In general, I advise not to discard such checks, because you never know whether future uses might have a problem with outlier data or input parameters. Moreover, in the specific case of fitdist they take only a very minor portion of the run-time (this may be different in other cases, such as the ismember function that I described years ago, where the sanity checks have a significant run-time impact compared to the core processing in the internal ismembc function).
However, since we wanted to significantly improve the total run-time and this was spent within the distribution class (prob.StableDistribution in the case of my client), we continue to drill-down into this class to determine what can be done.
It turns out that prob.StableDistribution basically does 3 things in its main fit() method:

1. pre-process the input data (prob.ToolboxFittableParametricDistribution.processFitArgs() and .removeCensoring() methods) – this turned out to be unnecessary in my client’s data, but has little run-time impact.
2. call stablefit() in order to get fitting parameters – this took about half the run-time
3. call stablelike() in order to get likelihood data – this took about half the run-time as well
4. call prob.StableDistribution.makeFitted() to create a probability-distribution object returned to the caller – this also took little run-time that was not worth bothering about.

The speed-up improvement process

With user-created code we could simply modify our code in-place. However, a more careful process is necessary when modifying built-in Matlab functions (either in the core Matlab distribution or in one of the toolboxes).
The basic idea here is to create a side-function that would replicate the core processing of fitdist. This is preferable to modifying Matlab’s installation files because we could then reuse the new function in multiple computers, without having to mess around in the Matlab installation (which may not even be possible if we do not have admin privileges). Also, if we ever upgrade our Matlab we won’t need to remember to update the installed files (and obviously retest).
I called the new function fitdist2 and inside it I initially placed only the very core functionality of prob.StableDistribution.fit():

% fitdist2 - fast replacement for fitdist(data,'stable')
% equivalent to pd = prob.StableDistribution.fit(data);
function pd = fitdist2(data)
    % Bypass pre-processing (not very important)
    [cens,freq,opt] = deal([]);
    %[data,cens,freq,opt] = prob.ToolboxFittableParametricDistribution.processFitArgs(data);
    %data = prob.ToolboxFittableParametricDistribution.removeCensoring(data,cens,freq,'stable');
    % Main processing in stablefit(), stablelike()
    params = stablefit(data,0.05,opt);
    [nll,cov] = stablelike(params,data);
    % Combine results into the returned probability distribution object
    pd = prob.StableDistribution.makeFitted(params,nll,cov,data,cens,freq);
end

If we try to run this as-is, we’d see errors because stablefit() and stablelike() are both sub-functions within %matlabroot%/toolbox/stats/stats/+prob/StableDistribution.m. So we copy these sub-functions (and their dependent subfunctions infoMtxCal(), intMle(), tabpdf(), neglog_pdf(), stable_nloglf(), varTrans) to the bottom of our fitdist2.m file, about 400 lines in total.
We also remove places that call checkargs(…) since that seems to be unnecessary – if you want to keep it then add the checkargs() function as well.
Now we re-run our code, after each speedup iteration verifying that the returned pd object returned by our fitdist2 is equivalent to the original object returned by fitdist.

Speeding-up stablefit()

A new profiling run shows that the vast majority of the time in stablefit() is spent in two lines:

1. s = load('private/StablePdfTable.mat');
2. [parmhat,~,err,output] = fminsearch(@(params)stable_nloglf(x,params),phi0,options);

The first of these lines is reloading a static data file. The very same static data file is later reloaded in stablelike(). Both of these data-loads is done in every single invocation of fitdist, so if we have 5000 data fits, we load the same static data file 10,000 times! This is certainly not indicative of good programming practice. It would be much faster to reload the static data once into memory, and then use this cached data for the next 9,999 invocation. Since the original authors of StableDistribution.m seem to like single-character global variables (another bad programming practice, for multiple reasons), we’ll follow their example (added lines are highlighted):

persistent s  % this should have a more meaningful name (but at least is not global...)!
if isempty(s)
    fit_path = fileparts(which('fitdist'));
    s = load([fit_path '/private/StablePdfTable.mat']);
    a = s.a;
    b = s.b;
    xgd = s.xgd;
    p = s.p;
end

In order to speed-up the second line (that calls fminsearch), we can reduce the tolerances used by this function, by updating the options structure passed to it, so that we use tolerances of 1e-3 rather than the default 1e-6 (in our specific case this resulted in acceptable errors of ~0.1%). Specifically, we modify the code from this:

function [parmhat,parmci] = stablefit(x,alpha,options)
...
if nargin < 3 || isempty(options)
    options = statset('stablefit');
else
    options = statset(statset('stablefit'),options);
end
% Maximize the log-likelihood with respect to the transformed parameters
[parmhat,~,err,output] = fminsearch(@(params)stable_nloglf(x,params),phi0,options);
...
end

to this (note that the line that actually calls fminsearch remains unchanged):

function [parmhat,parmci] = stablefit(x,alpha,unused_options)
...
persistent options
if isempty(options)
    options = statset('stablefit');
    options.TolX   = 1e-3;
    options.TolFun = 1e-3;
    options.TolBnd = 1e-3;
end
% Maximize the log-likelihood with respect to the transformed parameters
[parmhat,~,err,output] = fminsearch(@(params)stable_nloglf(x,params),phi0,options);
...
end

The fminsearch internally calls tabpdf() repeatedly. Drilling down in the profiling report we see that it recomputes a griddedInterpolant object that is essentially the same for all iterations (and therefore a prime candidate for caching), and also that it uses the costly cubic interpolation rather than a more straight-forward linear interpolation:

function y = tabpdf(x,alpha,beta)
...
persistent G  % this should have a more meaningful name (but at least is not global...)!
if isempty(G)
    G = griddedInterpolant({b, a, xgd}, p, 'linear','none');  % 'linear' instead of 'cubic'
end
%G = griddedInterpolant({b, a, xgd}, p, 'cubic','none');  % original
y = G({beta,alpha,x});
...

These cases illustrate two important speedup technique categories: caching data in order to reduce the number of times that a certain code hotspot is being run, and modifying the parameters/inputs in order to reduce the individual run-time of the hotspot code. Variations of these techniques form the essence of effective speedup and can often be achieved by just reflecting on the problem and asking yourself two questions:

1. can I reduce the number of times that this code is being run? and
2. can I reduce the run-time of this specific code section?

Additional important speed-up categories include parallelization, vectorization and algorithmic modification. These are sometimes more difficult programmatically and riskier in terms of functional equivalence, but may be required in case the two main techniques above are not feasible. Of course, we can always combine these techniques, we don't need to choose just one or the other.

Speeding-up stablelike()

We now turn our attentions to stablelike(). As for the loaded static file, we could simply use the cached s to load the data in order to avoid repeated reloading of the data from disk. But it turns out that this data is actually not used at all inside the function (!) so we just comment-out the old code:

%s = load('private/StablePdfTable.mat');
%a = s.a;
%b = s.b;
%xgd = s.xgd;
%p = s.p;

Think about this - the builtin Matlab code loads a data file from disk and then does absolutely nothing with it - what a waste!
Another important change is to reduce the run-time of the integral function, which is called thousands of times within a double loop. We do this by reducing the tolerances specified in the integral call from 1e-6:

F(i,j) = integral(@(x)infoMtxCal(x,params,step,i,j),-Inf,Inf,'AbsTol',1e-6,'RelTol',1e-4); % original
F(i,j) = integral(@(x)infoMtxCal(x,params,step,i,j),-Inf,Inf,'AbsTol',1e-3,'RelTol',1e-3); % new

You can see that once again these two cases follow the two techniques that I mentioned above: we reduced the number of times that we load the data file (to 0 in our case), and we improved the run-time of the individual integral calculation by reducing its tolerances.

Conclusions

The final result of applying the techniques above was a 6-fold speedup, reducing the total run-time from 45 minutes down to 7 minutes. I could probably have improved the run-time even further, but since we reached our target run-time I stopped there. The point after all was to make the code usable, not to reach a world speed record.
In my next article I will present another example of dramatically improving the run-time speed of a built-in Matlab function, this time a very commonly-used function in the Financial Toolbox that I was able to speed-up by a factor of 40.
Matlab releases improve continuously, so hopefully my techniques above (or alternatives) would find their way into the builtin Matlab functions, making them faster than today, out-of-the-box.
Until this happens, we should not lose hope when faced with a slow Matlab function, even if it is a built-in/internal one, as I hope to have clearly illustrated today, and will also show in my next article. Improving the performance is often easy. In fact, it took me much longer to write this article than it was to improve my client's code...
Let me know if you'd like me to assist with your Matlab project, either developing it from scratch or improving your existing code, or just training you in how to improve your Matlab code's run-time/robustness/usability/appearance. I will be visiting Boston and New York in early June and would be happy to meet you in person to discuss your specific needs.

The post Speeding-up builtin Matlab functions – part 1 appeared first on Undocumented Matlab.

The problem

The problem arose as I was refactoring some code that was written in R2006b to R2009b. The code in question uses the impoint tool on an image along with an associated text object that moves with the point to display information as it is dragged around the image. At the time of the R2006b release the impoint tool was written as an API. In R2006b the call to impoint returns a handle to an hggroup containing a structure of function handles in its application data under the tag ‘API’. This programming pattern was common before the advent of the new class syntax in MATLAB version 7.6 (R2008a).
Here is an example of how impoint would be used in R2006b:

function impointBehavior_R2006b
%IMPOINTBEHAVIOR_R2006B shows how impoint would be used in R2006b
%Note: RUN UNDER R2006B (will run under R2009b but actually uses
%classdef impoint so it will show the same issue)
  % Display the image in a figure window
  figure;  imshow('rice.png');
  % In R2006b calling impoint returns the hggroup handle
  h = impoint(gca,100,200);
  % In 2006b iptgetapi returns a structure of function handles
  api = iptgetapi(h);
  % Add a new position callback to set the text string
  api.addNewPositionCallback(@newPos_Callback);
  % Construct boundary constraint function so we can't go outside the axes
  fcn = makeConstrainToRectFcn('impoint',get(gca,'XLim'),get(gca,'YLim'));
  api.setDragConstraintFcn(fcn);
  % Fire callback so we get initial text
  newPos_Callback(api.getPosition());
  function newPos_Callback(newPos)
    % Display the current point position in a text label
    api.setString(sprintf('(%1.0f,%1.0f)',newPos(1),newPos(2)));
  end %newPos_Callback
end %impointBehavior_R2006b

The code above, when run in R2006b, produces the desired behavior of displaying a text object containing the point coordinates that moves around with the point as it is dragged around the axes.
In R2009b, impoint is now a true MATLAB class using the new classdef syntax, so I wanted to update the existing code. Initially this appeared to be a straightforward translation of the code to make use of the new impoint class syntax. The first attempt to rewrite the code was:

function impointBehavior_R2009b
%IMPOINTBEHAVIOR_R2009B shows the undesirable behavior when
%using the setString method in R2009b.
  % Display the image in a figure window
  figure;  imshow('rice.png');
  h = impoint(gca,100,200);
  % Add a new position callback to set the text string
  h.addNewPositionCallback(@newPos_Callback);
  % Construct boundary constraint function so we can't go outside the axes
  fcn = makeConstrainToRectFcn('impoint',get(gca,'XLim'),get(gca,'YLim'));
  % Enforce boundary constraint function
  h.setPositionConstraintFcn(fcn);
  % Fire callback so we get initial text
  newPos_Callback(h.getPosition());
  function newPos_Callback(newPos)
    % Display the current point position in a text label
    h.setString(sprintf('(%1.0f,%1.0f)',newPos(1),newPos(2)))
  end %newPos_Callback
end %impointBehavior_R2009b

Unfortunately, when this code is run, dragging the mouse around the axes produces a trail of labels as shown below:

The solution

So how do we work around this bug to get to the behavior we want? One solution would be to rewrite the offending MATLAB code but this is somewhat risky in terms of maintainability and compatibility.
A more elegant solution is to subclass the impoint class and substitute the setString behavior we want. Looking at the impoint code we find that impoint is a subclass of imroi. In the imroi property declarations we see a number of undocumented properties that are protected. We can access these properties in a subclass but not outside the class. One of these undocumented properties is h_group which is an hggroup that contains the handle graphic objects that make up the impoint on the screen. The label, when created, becomes part of this hggroup with its Tag property set to ‘label’. When performing the setString method the behavior we want to see is that if the text object exists we want to update its String property. If it does not exist we want it to perform its existing functionality:

classdef impointtextupdate < impoint
%IMPOINTTEXTUPDATE subclasses impoint to override the setString
%method of impoint so that it does not create a new text object
%each time it is called.
  methods
    function obj = impointtextupdate(varargin)
      obj = obj@impoint(varargin{:});
    end %impointtextupdate
    function setString(obj,str)
      %override impoint setString
      %check to see if there is already a text label
      label = findobj(obj.h_group,'Type','text','Tag','label');
      if isempty(label)
        %do the default behavior
        setString@impoint(obj,str);
      else
        %update the existing tag
        set(label(1),'String',str);
      end %if
    end %setString
  end %methods
end %impointtextupdate

Substituting calls to impoint with the new impointupdatetext subclass now produces the desired effect as shown below:

Conclusions

This case illustrates a couple of points:

• Much of the existing code in the MATLAB toolboxes is being updated to the new object oriented syntax. This presents many opportunities to easily and elegantly modify the default behavior without modifying provided toolbox code In the example above we retain all the desirable behavior of impoint while overriding the undesirable behavior.
• Many of the properties and methods in the provided toolbox objects are hidden or protected and are undocumented. It takes some simple detective work to find these out through examining the code. MATLAB is very generous in providing much of the existing code openly. Open the functions and classes you use in the editor to really find out how they work. Over the years I've learned and adopted a lot of useful MATLAB programming patterns by examining the code in the various toolboxes (there are a few coding horrors as well!).

I hope to explore some other under-documented features of the IPT and other toolboxes in future posts.

The post Solving a MATLAB bug by subclassing appeared first on Undocumented Matlab.