A note of caution: in the examples that follow, don’t take any speedup as an expected actual value – the actual value may well be different on your system. Your mileage may vary. I only mean to display the relative differences between different alternatives.

The underlying problem

Memory management has a direct influence on performance. I have already shown some examples of this in past articles here.

Preallocation solves a basic problem in simple program loops, where an array is iteratively enlarged with new data (dynamic array growth). Unlike other programming languages (such as C, C++, C# or Java) that use static typing, Matlab uses dynamic typing. This means that it is natural and easy to modify array size dynamically during program execution. For example:

fibonacci = [0, 1];
for idx = 3 : 100
   fibonacci(idx) = fibonacci(idx-1) + fibonacci(idx-2);
end

While this may be simple to program, it is not wise with regards to performance. The reason is that whenever an array is resized (typically enlarged), Matlab allocates an entirely new contiguous block of memory for the array, copying the old values from the previous block to the new, then releasing the old block for potential reuse. This operation takes time to execute. In some cases, this reallocation might require accessing virtual memory and page swaps, which would take an even longer time to execute. If the operation is done in a loop, then performance could quickly drop off a cliff.

The cost of such naïve array growth is theoretically quadratic. This means that multiplying the number of elements by N multiplies the execution time by about N². The reason for this is that Matlab needs to reallocate N times more than before, and each time takes N times longer due to the larger allocation size (the average block size multiplies by N), and N times more data elements to copy from the old to the new memory blocks.

A very interesting discussion of this phenomenon and various solutions can be found in a newsgroup thread from 2005. Three main solutions were presented: preallocation, selective dynamic growth (allocating headroom) and using cell arrays. The best solution among these in terms of ease of use and performance is preallocation.

The basics of pre-allocation

The basic idea of preallocation is to create a data array in the final expected size before actually starting the processing loop. This saves any reallocations within the loop, since all the data array elements are already available and can be accessed. This solution is useful when the final size is known in advance, as the following snippet illustrates:

% Regular dynamic array growth:
tic
fibonacci = [0,1];
for idx = 3 : 40000
   fibonacci(idx) = fibonacci(idx-1) + fibonacci(idx-2);
end
toc
   => Elapsed time is 0.019954 seconds.
 
% Now use preallocation – 5 times faster than dynamic array growth:
tic
fibonacci = zeros(40000,1);
fibonacci(1)=0; fibonacci(2)=1;
for idx = 3 : 40000, 
   fibonacci(idx) = fibonacci(idx-1) + fibonacci(idx-2);
end
toc
   => Elapsed time is 0.004132 seconds.

On pre-R2011a releases the effect of preallocation is even more pronounced: I got a 35-times speedup on the same machine using Matlab 7.1 (R14 SP3). R2011a (Matlab 7.12) had a dramatic performance boost for such cases in the internal accelerator, so newer releases are much faster in dynamic allocations, but preallocation is still 5 times faster even on R2011a.

Non-deterministic pre-allocation

Because the effect of preallocation is so dramatic on all Matlab releases, it makes sense to utilize it even in cases where the data array’s final size is not known in advance. We can do this by estimating an upper bound to the array’s size, preallocate this large size, and when we’re done remove any excess elements:

% The final array size is unknown – assume 1Kx3K upper bound (~23MB)
data = zeros(1000,3000);  % estimated maximal size
numRows = 0;
numCols = 0;
while (someCondition)
   colIdx = someValue1;   numCols = max(numCols,colIdx);
   rowIdx = someValue2;   numRows = max(numRows,rowIdx);
   data(rowIdx,colIdx) = someOtherValue;
end
 
% Now remove any excess elements
data(:,numCols+1:end) = [];   % remove excess columns
data(numRows+1:end,:) = [];   % remove excess rows

Variants for pre-allocation

It turns out that MathWorks’ official suggestion for preallocation, namely using the zeros function, is not the most efficient:

% MathWorks suggested variant
clear data1, tic, data1 = zeros(1000,3000); toc
   => Elapsed time is 0.016907 seconds.
 
% A much faster alternative - 500 times faster!
clear data1, tic, data1(1000,3000) = 0; toc
   => Elapsed time is 0.000034 seconds.

The reason for the second variant being so much faster is because it only allocates the memory, without worrying about the internal values (they get a default of 0, false or ”, in case you wondered). On the other hand, zeros has to place a value in each of the allocated locations, which takes precious time.

In most cases the differences are immaterial since the preallocation code would only run once in the program, and an extra 17ms isn’t such a big deal. But in some cases we may have a need to periodically refresh our data, where the extra run-time could quickly accumulate.

Pre-allocating non-default values

When we need to preallocate a specific value into every data array element, we cannot use Variant #2. The reason is that Variant #2 only sets the very last data element, and all other array elements get assigned the default value (0, ‘’ or false, depending on the array’s data type). In this case, we can use one of the following alternatives (with their associated timings for a 1000×3000 data array):

scalar = pi;  % for example...
 
data = scalar(ones(1000,3000));           % Variant A: 87.680 msecs
data(1:1000,1:3000) = scalar;             % Variant B: 28.646 msecs
data = repmat(scalar,1000,3000);          % Variant C: 17.250 msecs
data = scalar + zeros(1000,3000);         % Variant D: 17.168 msecs
data(1000,3000) = 0; data = data+scalar;  % Variant E: 16.334 msecs

As can be seen, Variants C-E are about twice as fast as Variant B, and 5 times faster than Variant A.

Pre-allocating non-double data

7.4.5 Preallocating non-double data
When preallocating an array of a type that is not double, we should be careful to create it using the desired type, to prevent memory and/or performance inefficiencies. For example, if we need to process a large array of small integers (int8), it would be inefficient to preallocate an array of doubles and type-convert to/from int8 within every loop iteration. Similarly, it would be inefficient to preallocate the array as a double type and then convert it to int8. Instead, we should create the array as an int8 array in the first place:

% Bad idea: allocates 8MB double array, then converts to 1MB int8 array
data = int8(zeros(1000,1000));   % 1M elements
   => Elapsed time is 0.008170 seconds.
 
% Better: directly allocate the array as a 1MB int8 array – x80 faster
data = zeros(1000,1000,'int8');
   => Elapsed time is 0.000095 seconds.

Pre-allocating cell arrays

To preallocate a cell-array we can use the cell function (explicit preallocation), or the maximal cell index (implicit preallocation). Explicit preallocation is faster than implicit preallocation, but functionally equivalent (Note: this is contrary to the experience with allocation of numeric arrays and other arrays):

% Variant #1: Explicit preallocation of a 1Kx3K cell array
data = cell(1000,3000);
   => Elapsed time is 0.004637 seconds. 
 
% Variant #2: Implicit preallocation – x3 slower than explicit
clear('data'), data{1000,3000} = [];
   => Elapsed time is 0.012873 seconds.

Pre-allocating arrays of structs

To preallocate an array of structs or class objects, we can use the repmat function to replicate copies of a single data element (explicit preallocation), or just use the maximal data index (implicit preallocation). In this case, unlike the case of cell arrays, implicit preallocation is much faster than explicit preallocation, since the single element does not actually need to be copied multiple times (ref):

% Variant #1: Explicit preallocation of a 100x300 struct array
element = struct('field1',magic(2), 'field2',{[]});
data = repmat(element, 100, 300);
   => Elapsed time is 0.002804 seconds. 
 
% Variant #2: Implicit preallocation – x7 faster than explicit 
element = struct('field1',magic(2), 'field2',{[]});
clear('data'), data(100,300) = element;
   => Elapsed time is 0.000429 seconds.

When preallocating structs, we can also use a third variant, using the built-in struct feature of replicating the struct when the struct function is passed a cell array. For example, struct('field1',cell(100,1), 'field2',5) will create 100 structs, each of them having the empty field field1 and another field called field2 with value 5. Unfortunately, this variant is slower than both of the previous variants.

Pre-allocating class objects

When preallocating in general, ensure that you are using the maximal expected array size. There is no point in preallocating an empty array or an array having a smaller size than the expected maximum, since dynamic memory reallocation will automatically kick-in within the processing-loop. For this reason, do not use the empty() method of class objects to preallocate, but rather repmat as explained above.

When using repmat to replicate class objects, always be careful to note whether you are replicating the object itself (this happens if your class does NOT derive from handle) or its reference handle (which happens if you derive the class from handle). If you are replicating objects, then you can safely edit any of their properties independently of each other; but if you replicate references, you are merely using multiple copies of the same reference, so that modifying referenced object #1 will also automatically affect all the other referenced objects. This may or may not be suitable for your particular program requirements, so be careful to check carefully. If you actually need to use independent object copies, you will need to call the class constructor multiple times, once for each new independent object.

Next week: what if we can’t avoid dynamic array resizing? – apparently, all is not lost. Stay tuned…

Do you have any similar allocation-related tricks you’re using? or unexpected differences such as the ones shown above? If so, then please do post a comment.

1. Matrix processing performance Matrix operations performance is affected by internal subscriptions in a counter-intuitive way....
2. Performance: scatter vs. line In many circumstances, the line function can generate visually-identical plots as the scatter function, much faster...
3. Matlab-Java memory leaks, performance Internal fields of Java objects may leak memory - this article explains how to avoid this without sacrificing performance. ...
4. datestr performance Caching is a simple and very effective means to improve code performance, as demonstrated for the datestr function....

In Matlab releases of the past few years, this has been addressed by expanding the information reported by the built-in memory function. In addition, an undocumented feature was added to the Matlab Profiler that enables monitoring memory usage.

Profile report with memory & JIT info

In Matlab release R2008a (but not on newer releases) we could also use a nifty parameter of the undocumented feature function:

>> feature mtic; a=ones(100); feature mtoc
ans = 
      TotalAllocated: 84216
          TotalFreed: 2584
    LargestAllocated: 80000
           NumAllocs: 56
            NumFrees: 43
                Peak: 81640

As can easily be seen in this example, allocating 100² doubles requires 80000 bytes of allocation, plus some 4KB others that were allocated (and 2KB freed) within the function ones. Running the same code line again gives a very similar result, but now there are 80000 more bytes freed when the matrix a is overwritten:

>> feature mtic; a=ones(100); feature mtoc
ans = 
      TotalAllocated: 84120
          TotalFreed: 82760
    LargestAllocated: 80000
           NumAllocs: 54
            NumFrees: 49
                Peak: 81328

This is pretty informative and very handy for debugging memory bottlenecks. Unfortunately, starting in R2008b, features mtic and mtoc are no longer supported “under the current memory manager“. Sometime around 2010 the mtic and mtoc features were completely removed. Users of R2008b and newer releases therefore need to use the internal structs returned by the memory function, and/or use the profiler’s memory-monitoring feature. If you ask me, using mtic/mtoc was much simpler and easier. I for one miss these features.

In a related matter, if we wish to monitor Java’s memory used within Matlab, we are in a bind, because there are no built-in tools to help us. there are several JVM switches that can be turned on in the java.opts file: -Xrunhprof[:help]|[:option=value,...], -Xprof, -Xrunprof, -XX:+PrintClassHistogram and so on. There are several memory-monitoring (so-called “heap-walking”) tools: the standard JDK jconsole, jmap, jhat and jvisualvm (with its useful plugins) provide good basic coverage. MathWorks has posted a tutorial on using jconsole with Matlab. There are a number of other third-party tools such as JMP (for JVMs 1.5 and earlier) or TIJMP (for JVM 1.6). Within Matlab, we can use utilities such as Classmexer to estimate a particular object’s size (both shallow and deep referencing), or use java.lang.Runtime.getRuntime()‘s methods (maxMemory(), freeMemory() and totalMemory()) to monitor overall Java memory (sample usage).

Specifically in R2011b (but in no other release), we can also use a built-in Java memory monitor. Unfortunately, this simple and yet useful memory monitor was removed in R2012a (or maybe it was just moved to another package and I haven’t found out where… yet…):

com.mathworks.xwidgets.JavaMemoryMonitor.invoke

Matlab R2011b's Java memory monitor

As I have already noted quite often, using undocumented Matlab features and functions carries the risk that they will not be supported in some future Matlab release. Today’s article is a case in point.

1. Matlab-Java memory leaks, performance Internal fields of Java objects may leak memory - this article explains how to avoid this without sacrificing performance. ...
2. Matlab’s internal memory representation Matlab's internal memory structure is explored and discussed. ...
3. Matlab mex in-place editing Editing Matlab arrays in-place can be an important technique for optimizing calculations. This article shows how to do it using Mex. ...
4. Types of undocumented Matlab aspects This article lists the different types of undocumented/unsupported/hidden aspects in Matlab...

Editing Matlab arrays in-place can be an important technique for optimizing calculations, especially when handling data that use large blocks of memory. The Matlab language itself has some limited support for in-place editing, but when we are really concerned with speed we often turn to writing C/C++ extensions using the Mex interface. Unfortunately, editing arrays in-place from Mex extensions is not officially supported in Matlab, and doing it incorrectly can cause data inconsistencies or can even cause Matlab to crash. In this article, I will introduce the problem and show a simple solution that exhibit the basic implementation details of Matlab’s internal copy-on-write mechanism.

Why edit in-place?

To demonstrate the techniques in this article, I use the fast_median function, which is part of my nth_element package on Matlab’s File Exchange. You can download the package and play with the code if you want. The examples are fairly self-explanatory, so if you do not want to try the code you should be okay just following along.

Let us try a few function calls to see how editing in-place can save time and memory:

>> A = rand(100000000, 1);
>> tic; median(A); toc    
Elapsed time is 4.122654 seconds.
 
>> tic; fast_median(A); toc
Elapsed time is 1.646448 seconds.
 
>> tic; fast_median_ip(A); toc
Elapsed time is 0.927898 seconds.

If you try running this, be careful not to make A too large; tune the example according to the memory available on your system. In terms of the execution time for the different functions, your mileage may vary depending on factors such as: your system, what Matlab version you are running, and whether your test data is arranged in a single vector or a multicolumn array.

This example illustrates a few general points: first, fast_median is significantly faster than Matlab’s native median function. This is because fast_median uses a more efficient algorithm; see the nth_element page for more details. Besides being a shameless plug, this demonstrates why we might want to write a Mex function in the first place: rewriting the median function in pure Matlab would be slow, but using C++ we can significantly improve on the status quo.

The second point is that the in-place version, fast_median_ip, yields an additional speed improvement. What is the difference? Let us look behind the scenes; here are the CPU and memory traces from my system monitor after running the above:

Memory and CPU usage for median vs. fast_median_ip

You can see four spikes in CPU use, and some associated changes in memory allocation:

The first spike in CPU is when we created the test data vector; memory use also steps up at that time.

The second CPU spike is the largest; that is Matlab’s median function. You can see that over that period memory use stepped up again, and then stepped back down; the median function makes a copy of the entire input data, and then throws its copy away when it is finished; this is expensive in terms of time and resources.

The fast_median function is the next CPU spike; it has a similar step up and down in memory use, but it is much faster.

Finally, in the case of fast_median_ip we see something different; there is a spike in CPU use, but memory use stays flat; the in-place version is faster and more memory efficient because it does not make a copy of the input data.

There is another important difference with the in-place version; it modifies its input array. This can be demonstrated simply:

>> A = randi(100, [10 1]);
>> A'
ans = 39    42    98    25    64    75     6    56    71    89
 
>> median(A)
ans = 60
 
>> fast_median(A)
ans = 60
>> A'
ans = 39    42    98    25    64    75     6    56    71    89
 
>> fast_median_ip(A)
ans = 60
>> A'
ans = 39     6    25    42    56    64    75    71    98    89

As you can see, all three methods get the same answer, but median and fast_median do not modify A in the workspace, whereas after running fast_median_ip, input array A has changed. This is how the in-place method is able to run without using new memory; it operates on the existing array rather than making a copy.

Pitfalls with in-place editing

Modifying a function’s input is common in many languages, but in Matlab there are only a few special conditions under which this is officially sanctioned. This is not necessarily a bad thing; many people feel that modifying input data is bad programming practice and makes code harder to maintain. But as we have shown, it can be an important capability to have if speed and memory use are critical to an application.

Given that in-place editing is not officially supported in Matlab Mex extensions, what do we have to do to make it work? Let us look at the normal, input-copying fast_median function as a starting point:

void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]) {
   mxArray *incopy = mxDuplicateArray(prhs[0]);
   plhs[0] = run_fast_median(incopy);
}

This is a pretty simple function (I have taken out a few lines of boiler plate input checking to keep things clean). It relies on helper function run_fast_median to do the actual calculation, so the only real logic here is copying the input array prhs[0]. Importantly, run_fast_median edits its inputs in-place, so the call to mxDuplicateArray ensures that the Mex function is overall well behaved, i.e. that it does not change its inputs.

Who wants to be well behaved though? Can we save time and memory just by taking out the input duplication step? Let us try it:

void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]) {
   plhs[0] = run_fast_median(const_cast<mxarray *>(prhs[0]));  // </mxarray>
}

Very bad behavior; note that we cast the original const mxArray* input to a mxArray* so that the compiler will let us mess with it; naughty.

But does this accomplish edit in-place for fast_median? Be sure to save any work you have open and then try it:

>> mex fast_median_tweaked.cpp
>> A = randi(100,[10 1]);
>> fast_median_tweaked(A)
ans = 43

Hmm, it looks like this worked fine. But in fact there are subtle problems:

>> A = randi(100,[10 1]);
>> A'
ans = 65    92    14    26    41     2    45    85    53     2
>> B = A;
>> B'
ans = 65    92    14    26    41     2    45    85    53     2
 
>> fast_median_tweaked(A)
ans = 43
>> A'
ans = 2     2    41    26    14    45    65    85    53    92
>> B'
ans = 2     2    41    26    14    45    65    85    53    92

Uhoh, spooky; we expected that running fast_median_tweaked would change input A, but somehow it has also changed B, even though B is supposed to be an independent copy. Not good. In fact, under some conditions this kind of uncontrolled editing in-place can crash the entire Matlab environment with a segfault. What is going on?

Matlab’s copy-on-write mechanism

The answer is that our simple attempt to edit in-place conflicts with Matlab’s internal copy-on-write mechanism. Copy-on-write is an optimization built into Matlab to help avoid expensive copying of variables in memory (actually similar to what we are trying to accomplish with edit in-place). We can see copy-on-write in action with some simple tests:

Matlab's Copy-on-Write memory and CPU usage

% Test #1: update, then copy
>> tic; A = zeros(100000000, 1); toc
Elapsed time is 0.588937 seconds.
>> tic; A(1) = 0; toc
Elapsed time is 0.000008 seconds.
>> tic; B = A; toc   
Elapsed time is 0.000020 seconds.
 
% Test #2: copy, then update
>> clear
>> tic; A = zeros(100000000, 1); toc
Elapsed time is 0.588937 seconds.
>> tic; B = A; toc   
Elapsed time is 0.000020 seconds.
>> tic; A(1) = 0; toc
Elapsed time is 0.678160 seconds.

In the first set of operations, time and memory are used to create A, but updating A and “copying” A into B take no memory and essentially no time. This may come as a surprise since supposedly we have made an independent copy of A in B; why does creating B take no time or memory when A is clearly a large, expensive block?

The second set of operations makes things more clear. In this case, we again create A and then copy it to B; again this operation is fast and cheap. But assigning into A at this point takes time and consumes a new block of memory, even though we are only assigning into a single index of A. This is copy-on-write: Matlab tries to save you from copying large blocks of memory unless you need to. So when you first assign B to equal A, nothing is copied; the variable B is simply set to point to the same memory location already used by A. Only after you try to change A (or B), does Matlab decide that you really need to have two copies of the large array.

There are some additional tricks Matlab does with copy-on-write. Here is another example:

>> clear
>> tic; A{1} = zeros(100000000, 1); toc
Elapsed time is 0.573240 seconds.
>> tic; A{2} = zeros(100000000, 1); toc
Elapsed time is 0.560369 seconds.
 
>> tic; B = A; toc                     
Elapsed time is 0.000016 seconds.
 
>> tic; A{1}(1) = 0; toc               
Elapsed time is 0.690690 seconds.
>> tic; A{2}(1) = 0; toc
Elapsed time is 0.695758 seconds.
 
>> tic; A{1}(1) = 0; toc
Elapsed time is 0.000011 seconds.
>> tic; A{2}(1) = 0; toc
Elapsed time is 0.000004 seconds.

This shows that for the purposes of copy-on-write, different elements of cell array A are treated independently. When we assign B equal to A, nothing is copied. Then when we change any part of A{1}, that whole element must be copied over. When we subsequently change A{2}, that whole element must also be copied over; it was not copied earlier. At this point, A and B are truly independent of each other, as both elements have experienced copy-on-write, so further assignments into either A or B are fast and require no additional memory.

Try playing with some struct arrays and you will find that copy-on-write also works independently for the elements of structs.

Reconciling edit in-place with copy-on-write: mxUnshareArray

Now it is clear why we cannot simply edit arrays in-place from Mex functions; not only is it naughty, it fundamentally conflicts with copy-on-write. Naively changing an array in-place can inadvertently change other variables that are still waiting for a copy-on-write, as we saw above when fast_median_tweaked inadvertently changed B in the workspace. This is, in the best case, an unmaintainable mess. Under more aggressive in-place editing, it can cause Matlab to crash with a segfault.

Luckily, there is a simple solution, although it requires calling internal, undocumented Matlab functions.

Essentially what we need is a Mex function that can be run on a Matlab array that will do the following:

1. Check whether the current array is sharing data with any other arrays that are waiting for copy-on-write.
2. If the array is shared, it must be unshared; the underlying memory must be copied and all the relevant pointers need to be fixed so that the array we want to work on is no longer accessible by anyone else.
3. If the array is not currently shared, simply proceed; the whole point is to avoid copying memory if we do not need to, so that we can benefit from the efficiencies of edit in-place.

If you think about it, this is exactly the operation that Matlab needs to run internally when it is deciding whether an assignment operation requires a copy-on-write. So it should come as no surprise that such a Mex function already exists in the form of a Matlab internal called mxUnshareArray. Here is how you use it:

extern "C" bool mxUnshareArray(mxArray *array_ptr, bool noDeepCopy);
 
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]) {
   mxUnshareArray(const_cast<mxarray *>(prhs[0]), true);  //</mxarray>
   plhs[0] = run_fast_median(const_cast<mxarray *>(prhs[0]));  //</mxarray>
}

This is the method actually used by fast_median_ip to efficiently edit in-place without risking conflicts with copy-on-write. Of course, if the array turns out to need to be unshared, then you do not get the benefit of edit in-place because the memory ends up getting copied. But at least things are safe and you get the in-place benefit as long as the input array is not being shared.

Further topics

The method shown here should allow you to edit normal Matlab numerical or character arrays in-place from Mex functions safely. For a Mex function in C rather than C++, omit the “C” in the extern declaration and of course you will have to use C-style casting rather than const_cast. If you need to edit cell or struct arrays in-place, or especially if you need to edit subsections of shared cell or struct arrays safely and efficiently while leaving the rest of the data shared, then you will need a few more tricks. A good place to get started is this article by Benjamin Schubert.

Unfortunately, over the last few years Mathworks seems to have decided to make it more difficult for users to access these kinds of internal methods to make our code more efficient. So be aware of the risk that in some future version of Matlab this method will no longer work in its current form.

Ultimately much of what is known about mxUnshareArray as well as the internal implementation details of how Matlab keeps track of which arrays are shared goes back to the work of Peter Boettcher, particularly his headerdump.c utility. Unfortunately, it appears that HeaderDump fails with Matlab releases >=R2010a, as Mathworks have changed some of the internal memory formats – perhaps some smart reader can pick up the work and adapt HeaderDump to the new memory format.

In a future article, I hope to discuss headerdump.c and its relevance for copy-on-write and edit in-place, and some other related tools for the latest Matlab releases that do not support HeaderDump.

1. Matlab’s internal memory representation Matlab's internal memory structure is explored and discussed. ...
2. Profiling Matlab memory usage mtic and mtoc were a couple of undocumented features that enabled users of past Matlab releases to easily profile memory usage. ...
3. Matlab-Java memory leaks, performance Internal fields of Java objects may leak memory - this article explains how to avoid this without sacrificing performance. ...
4. JMI – Java-to-Matlab Interface JMI enables calling Matlab functions from within Java. This article explains JMI's core functionality....

The problem: “Matlab crashes” – now go figure…

A client of one of my Matlab programs recently complained that Matlab crashes after several hours of extensive use of the program. The problem looked like something that is memory related (messages such as Matlab’s out-of-memory error or Java’s heap-space error). Apparently this happens even on 64-bit systems having lots of memory, where memory should never be a problem.

Well, we know that this is only in theory, but in practice Matlab’s internal memory management has problems that occasionally lead to such crashes. This is one of the reasons, by the way, that recent Matlab releases have added the preference option of increasing the default Java heap space (the previous way to do this was a bit complex). Still, even with a high Java heap space setting and lots of RAM, Matlab crashed after using my program for several hours.

Not pleasant at all, even a bit of an embarrassment for me. I’m used to crashing Matlab, but only as a result of my playing around with the internals – I would hate it to happen to my clients.

Finding the leak

While we can do little with Matlab’s internal memory manager, I started searching for the exact location of the memory leak and then to find a way to overcome it. I’ll save readers the description about the grueling task of finding out exactly where the memory leak occurred in a program that has thousands of lines of code and where events get fired asynchronously on a constant basis. Matlab Profiler’s undocumented memory profiling option helped me quite a bit, as well as lots of intuition and trial-and-error. Detecting memory leak is never easy, and I consider myself somewhat lucky this time to have both detected the leak source and a workaround.

It turned out that the leakage happens in a callback that gets invoked multiple times per second by a Java object (see related articles here and here). Each time the Matlab callback function is invoked, it reads the event information from the supplied Java event-data (the callback’s second input parameter). Apparently, about 1KB of memory gets leaked whenever this event-data is being read. This may appear a very small leak, but multiply this by some 50-100K callback invocations per hour and you get a leakage of 50-100MB/hour. Not a small leak at all; more of a flood you could say…

Using get()

The leakage culprit turned out to be the following code snippet:

% 160 uSecs per call, with memory leak
eventData  = get(hEventData,{'EventName','ParamNames','EventData'});
eventName  = eventData{1};
paramNames = eventData{2};
paramData  = eventData{3}.cell;

In this innocent-looking code, hEventData is a Java object that contains the EventName, ParamNames, EventData properties: EventName is a Java String, that is automatically converted by Matlab’s get() function into a Matlab string (char array); ParamNames is a Java array of Strings, that gets automatically converted into a Matlab cell-array of string; and EventData is a Java array of Objects that needs to be converted into a Matlab cell array using the built-in cell function, as described in one of my recent articles.

The code is indeed innocent, works really well and is actually extremely fast: each invocation takes of this code segment takes less than 0.2 millisecs. Unfortunately, because of the memory leak I needed to find a better alternative.

Using handle()

The first idea was to use the built-in handle() function, under the assumption that it would solve the memory leak, as reported here. In fact, MathWorks specifically advises to use handle() rather than to work with “naked” Java objects, when setting Java object callbacks. The official documentation of the set function says:

Do not use the set function on Java objects as it will cause a memory leak.

It stands to reason then that a similar memory leak happens with get and that a similar use of handle would solve this problem:

% 300 uSecs per call, with memory leak
s = get(handle(hEventData));
eventName  = s.EventName;
paramNames = s.ParamNames;
paramData  = cell(s.EventData);

Unfortunately, this variant, although working correctly, still leaks memory, and also performs almost twice as worse than the original version, taking some 0.3 milliseconds to execute per invocation. Looks like this is a dead end.

Using Java accessor methods

The next attempt was to use the Java object’s internal accessor methods for the requested properties. These are public methods of the form getXXX(), isXXX(), setXXX() that enable Matlab to treat XXX as a property by its get and set functions. In our case, we need to use the getter methods, as follows:

% 380 uSecs per call, no memory leak
eventName  = char(hEventData.getEventName);
paramNames = cell(hEventData.getParamNames);
paramData  = cell(hEventData.getEventData);

Here, the method getEventName() returns a Java String, that we convert into a Matlab string using the char function. In our previous two variants, the get function did this conversion for us automatically, but when we use the Java method directly we need to convert the results ourselves. Similarly, when we call getParamNames(), we need to use the cell function to convert the Java String[] array into a Matlab cell array.

This version at last doesn’t leak any memory. Unfortunately, it has an even worse performance: each invocation takes almost 0.4 milliseconds. The difference may seem insignificant. However, recall that this callback gets called dozens of times each second, so the total adds up quickly. It would be nice if there were a faster alternative that does not leak any memory.

Using struct()

Luckily, I found just such an alternative. At 0.24 millisecs per invocation, it is almost as fast as the leaky best-performance original get version. Best of all, it leaks no memory, at least none that I could detect.

The mechanism relies on the little-known fact that public fields of Java objects can be retrieved in Matlab using the built-in struct function. For example:

>> fields = struct(java.awt.Rectangle)
fields = 
             x: 0
             y: 0
         width: 0
        height: 0
      OUT_LEFT: 1
       OUT_TOP: 2
     OUT_RIGHT: 4
    OUT_BOTTOM: 8
 
>> fields = struct(java.awt.Dimension)
fields = 
     width: 0
    height: 0

Note that this useful mechanism is not mentioned in the main documentation page for accessing Java object fields, although it is indeed mentioned in another doc-page – I guess this is a documentation oversight.

In any case, I converted my Java object to use public (rather than private) fields, so that I could use this struct mechanism (Matlab can only access public fields). Yes I know that using private fields is a better programming practice and all that (I’ve programmed OOP for some 15 years…), but sometimes we need to do ugly things in the interest of performance. The latest version now looks like this:

% 240 uSecs per call, no memory leak
s = struct(hEventData);
eventName  = char(s.eventName);
paramNames = cell(s.paramNames);
paramData  = cell(s.eventData);

This solved the memory leakage issue for my client. I felt fortunate that I was not only able to detect Matlab’s memory leak but also find a working workaround without sacrificing performance or functionality.

In this particular case, I was lucky to have full control over my Java object, to be able to convert its fields to become public. Unfortunately, we do not always have similar control over the object that we use, because they were coded by a third party.

By the way, Matlab itself uses this struct mechanism in its code-base. For example, Matlab timers are implemented using Java objects (com.mathworks.timer.TimerTask). The timer callback in Matlab code converts the Java timer event data into a Matlab struct using the struct function, in %matlabroot%/toolbox/matlab/iofun/@timer/timercb.m. The users of the timer callbacks then get passed a simple Matlab EventData struct without ever knowing that the original data came from a Java object.

As an interesting corollary, this same struct mechanism can be used to detect internal properties of Matlab class objects. For example, in the timers again, we can get the underlying timer’s Java object as follows (note the highlighted warning, which I find a bit ironic given the context):

>> timerObj = timerfind
 
   Timer Object: timer-1
 
   Timer Settings
      ExecutionMode: singleShot
             Period: 1
           BusyMode: drop
            Running: off
 
   Callbacks
           TimerFcn: @myTimerFcn
           ErrorFcn: ''
           StartFcn: ''
            StopFcn: ''
 
>> timerFields = struct(timerObj)
Warning: Calling STRUCT on an object prevents the object from hiding its implementation details and should thus be avoided.Use DISP or DISPLAY to see the visible public details of an object. See 'help struct' for more information.(Type "warning off MATLAB:structOnObject" to suppress this warning.)timerFields = 
         ud: {}
    jobject: [1x1 javahandle.com.mathworks.timer.TimerTask]

1. Profiling Matlab memory usage mtic and mtoc were a couple of undocumented features that enabled users of past Matlab releases to easily profile memory usage. ...
2. Preallocation performance Preallocation is a standard Matlab speedup technique. Still, it has several undocumented aspects. ...
3. Performance: scatter vs. line In many circumstances, the line function can generate visually-identical plots as the scatter function, much faster...
4. Matlab-Java interface using a static control The switchyard function design pattern can be very useful when setting Matlab callbacks to Java GUI controls. This article explains why and how....

Numeric data array

In some cases, namely Java numeric arrays, Matlab also automatically converts the Java array into Matlab arrays. This is actually inconvenient when we would like to access the original Java reference in order to modify some value – since the Java reference is inaccessible from Matlab in this case, the data is immutable.

>> jColor = java.awt.Color.red
jColor =
java.awt.Color[r=255,g=0,b=0]
 
>> matlabData = jColor.getColorComponents([])
matlabData =
     1
     0     % < = immutable array of numbers, not a reference to int[]
     0

Non-numeric array

Very often we encounter cases in Java where the information is stored in an array of non-numeric data. In such cases we need to apply a non-automatic conversion from Java into Matlab.

If the objects are of exactly the same type, then we could store them in a simple Matlab array; otherwise (as can be seen in the example below), we could store them in either a simple array of handles, or in a simple cell array:

>> jFrames = java.awt.Frame.getFrames
jFrames =
java.awt.Frame[]:
    [javax.swing.SwingUtilities$SharedOwnerFrame ]
    [com.mathworks.mde.desk.MLMainFrame          ]
    [com.mathworks.mde.desk.MLMultipleClientFrame]
    [com.mathworks.mwswing.MJFrame               ]
 
% Alternative #1 - use a loop
>> mFrames = handle([]); for idx = 1 : length(jFrames); mFrames(idx)=handle(jFrames(idx)); end
>> mFrames
mFrames =
	handle: 1-by-4
>> mFrames(1)
ans =
	javahandle.javax.swing.SwingUtilities$SharedOwnerFrame
>> mFrames(2)
ans =
	javahandle.com.mathworks.mde.desk.MLMainFrame
 
% Alternative #2a - convert into a Matlab cell array
>> mFrames = jFrames.cell
mFrames = 
    [1x1 javax.swing.SwingUtilities$SharedOwnerFrame ]
    [1x1 com.mathworks.mde.desk.MLMainFrame          ]
    [1x1 com.mathworks.mde.desk.MLMultipleClientFrame]
    [1x1 com.mathworks.mwswing.MJFrame               ]
 
% Alternative #2b - convert to a cell array (equivalent variant of alternative 2a)
>> mFrames = cell(jFrames);

Note that if we only need to access a particular item in the Java vector or array, we could do that directly, without needing to convert the entire data into Matlab first. Simply use jFrames(1) to directly access the first item in the jFrames array, for example.

(note: Java Frames are discussed in chapters 7 and 8 of my Matlab-Java book).

Vectors and other Collections

Very often we encounter cases in Java where the information is stored in a Java Collection rather than in a simple Java array. The basic mechanism for the conversion in this case is to first convert the Java data into a simple Java array (in cases it was not so in the first place), and then to convert this into a Matlab array using either the automated conversion (if the data is numeric), or using a for loop (ugly and slow!), or into a cell array using the cell function, as explained above.

Different Collections have different manners of converting into a Java array: some Collections return an Iterator/Enumerator that can be processed in a loop (be careful not to reset the iterator reference by re-reading it within the loop):

% Wrong way - causes an infinite loop
idx = 1;
props = java.lang.System.getProperties;
while props.elements.hasMoreElements
    mPropValue{idx} = props.elements.nextElement;
end
 
% Right way
idx = 1;
propValues = java.lang.System.getProperties.elements;  % Enumerator
while propValues.hasMoreElements
    mPropValue{idx} = propValues.nextElement;
end

(note: system properties are discussed in section 1.9 of my Matlab-Java book; Collections are discussed in section 2.1)

Other Collections, such as java.util.Vector, have a toArray() method that directly converts into a Java array, and we can process from there as described above:

>> jVector = java.util.Vector;
>> jVector.add(1); jVector.add(2); jVector.add(3);
>> jVector.addAll(jv); jVector.addAll(jv);
>> jVector
jVector =
[1.0, 2.0, 3.0, 1.0, 2.0, 3.0, 1.0, 2.0, 3.0, 1.0, 2.0, 3.0]
 
% Now convert into a Matlab cell array via a Java simple array
>> mCellArray = jVector.toArray.cell
mCellArray = 
    [1]
    [2]
    [3]
    [1]
    [2]
    [3]
    [1]
    [2]
    [3]
    [1]
    [2]
    [3]

Performance

It so happens, that the undocumented built-in feature function (or its near-synonym system_dependent) enables improved performance in this conversion process. feature(44) accepts a java.util.Vector and converts it directly into a Matlab cell-array, in one third to one-half the time that it would take the equivalent toArray.cell() (the third input argument is the number of columns in the result - the reshaping is done automatically):

>> mCellArray = feature(44,jVector,jVector.size)   % jVector.size = 12
mCellArray = 
    [1]    [2]    [3]    [1]    [2]    [3]    [1]    [2]    [3]    [1]    [2]    [3]
 
>> mCellArray = feature(44,jVector,4)
mCellArray = 
    [1]    [1]    [1]    [1]
    [2]    [2]    [2]    [2]
    [3]    [3]    [3]    [3]

The conversion process is pretty efficient: On my system, the regular toArray.cell() takes 0.45 seconds for a 100K vector, compared to 0.21 seconds for the feature alternative. However, this small difference could be important in cases where performance is crucial, for example in processing of highly-active Java events in Matlab callbacks, or when retrieving data from a database. And this latter case is indeed where a sample usage of this feature can be found, namely in the cursor.fetch.m function (where it appears as system_dependent(44)).

Please note that both feature and system_dependent are highly prone to change without prior warning in some future Matlab release. On the other hand, the conversion methods that I presented above, excluding feature, will probably still be valid in all Matlab releases in the near future.

1. Matlab-Java memory leaks, performance Internal fields of Java objects may leak memory - this article explains how to avoid this without sacrificing performance. ...
2. Matlab callbacks for Java events Events raised in Java code can be caught and handled in Matlab callback functions - this article explains how...
3. Java stack traces in Matlab Matlab does not normally provide information about the Java calls on the stack trace. A simple trick can show us this information....
4. JBoost – Integrating an external Java library in Matlab This article shows how an external Java library can be integrated in Matlab...

Today I wish to share a related experience that happened to me yesterday, when I needed to improve the performance of a client’s application. When profiling the application, I found that a major performance hotspot was a repeated call to the datestr function, each time with several thousand date items.

datestr is the opposite function of datenum: it receives date values and returns date strings. Unlike datenum, however, datestr does not use a highly optimized native-code library function that we could use directly. Instead, it loops over all date values and sequentially applies the requested string pattern.

The natural reaction in such a case would perhaps be to vectorize the code (something that MathWorks should have done in the first place I guess). But in this case I used a different solution, that I would like to share today:

In any programming languages, Matlab included, the most effective performance tip is to cache processing results. Caching often makes the code slightly more complex and less maintainable, but the performance benefits are immediate and significant. In Matlab, benefits of caching can often surpass even those of vectorization (using both vectorization and caching is of course even better).

In the case of datestr, if we can be certain of the precondition that the output string format is the same, we can cache the results, and even use vectorization. In my case, I plotted historical daily stock quotes data and so I was assured that (1) all dates are integers and that (2) I always use the same date-string format ‘dd-mmm-yyyy’.

First, let’s define the wrapper function datestr2 with the necessary caching and vectorization:

% datestr2 - faster variant of datestr, for integer date values since 1/1/2000
function dateStrs = datestr2(dateVals,varargin)
 
  persistent dateStrsCache
  persistent dateValsCache
 
  if isempty(dateStrsCache)
      origin = datenum('1-Jan-2000');
      dateValsCache = origin:(now+100);
      dateStrsCache = datestr(dateValsCache,varargin{:});
  end
 
  [tf,loc] = ismember(dateVals, dateValsCache);
  if all(tf)
      dateStrs = dateStrsCache(loc,:);
  else
      dateStrs = datestr(dateVals,varargin{:});
  end
end  % datestr2

As can be seen, the first time that datestr2 is called, it computes and caches all datestr values for all the dates since Jan 1, 2000. Subsequent calls to datestr2 simply retrieve the relevant cache values. Note that the input date entries need not be sorted.

In case that an input date number is not found in the cache, datestr2 automatically falls-back to using the built-in datestr for the entire input list. This could of course be improved to add the new entries to the cache – I leave this as a reader exercise.

The bottom line was a 150-times (!!!) speed improvement for a 1000-item date vector (50mS => 0.3mS on my system):

% Prepare a 1000-vector of dates, starting 3 years ago until today
>> dateVals = fix(now)+(-1000:0);
 
% Run the standard datestr function => 50mS
>> tic; s1=datestr(dateVals); toc
Elapsed time is 0.049089 seconds.
>> tic; s1=datestr(dateVals); toc
Elapsed time is 0.048086 seconds.
 
% Now run our datestr2 function (caching already done before) => 0.3 mS
>> tic; s2=datestr2(dateVals); toc
Elapsed time is 0.222031 seconds.   % initial cache preparation takes 222 mS
>> tic; s2=datestr2(dateVals); toc
Elapsed time is 0.000313 seconds.   % subsequent datestr2 calls take 0.3 mS
>> tic; s2=datestr2(dateVals); toc
Elapsed time is 0.000296 seconds.
 
% Ensure that the two functions give exactly the same results
>> isequal(s1,s2)
ans =
     1

So what have we learned from this?

1. To improve performance we must profile the code. Very often the performance bottlenecks occur in non-intuitive very specific places that can be surgically handled without requiring any major redesign. In this case, I simply had to replace calls to datestr with datestr2 in the application’s code.
2. Vectorization is not always as cost-effective as caching
3. Major performance improvements do NOT necessarily involve undocumented functions or tricks: In fact, today’s post about caching uses fully-documented pure-Matlab code.
4. Different performance hotspots can have different solutions: caching, vectorization, internal library functions, undocumented graphics properties, smart property selection, smart function selection, smart indexing, smart parameter selection etc.

In a later post I will show how similar modifications to internal Matlab functions can dramatically improve the performance of the uitable function. Anyone who has tried using uitable with more than a few dozen cells will surely understand why this is important…

Do you have a favorite performance trick not mentioned above? If so, please post a comment.

1. Performance: scatter vs. line In many circumstances, the line function can generate visually-identical plots as the scatter function, much faster...
2. Plot performance Undocumented inner plot mechanisms can be used to significantly improved plotting performance...
3. Datenum performance The performance of the built-in Matlab function datenum can be significantly improved by using an undocumented internal help function...
4. Preallocation performance Preallocation is a standard Matlab speedup technique. Still, it has several undocumented aspects. ...

Roy’s translated post: “Anyone who adds, detracts (from execution time)”

In the story of Eve and the serpent, the first woman told the serpent about the prohibition of eating from the Tree of Knowledge, adding to that prohibition a ban on touching the tree (something that God has not commanded). The snake used this inaccuracy in her words, showing her that one can touch the tree without fear, and therefore argued that the prohibition to eat its fruit is similarly not true. As a result, Eve was tempted to eat the fruit, and the rest is known. Jewish sages said of the imaginary prohibition which Eve has added, that this is an example where “Anyone who adds, [in effect] detracts”.

Recently I [Roy] came across an interesting phenomenon, that in MATLAB, adding elements to a vector on which an action is performed, does not degrade the execution time, but rather the reverse. Adding vector elements actually reduces execution time!

Here’s an example. Try to rank the following tic-toc segments from fastest to slowest:

x = rand(1000,1000);
 
% Segment #1
y = ones(1000,1000);
tic
for i=1:100
    y = x .* y;
end
toc
 
% Segment #2
y=ones(1000,1000);
tic
for i=1:100
    y(:,1:999) = x(:,1:999) .* y(:,1:999);
end
toc
 
% Segment #3
y=ones(1000,1000);
tic
for i=1:100
    y(1:999,:) = x(1:999,:) .* y(1:999,:);
end
toc

The first loop multiplies all the elements of the x and y matrices, and should therefore run longer than the other loops, which multiply matrices that are one row or one column smaller. However, in practice, the first loop was the fastest – just 0.25 seconds on my computer, whereas the second ran for 1.75 seconds, and the third – 6.65 seconds [YMMV].

Why is the first loop the fastest?

The subscription operation performed in each of the latter two loops is a wasteful action, and therefore in such cases I would suggest that you run your operation on the full matrix, and then get rid of the unnecessary row or column.

And why does the second loop run faster than the third?

This is related to the fact that MATLAB prefers operations on columns rather than rows. In the second loop, all the elements are multiplied except those in the last column, while in the third loop all the elements that have been extracted from all rows are multiplied, except for the last row.

In your work with MATLAB, have you encountered similar phenomena that are initially counter-intuitive, such as the example described above? If so, please post a comment below, or directly on Roy’s blog.

Is all of this undocumented? I really don’t know. But it is certainly unexpected and interesting…

1. Preallocation performance Preallocation is a standard Matlab speedup technique. Still, it has several undocumented aspects. ...
2. Performance: scatter vs. line In many circumstances, the line function can generate visually-identical plots as the scatter function, much faster...
3. Plot performance Undocumented inner plot mechanisms can be used to significantly improved plotting performance...
4. datestr performance Caching is a simple and very effective means to improve code performance, as demonstrated for the datestr function....

Since that time I have found a few additional related titbits that I would like to share:

cpucount

It appears that the CPU count value returned by tic is also returned by the internal executable application cpucount (or cpucount.exe), which is located beneath the bin folder of the Matlab installation (for example: C:\Program Files\Matlab\R2011a\bin\win32\cpucount.exe or /bin/matlab/R2011a/bin/glnx86/cpucount):

>> !cpucount
7144479469070 
 
>> uint64(str2num(evalc('!cpucount')))
ans =
        7144486156548
 
>> tStart = tic
tStart =
        7144497276916
 
>> uint64(str2num(evalc('!cpucount'))) - tic
ans =
                    0

Note that the cpucount application should not be confused with the built-in cputime function, which returns the total CPU time (in seconds) used by the current Matlab process.

>> cputime
ans =
                 210.15625

Note that the CPU count value itself should typically not be used to profile performance, but rather as a unique seed for random numbers and for multiple independent tic/toc invocations. This reminds me of a discussion that arose a decade ago, about the flops function’s removal in Matlab 6.0, when LAPACK was introduced. There are far better ways today for code profiling.

Anyway, the value reported by both cpucount and tic, is also returned by feature('timing','winperfcount'). Which brings us to:

feature(‘timing’)

A related tidbit is the ‘timing’ option of the built-in undocumented feature function, which I explored last year:

>> feature('timing')
??? Error using ==> feature
Choose second argument from:
    'resolution_tictoc'  - Resolution of Tic/Toc clock in sec (double)
    'overhead_tictoc'    - Overhead of Tic/Toc command in sec (double)
    'cpucount'           - Current CPU cycles used (uint64) [Using utCPUcount]
    'getcpuspeed_tictoc' - Stored CPU cycles/sec (double) [Used by tic/toc]
    'cpuspeed'           - Current CPU cycles/sec (double) [Simple MathWorks]
    'winperfcount'       - Current CPU cycles used (uint64) [Windows call]
    'winperfspeed'       - Current CPU cycles/sec (double) [Windows call]
    'wintime'            - Current Windows time (uint32) [Windows call]
                           units: msec since startup [Wraps]
    'wintimeofday'       - Current time of day converted to file time (unit64)[Windows call]
                           units: 100 nsec since 01-Jan-1601 (UTC)
    'clocks_per_sec'     - clock() speed in cycles/sec [CLOCKS_PER_SEC]
Choose second and third arguments from:
    'cpuspeed', double num             - Current CPU cycles/sec (double) [MathWorks - num iterations]
    'setcpuspeed_tictoc', double speed - Set the CPU cycles/sec [Used by tic/toc]
     uint64 arg2, uint64 arg3          - uint64 difference = arg2 - arg3 (uint64)
 
>> feature('timing','resolution_tictoc')
Resolution of Tic/Toc clock is 1.676191e-006 sec.
 
>> feature('timing','overhead_tictoc')
Overhead of Tic/Toc command is 1.676191e-006 sec.
 
>> feature('timing','cpucount')
CPU counter is 17548354355882 cycles.
 
>> feature('timing','getcpuspeed_tictoc')
tic/toc CPU speed is 2.533333e+009 cycles/sec.
 
>> feature('timing','cpuspeed')
Current CPU speed is 2.535690e+009 cycles/sec.
 
>> feature('timing','winperfcount')
Counter is 7150537513228 cycles.
 
>> feature('timing','winperfspeed')
Speed is 3.579545e+006 cycles/sec.
 
>> feature('timing','wintime')
Time is 1997625531 msec.
 
>> feature('timing','wintimeofday')
Time is 129543477454370000 units (100 nsec).
 
>> feature('timing','clocks_per_sec')
clock() speed is 1.000000e+003 cycles/sec.
 
>> feature('timing','cpuspeed')
Current CPU speed is 2.535746e+009 cycles/sec.

All these features return a numeric value, if requested:

>> cpuspeed = feature('timing','cpuspeed')
cpuspeed =
          2535702174.31193

In fact, feature('timing','cpucount') is used internally by the built-in tempname function, to generate a unique temporary filename:

if usejava('jvm')
    tmp_name = fullfile(dirname, ['tp' strrep(char(java.util.UUID.randomUUID),'-','_')]);
else
    tmp_name = fullfile(dirname, ['tp' num2str(feature('timing','cpucount'))]);
end

feature('timing','wintime') returns the system time in nano-seconds, that can also be gotten directly via Java:

>> uint64(java.lang.System.nanoTime)
ans =
     1997449616528637

It may be interesting to learn the nuances between ‘wintime’, ‘wintimeofday’ and java.lang.System.currentTimeMillis, which returns yet another value. If anyone knows, please post a comment.

-timing startup option

Finally, note that Matlab has had a startup (command-line) option of -timing for a long time. This was documented up to R2009a, but removed from the documentation in R2009b, although the functionality remains to this day.

If Matlab is started with the -timing command-line option, it creates a log file of the time it took different segments of its startup process. Matlab displays the log file contents in the Matlab Command Window when initialization is done. This could help debug the numerous startup problems that users on a variety of platforms and system configurations report:

  Toolbox Path Cache read in 0.08 seconds.
  MATLAB Path initialized in 1.27 seconds.
 
Opening timing log C:\DOCUME~1\Yair\LOCALS~1\Temp\timing_log.9104 ..
    MATLAB Startup Performance Metrics (In Seconds)
 
total   item     gap      description         % Yair's comments/hunches
=====================================         % =================================
 0.00   0.00    0.00   MATLAB script          % A: Starting point
 5.70   5.70    0.00   main                   % B: (??? - why so long after the initial Matlab script started?)
 6.14   0.08    0.36   LM Startup             % C: License-manager check of the license validity (from B: 5.70+0.36+0.08=6.14)
 6.21   0.00    0.07   splash                 % D: Matlab splash screen (can be bypassed with the '-nosplash' startup option) - (from C: 6.14+0.07=6.21)
 6.32   0.00    0.11   mnSigInit              % E: (start the initialization part ???) - (from D: 6.21+0.11=6.32)
 7.84   1.49    0.02      InitSunVM           % F: Start the Java Virtual Machine (JVM) used by Matlab GUI (from E: 6.32+0.02+1.49=7.84)
12.66   1.79    3.03      PostVMInit          % G: End of JVM initialization (from F: 7.84+3.03+1.79=12.66)
12.67   6.35    0.01     mljInit              % H: Initialization of Matlab's Java portion (from E: 6.32+0.01+6.35=12.67)
13.64   0.97    0.00     StartDesktop         % I: Start to prepare & initialize the Matlab Desktop (workspace, editor etc.) - (from H: 12.67+0.97=13.64)
13.64   7.31    0.01   Java initialization    % J: ??? (from E: 6.32+0.01+7.31=13.64)
14.04   0.40    0.00   hgInitialize           % K: Handle-Graphics initialization (from J: 13.64+0.40=14.04)
15.13   0.98    0.12   psParser               % L: ??? (from K: 14.04+0.12+0.98=15.13)
15.67   0.07    0.46   cachepath              % M: Toolbox Path Cache initialization (from L: 15.13+0.46+0.07=15.67)
18.53   1.27    1.60     matlabpath           % N: Matlab path initialization (from M: 15.67+1.60+1.27=18.53)
21.54   0.00    3.01     matlabpath           % O: ??? (from N: 18.53+3.01=21.52)
36.78  23.14   13.64   Init Desktop           % P: I believe this indicates the end of the Desktop's init (from I: 13.64+23.14=36.78)
37.05  21.38    0.00   matlabrc               % Q: indicates the end (?) of the m-file (matlabrc.m) initialization process (from M: 15.67+21.38=37.05)
=====================================
Items shown account for 159.4% of total startup time [TIMER: 3 MHz]

In the above list, note the unexplained gaps between the items. I’m not sure I understand them correctly. I posted my hunches as comments next to the relevant lines. I am not sure in some items whether they refer to the start or the end of their associated functionality. If anyone has other ideas or insight that will improve understanding of this list, please share.

1. tic / toc – undocumented option Matlab's built-in tic/toc functions have an undocumented option enabling multiple nested clockings...
2. Undocumented feature() function Matlab's undocumented feature function enables access to some internal experimental features...
3. Undocumented scatter plot behavior The scatter plot function has an undocumented behavior when plotting more than 100 points: it returns a single unified patch object handle, rather than a patch handle for each specific...
4. Types of undocumented Matlab aspects This article lists the different types of undocumented/unsupported/hidden aspects in Matlab...

In both cases, the solution to the performance question can be found by simply using Matlab’s built-in profiler in order to extract just the core processing functionality. It is often found that in a particular situation there is no need for all the input arguments data validity checks, and under some known limitations we can indeed use the core functionality directly.

In the case of ismember, it turned out that if we are assured in advance that the input data are sorted non-sparse non-NaN values, then we can use the undocumented built-in helper functions ismembc or ismembc2 for much-improved performance over the standard ismember. Both ismembc and ismembc2 happen to be mex files, although this is not always the case for helper functions.

Our datenum case is very similar. It turns out that datenum uses the undocumented built-in helper function dtstr2dtnummx for the actual processing – converting a date from text to floating-point number. As I noted in my response to the StackOverflow question, we can directly use this helper function for improved performance: On my particular computer, dtstr2dtnummx is over 3 times faster than the standard datenum function:

% Fast - using dtstr2dtnummx
>> tic, for i=1:1000; dateNum=dtstr2dtnummx({'2010-12-12 12:21:12.123'},'yyyy-MM-dd HH:mm:ss'); end; dateNum,toc
dateNum =
          734484.514722222
Elapsed time is 0.218423 seconds.
 
% Slower - using datenum
>> tic, for i=1:1000; dateNum=datenum({'2010-12-12 12:21:12.123'},'yyyy-mm-dd HH:MM:SS'); end; dateNum,toc
dateNum =
          734484.514722222   % Same value as dtstr2dtnummx - good!
Elapsed time is 0.658352 seconds.   % 3x slower than dtstr2dtnummx - bad!

While the difference in timing may appear negligible, if you are using this function to parse a text file with thousands of lines, each with its own timestamp, then these seemingly negligible time differences quickly add up. Of course, this only makes sense to do if you find out (using the profiler again) that this date parsing is a performance hotspot in your particular application. It was indeed such a performance hotspot in one of my applications, as it apparently was also for the original poster on StackOverflow.

Like ismembc, dtstr2dtnummx is an internal mex function. On my Windows system it is located in C:\Program Files\Matlab\R2011a\toolbox\matlab\timefun\private\dtstr2dtnummx.mexw32. It will have a different extension non-Windows systems, but you will easily find it in its containing folder.

To gain access to dtstr2dtnummx, simply add its folder to the Matlab path using the addpath function, or copy the dtstr2dtnummx.mexw32 file to another folder that is already on your Matlab path.

Note that the string format is different between dtstr2dtnummx and datenum: In the test case above, dtstr2dtnummx used 'yyyy-MM-dd HH:mm:ss', while datenum required 'yyyy-mm-dd HH:MM:SS'. I have no idea why MathWorks did not keep consistent formatting strings. But because of this, we need to be extra careful (example1, example2). If you are interested in finding out how the datenum format strings translates into a dtstr2dtnummx, take a look at the helper function cnv2icudf, which is a very readable m-file located in the same folder as dtstr2dtnummx.

To those interested, the folder that contains dtstr2dtnummx also contains some other interesting date conversion functions, so explore and enjoy!

Perhaps the main lesson that can be learned from this article, and its ismembc predecessor of two years ago, is that it is very useful to profile the code for performance hotspots. When such a hotspot is found, don’t stop your profiling at the built-in Matlab functions – keep digging in the profiler results and perhaps you’ll find that you can improve performance by taking an internal shortcut.

Have you discovered any other performance shortcuts in a built-in Matlab function? If so, please post a comment to tell us all about it.

1. datestr performance Caching is a simple and very effective means to improve code performance, as demonstrated for the datestr function....
2. Plot performance Undocumented inner plot mechanisms can be used to significantly improved plotting performance...
3. Performance: scatter vs. line In many circumstances, the line function can generate visually-identical plots as the scatter function, much faster...
4. cellfun – undocumented performance boost Matlab's built-in cellfun function has an undocumented option to significantly improve performance in some cases....

I begin by stating the obvious: whenever possible, try to vectorize your code, preallocate the data and other performance-improving techniques suggested by Matlab. Unfortunately, sometimes (as in my specific case above) all these cannot help. Even in such cases, we can still find important performance tricks, such as these:

Performance hack #1: manual limits

Whenever Matlab updates plot data, it checks whether any modification needs to be done to any of its limits. This computation-intensive task is done for any limit that is set to ‘Auto’ mode, which is the default axes limits mode. If instead we manually set the axes limits to the requested range, Matlab skips these checks, enabling much faster plotting performance.

Let us simulate the situation by adding 500 data points to a plot, one at a time:

>> x=0:0.02:10; y=sin(x);
>> clf; cla; tic;
>> drawnow; plot(x(1),y(1)); hold on; legend data;
>> for idx = 1 : length(x); plot(x(idx),y(idx)); drawnow; end;
>> toc
 
Elapsed time is 21.583668 seconds.

simple plot with 500 data points

And now let’s use static axes limits:

>> x=0:0.02:10; y=sin(x);
>> clf; cla; tic; drawnow; 
>> plot(x(1),y(1)); hold on; legend data;
 
>> xlim([0,10]); ylim([-1,1]);  % static limits
 
>> for idx = 1 : length(x); plot(x(idx),y(idx)); drawnow; end;
>> toc
 
Elapsed time is 16.863090 seconds.

Note that this trick is the basis for the performance improvement that occurs when using the plot’s undocumented set of LimInclude properties.

Of course, setting manual limits prevents the axes limits from growing and shrinking automatically with the data, which can actually be a very useful feature sometimes. But if performance is important, we now know that we have this tool to improve it.

Performance hack #2: static legend

Hack #1 gave us a 22% performance boost, but we can do much better. Running the profiler on the code above we see that much of the time is spent recomputing the legend. Looking inside the legend code (specifically, the legendcolorbarlayout function), we detect several short-circuits that we can use to make the legend static and prevent recomputation:

>> x=0:0.02:10; y=sin(x);
>> clf; cla; tic; drawnow; 
>> plot(x(1),y(1)); hold on; legend data;
 
>> xlim([0,10]); ylim([-1,1]);  % static limits
 
>> % Static legend
>> set(gca,'LegendColorbarListeners',[]); 
>> setappdata(gca,'LegendColorbarManualSpace',1);
>> setappdata(gca,'LegendColorbarReclaimSpace',1);
 
>> for idx = 1 : length(x); plot(x(idx),y(idx)); drawnow; end;
>> toc
 
Elapsed time is 5.209053 seconds.

Now this is much much better – a 76% performance boost compared to the original plot (i.e., 4 times faster!). Of course, it prevents the legend from being dynamically updated. Sometimes we actually wish for this dynamic effect (last year I explained how to use the legend’s undocumented -DynamicLegend feature for even greater dynamic control). But when performance is important, we can still display a legend without its usual performance cost.

In conclusion, I have demonstrated that Matlab performance can often be improved significantly, even in the absence of any vectorization, by simply understanding the internal mechanisms and bypassing those which are irrelevant in our specific case.

Have you found other similar performance hacks? If so, please share them in the comments section below.

1. Plot LimInclude properties The plot objects' XLimInclude, YLimInclude, ZLimInclude, ALimInclude and CLimInclude properties are an important feature, that has both functional and performance implications....
2. Undocumented scatter plot behavior The scatter plot function has an undocumented behavior when plotting more than 100 points: it returns a single unified patch object handle, rather than a patch handle for each specific...
3. Controlling plot data-tips Data-tips are an extremely useful plotting tool that can easily be controlled programmatically....
4. Performance: scatter vs. line In many circumstances, the line function can generate visually-identical plots as the scatter function, much faster...