Editing Matlab arrays in-place can be an important technique for optimizing calculations, especially when handling data that use large blocks of memory. The Matlab language itself has some limited support for in-place editing, but when we are really concerned with speed we often turn to writing C/C++ extensions using the Mex interface. Unfortunately, editing arrays in-place from Mex extensions is not officially supported in Matlab, and doing it incorrectly can cause data inconsistencies or can even cause Matlab to crash. In this article, I will introduce the problem and show a simple solution that exhibit the basic implementation details of Matlab’s internal copy-on-write mechanism.

Why edit in-place?

To demonstrate the techniques in this article, I use the fast_median function, which is part of my nth_element package on Matlab’s File Exchange. You can download the package and play with the code if you want. The examples are fairly self-explanatory, so if you do not want to try the code you should be okay just following along.

Let us try a few function calls to see how editing in-place can save time and memory:

>> A = rand(100000000, 1);
>> tic; median(A); toc    
Elapsed time is 4.122654 seconds.
 
>> tic; fast_median(A); toc
Elapsed time is 1.646448 seconds.
 
>> tic; fast_median_ip(A); toc
Elapsed time is 0.927898 seconds.

If you try running this, be careful not to make A too large; tune the example according to the memory available on your system. In terms of the execution time for the different functions, your mileage may vary depending on factors such as: your system, what Matlab version you are running, and whether your test data is arranged in a single vector or a multicolumn array.

This example illustrates a few general points: first, fast_median is significantly faster than Matlab’s native median function. This is because fast_median uses a more efficient algorithm; see the nth_element page for more details. Besides being a shameless plug, this demonstrates why we might want to write a Mex function in the first place: rewriting the median function in pure Matlab would be slow, but using C++ we can significantly improve on the status quo.

The second point is that the in-place version, fast_median_ip, yields an additional speed improvement. What is the difference? Let us look behind the scenes; here are the CPU and memory traces from my system monitor after running the above:

Memory and CPU usage for median vs. fast_median_ip

You can see four spikes in CPU use, and some associated changes in memory allocation:

The first spike in CPU is when we created the test data vector; memory use also steps up at that time.

The second CPU spike is the largest; that is Matlab’s median function. You can see that over that period memory use stepped up again, and then stepped back down; the median function makes a copy of the entire input data, and then throws its copy away when it is finished; this is expensive in terms of time and resources.

The fast_median function is the next CPU spike; it has a similar step up and down in memory use, but it is much faster.

Finally, in the case of fast_median_ip we see something different; there is a spike in CPU use, but memory use stays flat; the in-place version is faster and more memory efficient because it does not make a copy of the input data.

There is another important difference with the in-place version; it modifies its input array. This can be demonstrated simply:

>> A = randi(100, [10 1]);
>> A'
ans = 39    42    98    25    64    75     6    56    71    89
 
>> median(A)
ans = 60
 
>> fast_median(A)
ans = 60
>> A'
ans = 39    42    98    25    64    75     6    56    71    89
 
>> fast_median_ip(A)
ans = 60
>> A'
ans = 39     6    25    42    56    64    75    71    98    89

As you can see, all three methods get the same answer, but median and fast_median do not modify A in the workspace, whereas after running fast_median_ip, input array A has changed. This is how the in-place method is able to run without using new memory; it operates on the existing array rather than making a copy.

Pitfalls with in-place editing

Modifying a function’s input is common in many languages, but in Matlab there are only a few special conditions under which this is officially sanctioned. This is not necessarily a bad thing; many people feel that modifying input data is bad programming practice and makes code harder to maintain. But as we have shown, it can be an important capability to have if speed and memory use are critical to an application.

Given that in-place editing is not officially supported in Matlab Mex extensions, what do we have to do to make it work? Let us look at the normal, input-copying fast_median function as a starting point:

void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]) {
   mxArray *incopy = mxDuplicateArray(prhs[0]);
   plhs[0] = run_fast_median(incopy);
}

This is a pretty simple function (I have taken out a few lines of boiler plate input checking to keep things clean). It relies on helper function run_fast_median to do the actual calculation, so the only real logic here is copying the input array prhs[0]. Importantly, run_fast_median edits its inputs in-place, so the call to mxDuplicateArray ensures that the Mex function is overall well behaved, i.e. that it does not change its inputs.

Who wants to be well behaved though? Can we save time and memory just by taking out the input duplication step? Let us try it:

void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]) {
   plhs[0] = run_fast_median(const_cast<mxarray *>(prhs[0]));  // </mxarray>
}

Very bad behavior; note that we cast the original const mxArray* input to a mxArray* so that the compiler will let us mess with it; naughty.

But does this accomplish edit in-place for fast_median? Be sure to save any work you have open and then try it:

>> mex fast_median_tweaked.cpp
>> A = randi(100,[10 1]);
>> fast_median_tweaked(A)
ans = 43

Hmm, it looks like this worked fine. But in fact there are subtle problems:

>> A = randi(100,[10 1]);
>> A'
ans = 65    92    14    26    41     2    45    85    53     2
>> B = A;
>> B'
ans = 65    92    14    26    41     2    45    85    53     2
 
>> fast_median_tweaked(A)
ans = 43
>> A'
ans = 2     2    41    26    14    45    65    85    53    92
>> B'
ans = 2     2    41    26    14    45    65    85    53    92

Uhoh, spooky; we expected that running fast_median_tweaked would change input A, but somehow it has also changed B, even though B is supposed to be an independent copy. Not good. In fact, under some conditions this kind of uncontrolled editing in-place can crash the entire Matlab environment with a segfault. What is going on?

Matlab’s copy-on-write mechanism

The answer is that our simple attempt to edit in-place conflicts with Matlab’s internal copy-on-write mechanism. Copy-on-write is an optimization built into Matlab to help avoid expensive copying of variables in memory (actually similar to what we are trying to accomplish with edit in-place). We can see copy-on-write in action with some simple tests:

Matlab's Copy-on-Write memory and CPU usage

% Test #1: update, then copy
>> tic; A = zeros(100000000, 1); toc
Elapsed time is 0.588937 seconds.
>> tic; A(1) = 0; toc
Elapsed time is 0.000008 seconds.
>> tic; B = A; toc   
Elapsed time is 0.000020 seconds.
 
% Test #2: copy, then update
>> clear
>> tic; A = zeros(100000000, 1); toc
Elapsed time is 0.588937 seconds.
>> tic; B = A; toc   
Elapsed time is 0.000020 seconds.
>> tic; A(1) = 0; toc
Elapsed time is 0.678160 seconds.

In the first set of operations, time and memory are used to create A, but updating A and “copying” A into B take no memory and essentially no time. This may come as a surprise since supposedly we have made an independent copy of A in B; why does creating B take no time or memory when A is clearly a large, expensive block?

The second set of operations makes things more clear. In this case, we again create A and then copy it to B; again this operation is fast and cheap. But assigning into A at this point takes time and consumes a new block of memory, even though we are only assigning into a single index of A. This is copy-on-write: Matlab tries to save you from copying large blocks of memory unless you need to. So when you first assign B to equal A, nothing is copied; the variable B is simply set to point to the same memory location already used by A. Only after you try to change A (or B), does Matlab decide that you really need to have two copies of the large array.

There are some additional tricks Matlab does with copy-on-write. Here is another example:

>> clear
>> tic; A{1} = zeros(100000000, 1); toc
Elapsed time is 0.573240 seconds.
>> tic; A{2} = zeros(100000000, 1); toc
Elapsed time is 0.560369 seconds.
 
>> tic; B = A; toc                     
Elapsed time is 0.000016 seconds.
 
>> tic; A{1}(1) = 0; toc               
Elapsed time is 0.690690 seconds.
>> tic; A{2}(1) = 0; toc
Elapsed time is 0.695758 seconds.
 
>> tic; A{1}(1) = 0; toc
Elapsed time is 0.000011 seconds.
>> tic; A{2}(1) = 0; toc
Elapsed time is 0.000004 seconds.

This shows that for the purposes of copy-on-write, different elements of cell array A are treated independently. When we assign B equal to A, nothing is copied. Then when we change any part of A{1}, that whole element must be copied over. When we subsequently change A{2}, that whole element must also be copied over; it was not copied earlier. At this point, A and B are truly independent of each other, as both elements have experienced copy-on-write, so further assignments into either A or B are fast and require no additional memory.

Try playing with some struct arrays and you will find that copy-on-write also works independently for the elements of structs.

Reconciling edit in-place with copy-on-write: mxUnshareArray

Now it is clear why we cannot simply edit arrays in-place from Mex functions; not only is it naughty, it fundamentally conflicts with copy-on-write. Naively changing an array in-place can inadvertently change other variables that are still waiting for a copy-on-write, as we saw above when fast_median_tweaked inadvertently changed B in the workspace. This is, in the best case, an unmaintainable mess. Under more aggressive in-place editing, it can cause Matlab to crash with a segfault.

Luckily, there is a simple solution, although it requires calling internal, undocumented Matlab functions.

Essentially what we need is a Mex function that can be run on a Matlab array that will do the following:

1. Check whether the current array is sharing data with any other arrays that are waiting for copy-on-write.
2. If the array is shared, it must be unshared; the underlying memory must be copied and all the relevant pointers need to be fixed so that the array we want to work on is no longer accessible by anyone else.
3. If the array is not currently shared, simply proceed; the whole point is to avoid copying memory if we do not need to, so that we can benefit from the efficiencies of edit in-place.

If you think about it, this is exactly the operation that Matlab needs to run internally when it is deciding whether an assignment operation requires a copy-on-write. So it should come as no surprise that such a Mex function already exists in the form of a Matlab internal called mxUnshareArray. Here is how you use it:

extern "C" bool mxUnshareArray(mxArray *array_ptr, bool noDeepCopy);
 
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]) {
   mxUnshareArray(const_cast<mxarray *>(prhs[0]), true);  //</mxarray>
   plhs[0] = run_fast_median(const_cast<mxarray *>(prhs[0]));  //</mxarray>
}

This is the method actually used by fast_median_ip to efficiently edit in-place without risking conflicts with copy-on-write. Of course, if the array turns out to need to be unshared, then you do not get the benefit of edit in-place because the memory ends up getting copied. But at least things are safe and you get the in-place benefit as long as the input array is not being shared.

Further topics

The method shown here should allow you to edit normal Matlab numerical or character arrays in-place from Mex functions safely. For a Mex function in C rather than C++, omit the “C” in the extern declaration and of course you will have to use C-style casting rather than const_cast. If you need to edit cell or struct arrays in-place, or especially if you need to edit subsections of shared cell or struct arrays safely and efficiently while leaving the rest of the data shared, then you will need a few more tricks. A good place to get started is this article by Benjamin Schubert.

Unfortunately, over the last few years Mathworks seems to have decided to make it more difficult for users to access these kinds of internal methods to make our code more efficient. So be aware of the risk that in some future version of Matlab this method will no longer work in its current form.

Ultimately much of what is known about mxUnshareArray as well as the internal implementation details of how Matlab keeps track of which arrays are shared goes back to the work of Peter Boettcher, particularly his headerdump.c utility. Unfortunately, it appears that HeaderDump fails with Matlab releases >=R2010a, as Mathworks have changed some of the internal memory formats – perhaps some smart reader can pick up the work and adapt HeaderDump to the new memory format.

In a future article, I hope to discuss headerdump.c and its relevance for copy-on-write and edit in-place, and some other related tools for the latest Matlab releases that do not support HeaderDump.

1. Undocumented profiler options The Matlab profiler has some undocumented options that facilitate debugging memory bottlenecks and JIT (Just-In-Time Java compilation) problems....
2. Matlab-Java memory leaks, performance Internal fields of Java objects may leak memory - this article explains how to avoid this without sacrificing performance. ...
3. JMI – Java-to-Matlab Interface JMI enables calling Matlab functions from within Java. This article explains JMI's core functionality....
4. Matlab callbacks for Java events Events raised in Java code can be caught and handled in Matlab callback functions - this article explains how...

Here’s a screenshot of the message that mlint reported in the editor:

erroneous mlint error report

Mlint’s confusion would not have bothered us if it were not for a seemingly-related issue: When we tried to save the file, the Matlab editor would only allow us to save the file under a different name. When we attempted to restart Matlab, the same behavior repeated for this file, and we could not proceed with our actual work, modifying and saving our m-file.

Not easily intimidated, I tried saving the file under a different name in the editor, then renaming it back outside Matlab. No good, same problem.

MLintFailureFiles or: yet another use for prefdir

The clue that eventually led to the fix was that somehow Matlab remembered the “faulty” state of the m-file even after I restarted Matlab. Matlab likes to store persistent cross-session information in a single folder, which is the user’s prefdir folder. I have used this to my advantage only a few weeks ago, when I explained how to use the stored editor state file that is stored in that folder.

My hunch was that Matlab might store the mlint’s “faulty” m-file state somewhere in there. I didn’t have to look very far: one of the files most recently modified had the tell-tale name of MLintFailureFiles. This turns out to be an empty file that contains the full pathname of the “faulty” m-files, one file per line.

The fix turned out to be very easy: simply remove my file from the MLintFailureFiles file (or delete MLintFailureFiles entirely) and restart Matlab. Problem solved – everything back to normal. Resume breathing.

Undocumented? well, not exactly

I then went back to the suggestion in the mlint message and true enough I found related information in the documentation, at the bottom of the doc-page for using mlint. This is from R2008a’s doc page:

About M-Lint and Unexpected MATLAB® Termination

Under some circumstances, when you are editing an M-file, M-Lint can cause the MATLAB session to terminate unexpectedly. The next time you start MATLAB, it displays the following message and disables M-Lint for the M-file that was open in the editor when MATLAB terminated.

M-Lint caused your previous MATLAB session to terminate unexpectedly.
Please send this message and file name to The MathWorks.
See "About M-Lint and Unexpected MATLAB Termination" in the MATLAB documentation for details.

If you want, while waiting for a response from The MathWorks™, you can attempt to reenable M-Lint for the file by following these steps:

1. In the Editor, reopen the file that you were editing when MATLAB terminated.
2. Remove the lines of code that you believe M-Lint could not handle.
3. In a text editor, open the MLintFailureFiles file in your preferences directory. (This is the directory that MATLAB returns when you run prefdir.)
4. Save and reopen the M-file.

As you can see, this is not very helpful because it’s missing the step of actually editing (or deleting) the MLintFailureFiles file.

Additional documentation of this issue and workaround can be found in two separate Mathworks technical notes: here and here.

A Matlab user has even created a utility on the File Exchange that simply deletes this file, basing his work on the first note above.

The second note indicates that the problem (and workaround) occur in Matlab releases R2008a-R2010b, and indeed the above-mentioned issue (and workaround) can no longer be found in the latest docs (as far as I could see). However, even if this issue (and MLintFailureFiles) may no longer be relevant in the latest Matlab releases, we may still see MLintFailureFiles in our prefdir, because the prefdir contents are automatically copied from the previous release whenever we install a newer Matlab release.

1. Recovering previous editor state Recovering the previous state of the Matlab editor and its loaded documents is possible using a built-in backup config file. ...
2. More undocumented timing features There are several undocumented ways in Matlab to get CPU and clock data...
3. Undocumented scatter plot behavior The scatter plot function has an undocumented behavior when plotting more than 100 points: it returns a single unified patch object handle, rather than a patch handle for each specific...
4. UDD Events and Listeners UDD event listeners can be used to listen to property value changes and other important events of Matlab objects...

The problem: “Matlab crashes” – now go figure…

A client of one of my Matlab programs recently complained that Matlab crashes after several hours of extensive use of the program. The problem looked like something that is memory related (messages such as Matlab’s out-of-memory error or Java’s heap-space error). Apparently this happens even on 64-bit systems having lots of memory, where memory should never be a problem.

Well, we know that this is only in theory, but in practice Matlab’s internal memory management has problems that occasionally lead to such crashes. This is one of the reasons, by the way, that recent Matlab releases have added the preference option of increasing the default Java heap space (the previous way to do this was a bit complex). Still, even with a high Java heap space setting and lots of RAM, Matlab crashed after using my program for several hours.

Not pleasant at all, even a bit of an embarrassment for me. I’m used to crashing Matlab, but only as a result of my playing around with the internals – I would hate it to happen to my clients.

Finding the leak

While we can do little with Matlab’s internal memory manager, I started searching for the exact location of the memory leak and then to find a way to overcome it. I’ll save readers the description about the grueling task of finding out exactly where the memory leak occurred in a program that has thousands of lines of code and where events get fired asynchronously on a constant basis. Matlab Profiler’s undocumented memory profiling option helped me quite a bit, as well as lots of intuition and trial-and-error. Detecting memory leak is never easy, and I consider myself somewhat lucky this time to have both detected the leak source and a workaround.

It turned out that the leakage happens in a callback that gets invoked multiple times per second by a Java object (see related articles here and here). Each time the Matlab callback function is invoked, it reads the event information from the supplied Java event-data (the callback’s second input parameter). Apparently, about 1KB of memory gets leaked whenever this event-data is being read. This may appear a very small leak, but multiply this by some 50-100K callback invocations per hour and you get a leakage of 50-100MB/hour. Not a small leak at all; more of a flood you could say…

Using get()

The leakage culprit turned out to be the following code snippet:

% 160 uSecs per call, with memory leak
eventData  = get(hEventData,{'EventName','ParamNames','EventData'});
eventName  = eventData{1};
paramNames = eventData{2};
paramData  = eventData{3}.cell;

In this innocent-looking code, hEventData is a Java object that contains the EventName, ParamNames, EventData properties: EventName is a Java String, that is automatically converted by Matlab’s get() function into a Matlab string (char array); ParamNames is a Java array of Strings, that gets automatically converted into a Matlab cell-array of string; and EventData is a Java array of Objects that needs to be converted into a Matlab cell array using the built-in cell function, as described in one of my recent articles.

The code is indeed innocent, works really well and is actually extremely fast: each invocation takes of this code segment takes less than 0.2 millisecs. Unfortunately, because of the memory leak I needed to find a better alternative.

Using handle()

The first idea was to use the built-in handle() function, under the assumption that it would solve the memory leak, as reported here. In fact, MathWorks specifically advises to use handle() rather than to work with “naked” Java objects, when setting Java object callbacks. The official documentation of the set function says:

Do not use the set function on Java objects as it will cause a memory leak.

It stands to reason then that a similar memory leak happens with get and that a similar use of handle would solve this problem:

% 300 uSecs per call, with memory leak
s = get(handle(hEventData));
eventName  = s.EventName;
paramNames = s.ParamNames;
paramData  = cell(s.EventData);

Unfortunately, this variant, although working correctly, still leaks memory, and also performs almost twice as worse than the original version, taking some 0.3 milliseconds to execute per invocation. Looks like this is a dead end.

Using Java accessor methods

The next attempt was to use the Java object’s internal accessor methods for the requested properties. These are public methods of the form getXXX(), isXXX(), setXXX() that enable Matlab to treat XXX as a property by its get and set functions. In our case, we need to use the getter methods, as follows:

% 380 uSecs per call, no memory leak
eventName  = char(hEventData.getEventName);
paramNames = cell(hEventData.getParamNames);
paramData  = cell(hEventData.getEventData);

Here, the method getEventName() returns a Java String, that we convert into a Matlab string using the char function. In our previous two variants, the get function did this conversion for us automatically, but when we use the Java method directly we need to convert the results ourselves. Similarly, when we call getParamNames(), we need to use the cell function to convert the Java String[] array into a Matlab cell array.

This version at last doesn’t leak any memory. Unfortunately, it has an even worse performance: each invocation takes almost 0.4 milliseconds. The difference may seem insignificant. However, recall that this callback gets called dozens of times each second, so the total adds up quickly. It would be nice if there were a faster alternative that does not leak any memory.

Using struct()

Luckily, I found just such an alternative. At 0.24 millisecs per invocation, it is almost as fast as the leaky best-performance original get version. Best of all, it leaks no memory, at least none that I could detect.

The mechanism relies on the little-known fact that public fields of Java objects can be retrieved in Matlab using the built-in struct function. For example:

>> fields = struct(java.awt.Rectangle)
fields = 
             x: 0
             y: 0
         width: 0
        height: 0
      OUT_LEFT: 1
       OUT_TOP: 2
     OUT_RIGHT: 4
    OUT_BOTTOM: 8
 
>> fields = struct(java.awt.Dimension)
fields = 
     width: 0
    height: 0

Note that this useful mechanism is not mentioned in the main documentation page for accessing Java object fields, although it is indeed mentioned in another doc-page – I guess this is a documentation oversight.

In any case, I converted my Java object to use public (rather than private) fields, so that I could use this struct mechanism (Matlab can only access public fields). Yes I know that using private fields is a better programming practice and all that (I’ve programmed OOP for some 15 years…), but sometimes we need to do ugly things in the interest of performance. The latest version now looks like this:

% 240 uSecs per call, no memory leak
s = struct(hEventData);
eventName  = char(s.eventName);
paramNames = cell(s.paramNames);
paramData  = cell(s.eventData);

This solved the memory leakage issue for my client. I felt fortunate that I was not only able to detect Matlab’s memory leak but also find a working workaround without sacrificing performance or functionality.

In this particular case, I was lucky to have full control over my Java object, to be able to convert its fields to become public. Unfortunately, we do not always have similar control over the object that we use, because they were coded by a third party.

By the way, Matlab itself uses this struct mechanism in its code-base. For example, Matlab timers are implemented using Java objects (com.mathworks.timer.TimerTask). The timer callback in Matlab code converts the Java timer event data into a Matlab struct using the struct function, in %matlabroot%/toolbox/matlab/iofun/@timer/timercb.m. The users of the timer callbacks then get passed a simple Matlab EventData struct without ever knowing that the original data came from a Java object.

As an interesting corollary, this same struct mechanism can be used to detect internal properties of Matlab class objects. For example, in the timers again, we can get the underlying timer’s Java object as follows (note the highlighted warning, which I find a bit ironic given the context):

>> timerObj = timerfind
 
   Timer Object: timer-1
 
   Timer Settings
      ExecutionMode: singleShot
             Period: 1
           BusyMode: drop
            Running: off
 
   Callbacks
           TimerFcn: @myTimerFcn
           ErrorFcn: ''
           StartFcn: ''
            StopFcn: ''
 
>> timerFields = struct(timerObj)
Warning: Calling STRUCT on an object prevents the object from hiding its implementation details and should thus be avoided.Use DISP or DISPLAY to see the visible public details of an object. See 'help struct' for more information.(Type "warning off MATLAB:structOnObject" to suppress this warning.)timerFields = 
         ud: {}
    jobject: [1x1 javahandle.com.mathworks.timer.TimerTask]

1. Performance: scatter vs. line In many circumstances, the line function can generate visually-identical plots as the scatter function, much faster...
2. Plot performance Undocumented inner plot mechanisms can be used to significantly improved plotting performance...
3. datestr performance Caching is a simple and very effective means to improve code performance, as demonstrated for the datestr function....
4. Matrix processing performance Matrix operations performance is affected by internal subscriptions in a counter-intuitive way....

A few days ago, a Matlab user asked whether it is possible to solve the flashing effect whenever he cleared the Command Window. It turns out that this user runs a long process, and wanted to display interim information (progress etc.) in the Command Window. The obvious solution that he employed was to display the data in the Command Window, and then erase the window (using clc) and rewrite the data periodically.

Using control characters

Now, there are obviously other ways of doing this, the most obvious being a dedicated GUI window that displays the information and updates periodically. But to solve the user’s immediate issue, the solution I provided (acting on the 80-20 Pareto principle – a dedicated GUI window is better but would take longer to set up) was to use control characters to erase old data when displaying new data, rather than clearing the window. Here is the basic pseudo-code that I suggested (slightly modified):

reverseStr = '';
for idx = 1 : someLargeNumber
 
   % Do some computation here...
 
   % Display the progress
   percentDone = 100 * idx / someLargeNumber;
   msg = sprintf('Percent done: %3.1f', percentDone)
   fprintf([reverseStr, msg]);
   reverseStr = repmat(sprintf('\b'), 1, length(msg));
end

The idea is to use the backspace control-character (BS, or sprintf(‘\b’), or char(8)) repeatedly in order to erase the preceding characters from the Command Window, then print the new data.

(see related comments by Helge here)

Compatibility aspects

Back in the good-ol’-days, when I was hacking away c-shell scripts, ages ago when I created command-prompt C apps, and aeons ago on Vax (R.I.P.), I would have used the carriage-return control-char (CR, or sprintf(‘\r’), or char(13)) to completely erase everything up to the beginning of the current line. This would be much simpler than the repeated backspaces (using repmat in the pseudo-code above). Unfortunately, CRs behave differently in Matlab’s Command Window, causing \r to jump to the next line (similarly to \n), rather than erase to the beginning of the line. This is probably due to Java’s JTextArea’s implementation (on which the Command Window is based), and not due to Matlab itself. In any case, it shows that not all control characters behave the same way on different environments, which is actually not very surprising.

Another widely-used control character, the bell control-char (BEL, or sprintf(‘\g’), or char(7)) is not accepted at all by Matlab’s implementation of sprintf and its variants. Matlab users can always use the beep function of course, I’m just pointing this out as another inconsistency between Matlab’s *printf() implementation and that of other environments. And don’t even think of using raw device escape sequences (ah, the good-ol’-days, long gone now…).

Control characters that are accepted in Matlab and behave as expected include: FF (\f), NL (\n) and TAB (\t). These can be used in fprintf to modify the text spacing. The sprintf documentation mentions which of the control characters are accepted, so this is not, strictly speaking, an undocumented aspect. However, note that there is a discrepancy between the list of accepted control chars in the online/doc page compared to the help section (\a and \v are mentioned in the former but not in the latter).

For another type of Command Window text manipulation, namely colors, refer to my cprintf utility. For anyone who hasn’t noticed, I recently updated the utility on the Matlab File Exchange. Due to some internal modifications by MathWorks to the Command Window implementation in R2011b, cprintf no longer needs to pad color segments with spaces, and separate color segments can now be directly adjacent to each other (in R2011a and earlier, the spaces are unfortunately needed).

Have you used control characters in an innovative manner in your work? If so, please share your experience in a comment.

Happy New Year everyone!

1. cprintf – display formatted color text in the Command Window cprintf is a utility that utilized undocumented Matlab desktop functionalities to display color and underline-styled formatted text in the Command Window...
2. Changing Matlab’s Command Window colors – part 2 The Matlab Command Window enables a limited degree of inline color customization - this post describes how to use it...
3. EditorMacro v2 – setting Command Window key-bindings The EditorMacro utility was extended to support built-in Matlab Editor and Command-Window actions and key-bindings. This post describes the changes and the implementation details....
4. Changing Matlab’s Command Window colors Matlab's Command Window foreground and background colors can be modified programmatically, using some of Matlab's undocumented internal Java classes. Here's how....

I have encountered this question myself some time ago, when I tried to customize a radar plot: The grid in radar plots does not automatically re-draw when the plot is zoomed, as in regular rectangular plots. Instead, the radial (angle) and circular (range) grid lines are statically painted when the plot is created, and remain fixed when the plot is zoomed or panned. Therefore, if we zoom in on a small-enough plot segment, we will not see any grid lines at all. It would be beneficial to repaint the grid-lines upon every zoom and pan event. I will not dive into the details today, but the important thing for today’s article is that the algorithm needed to know whether or not the axes is currently zoomed or not.

The official response is that this information is not readily available. We could of course store the axes limits somewhere and then compare them to the current limits in run-time. This will cause complications if the plotted data (and thereby the axes limits) change automatically during program use, even without any zooming/panning. It is also problematic if the user resets the zoom limits (as Jiro has correctly pointed out).

Today I’ll highlight another possible solution, that perhaps not many Matlab users are familiar with: Apparently, whenever zooming or panning occur, a hidden ApplicationData object is automatically added to the affected axes, with information about the original axes meta-data: limits, aspect ratio, camera info and view angles. This object is also automatically updated whenever we use zoom reset to reset the zoom limits.

So basically, all we have to do in run-time is to compare our current access limits with the stored info: if they are the same (or if the app-data object is missing) then the plot is unzoomed and unpanned; otherwise it is. So simple.

origInfo = getappdata(gca, 'matlab_graphics_resetplotview');
if isempty(origInfo)
   isZoomed = false;
elseif isequal(get(gca,'XLim'), origInfo.XLim) && ...
       isequal(get(gca,'YLim'), origInfo.YLim) && ...
       isequal(get(gca,'ZLim'), origInfo.ZLim)
   isZoomed = false;
else
   isZoomed = true;
end

This still does not solve the problem of limit changes when the plot data is changed, but it may perhaps be a bit easier to use than manually storing the limits before plotting. (And yes, of course the if-else statement above could be made a one-liner logical construct, but I think this way is more readable).

Zooming and panning add some additional data to ApplicationData, and also use/modify some other hidden properties of the axes handle (use the getundoc function to see them). If you use one of them for any useful purpose, please report your usage in a comment below.

1. Axes LooseInset property Matlab plot axes have an undocumented LooseInset property that sets empty margins around the axes, and can be set to provide a tighter fit of the axes to their surroundings....
2. Tri-state checkbox Matlab checkboxes can easily be made to support tri-state functionality....
3. Recovering previous editor state Recovering the previous state of the Matlab editor and its loaded documents is possible using a built-in backup config file. ...
4. Plot LimInclude properties The plot objects' XLimInclude, YLimInclude, ZLimInclude, ALimInclude and CLimInclude properties are an important feature, that has both functional and performance implications....

Basic multi-line tooltips

The basic multi-line tooltip consists of a simple string that includes the newline character. for example:

set(hControl, 'TooltipString', ['this is line 1' 10 'and this is line 2'])

Or better (platform independent):

set(hControl, 'TooltipString', sprintf('this is line 1\nand this is line 2'))

A simple multi-line tooltip string

As can be seen, this is very simple to set up. Unfortunately, the font cannot be customized. Which leads us to:

HTML-rendered multi-line tooltips

Tooltips, like most other Matlab controls, supports HTML rendering. This is a natural corollary of the fact that Java Swing, on which all Matlab controls are based, automatically supports HTML. All we need to do is to add ‘<HTML>‘ at the very beginning of our tooltip string, and then we can embed HTML tags within the rest of the string:

set(hControl, 'tooltipString', '<html><b>line #1</b><br /><i><font color="red">line#2</font></i></html>');

Multi-line HTML'ed tooltip

And a more sophisticated example, used within my createTable utility on the File Exchange:

A more complex multi-line HTML-based tooltip

Note that there is no need to close the final HTML tags in the tooltip string, although it’s not an error to do so (as I have done above).

The problem with formatted multi-line tooltip

Unfortunately, none of these two methods work when we wish to display formatted multi-line tooltips. For example, suppose that we wish to display struct data in a tooltip:

>> myData = struct('a',pi, 'b',-4, 'very_long_field_name','short txt')
myData = 
                       a: 3.14159265358979
                       b: -4
    very_long_field_name: 'short txt'
 
>> myDataStr = evalc('disp(myData)');
>> set(hControl, 'TooltipString', myDataStr);

Badly-formatted multi-line tooltip

If we try to use HTML, the result looks even worse, because if the HTML-renderer detects a newline character embedded in the string, it simply uses the regular (non-HTML) renderer:

set(hControl, 'TooltipString', ['<html>' myDataStr '</html>'])

Failure to parse a string using HTML

Even if we fix this by replacing all line-separators with ‘<br/>‘, we still get a badly-formatted tooltip, because HTML ignores all white spaces:

>> myDataStr2 = strrep(myDataStr, sprintf('\n'), '<br />');
>> set(hControl, 'TooltipString', ['<html>' myDataStr2 '</html>'])

Badly-formatted multi-line tooltip

(you can see that it’s HTML-formatted by the fact that the tooltip contents do not have internal margins like in the non-HTML tooltip above)

Fixed-width font multi-line tooltips

We now recall the HTML <pre> tag, which tells HTML not to ignore white-spaces. In most web browsers, it also defaults to using a fixed-width font. Unfortunately, this is not the case here:

set(hControl, 'TooltipString', ['<html><pre>' myDataStr2])

Still not quite right...

Which brings us to the final customization of the day, namely explicitly forcing the tooltip to use a fixed-width font:

set(hControl, 'TooltipString', ['<html><pre><font face="courier new">' myDataStr2 '</font>'])

Finally - a well-formatted multi-line tooltip

Even more powerful customizations can be achieved using CSS rather than pure HTML. This will be discussed a future article.

1. Spicing up Matlab uicontrol tooltips Matlab uicontrol tooltips can be spiced-up using HTML and CSS, including fonts, colors, tables and images...
2. Multi-column (grid) legend This article explains how to use undocumented axes listeners for implementing multi-column plot legends...
3. Inactive Control Tooltips & Event Chaining Inactive Matlab uicontrols cannot normally display their tooltips. This article shows how to do this with a combination of undocumented Matlab and Java hacks....
4. Performance: scatter vs. line In many circumstances, the line function can generate visually-identical plots as the scatter function, much faster...

Today I wish to share a related experience that happened to me yesterday, when I needed to improve the performance of a client’s application. When profiling the application, I found that a major performance hotspot was a repeated call to the datestr function, each time with several thousand date items.

datestr is the opposite function of datenum: it receives date values and returns date strings. Unlike datenum, however, datestr does not use a highly optimized native-code library function that we could use directly. Instead, it loops over all date values and sequentially applies the requested string pattern.

The natural reaction in such a case would perhaps be to vectorize the code (something that MathWorks should have done in the first place I guess). But in this case I used a different solution, that I would like to share today:

In any programming languages, Matlab included, the most effective performance tip is to cache processing results. Caching often makes the code slightly more complex and less maintainable, but the performance benefits are immediate and significant. In Matlab, benefits of caching can often surpass even those of vectorization (using both vectorization and caching is of course even better).

In the case of datestr, if we can be certain of the precondition that the output string format is the same, we can cache the results, and even use vectorization. In my case, I plotted historical daily stock quotes data and so I was assured that (1) all dates are integers and that (2) I always use the same date-string format ‘dd-mmm-yyyy’.

First, let’s define the wrapper function datestr2 with the necessary caching and vectorization:

% datestr2 - faster variant of datestr, for integer date values since 1/1/2000
function dateStrs = datestr2(dateVals,varargin)
 
  persistent dateStrsCache
  persistent dateValsCache
 
  if isempty(dateStrsCache)
      origin = datenum('1-Jan-2000');
      dateValsCache = origin:(now+100);
      dateStrsCache = datestr(dateValsCache,varargin{:});
  end
 
  [tf,loc] = ismember(dateVals, dateValsCache);
  if all(tf)
      dateStrs = dateStrsCache(loc,:);
  else
      dateStrs = datestr(dateVals,varargin{:});
  end
end  % datestr2

As can be seen, the first time that datestr2 is called, it computes and caches all datestr values for all the dates since Jan 1, 2000. Subsequent calls to datestr2 simply retrieve the relevant cache values. Note that the input date entries need not be sorted.

In case that an input date number is not found in the cache, datestr2 automatically falls-back to using the built-in datestr for the entire input list. This could of course be improved to add the new entries to the cache – I leave this as a reader exercise.

The bottom line was a 150-times (!!!) speed improvement for a 1000-item date vector (50mS => 0.3mS on my system):

% Prepare a 1000-vector of dates, starting 3 years ago until today
>> dateVals = fix(now)+(-1000:0);
 
% Run the standard datestr function => 50mS
>> tic; s1=datestr(dateVals); toc
Elapsed time is 0.049089 seconds.
>> tic; s1=datestr(dateVals); toc
Elapsed time is 0.048086 seconds.
 
% Now run our datestr2 function (caching already done before) => 0.3 mS
>> tic; s2=datestr2(dateVals); toc
Elapsed time is 0.222031 seconds.   % initial cache preparation takes 222 mS
>> tic; s2=datestr2(dateVals); toc
Elapsed time is 0.000313 seconds.   % subsequent datestr2 calls take 0.3 mS
>> tic; s2=datestr2(dateVals); toc
Elapsed time is 0.000296 seconds.
 
% Ensure that the two functions give exactly the same results
>> isequal(s1,s2)
ans =
     1

So what have we learned from this?

1. To improve performance we must profile the code. Very often the performance bottlenecks occur in non-intuitive very specific places that can be surgically handled without requiring any major redesign. In this case, I simply had to replace calls to datestr with datestr2 in the application’s code.
2. Vectorization is not always as cost-effective as caching
3. Major performance improvements do NOT necessarily involve undocumented functions or tricks: In fact, today’s post about caching uses fully-documented pure-Matlab code.
4. Different performance hotspots can have different solutions: caching, vectorization, internal library functions, undocumented graphics properties, smart property selection, smart function selection, smart indexing, smart parameter selection etc.

In a later post I will show how similar modifications to internal Matlab functions can dramatically improve the performance of the uitable function. Anyone who has tried using uitable with more than a few dozen cells will surely understand why this is important…

Do you have a favorite performance trick not mentioned above? If so, please post a comment.

1. Performance: scatter vs. line In many circumstances, the line function can generate visually-identical plots as the scatter function, much faster...
2. Plot performance Undocumented inner plot mechanisms can be used to significantly improved plotting performance...
3. Datenum performance The performance of the built-in Matlab function datenum can be significantly improved by using an undocumented internal help function...
4. Matrix processing performance Matrix operations performance is affected by internal subscriptions in a counter-intuitive way....

Matlab users who use non-Latin computer Locales are aware of the issues that the Matlab Command Window has had with such languages for many years. I am not sure whether these problems are due to the LTR nature of Hebrew/Arabic, or their use of a non-supported code-page, or some other reason. To this day (R2011b), I am not aware of any fix or workaround for these issues.

But it seems that in addition, Matlab has a problem reading files that contain text in these languages, even when the computer’s Locale is set correctly, to a Locale that supports the non-Latin text. This is where Ro’ee’s workaround helps. In his words:

To give some more background, this used to work with a 32bit system, and an older version of Matlab (7.1). Now it doesn’t. Saving the file in UTF-8 and using fopen and textscan instead of importdata gives me this:

nowords =
‘שלבק’
‘התלכב’
‘× ×™×›×˜×¨’
‘תלפורש’
‘×œ×§×˜× ‘
‘מזוחש’
‘שלטיק’
‘טיבר’
‘עולג’
‘סלבוחד’
‘משוחגות’
‘מלוגסות’
‘סבק’
‘צמשר’
‘הכריב’
‘תמציל’

The solution is as follows (requires Simulink):

1) Change system Locale to Hebrew: http://windows.microsoft.com/en-US/windows7/Change-the-system-locale

(this doesn’t change the language of the OS etc.).

2) Change the encoding that Matlab uses:
http://www.mathworks.com/help/toolbox/simulink/slref/slcharacterencoding.html

They tell you not to, but I did… – you must change it to encoding that works for Hebrew: http://www.iana.org/assignments/character-sets

Any other language should work as well (I hope…). For Hebrew the code that works for me is ISO_8859-8

3) You should now be able to read TXT files that have Hebrew characters in them.

>> a='הצלחה!'
a =
!
 
>> currentCharacterEncoding = slCharacterEncoding();
>> currentCharacterEncoding = get_param(0, 'CharacterEncoding')  % equivalent alternative
currentCharacterEncoding =
windows-1252
 
% Now modify the default encoding to something more useful
>> slCharacterEncoding('ISO_8859-8')
>> set_param(0, 'CharacterEncoding', 'ISO_8859-8');   % equivalent alternative
 
>> currentCharacterEncoding = slCharacterEncoding()
currentCharacterEncoding =
ISO-8859-8
 
>> a='הצלחה!'
a =
!                  % still no good in the Command Window...
 
% Let's try to read a file with some Hebrew words:
>> neutral = importdata('neutral.txt')
neutral = 
שולחן'
    'כסא'
    'מנורה'
    'צלחת'
    'סיר'
    'מזלג'

So, it appears that while we did not solve the problems with the Command Window, at least we can now read the prayer book for our New Year prayers…

Let this be a year of fulfillment, prosperity, health and happiness to all. Shana Tova everybody!

1. FIG files format FIG files are actually MAT files in disguise. This article explains how this can be useful in Matlab applications....
2. Command Window text manipulation Special control characters can be used to format text output in Matlab's Command Window. ...
3. cprintf – display formatted color text in the Command Window cprintf is a utility that utilized undocumented Matlab desktop functionalities to display color and underline-styled formatted text in the Command Window...
4. Setting status-bar text The Matlab desktop and figure windows have a usable statusbar which can only be set using undocumented methods. This post shows how to set the status-bar text....

Hidden properties are object properties that for some reason or other MathWorks has decided not to expose to the general public. They can still be used by Matlab users, just like any other regular property. But if you use the built-in get and set functions to list the object’s properties, you will not find the hidden properties listed. You need to know the hidden properties’ exact name in order to use them. Which is where today’s post can help, by showing you how to list these hidden properties. I wrote about this a couple of years ago, and today’s article will expand on that original post.

Hidden properties are by their very nature undocumented and not officially supported. For this reason, you should take extra care when relying on them in your code. They could change functionality or even be removed without prior notice in any future Matlab release. Still, some of these properties enable very important functionality, as I have often shown on this website.

HideUndocumented

There are two distinct manners by which undocumented properties can be seen in Matlab. The simplest was reported by Hans Olsson all the way back in 1997, in one of the very earliest posts on the CSSM newsgroup (there is no earlier public report, as far as I could tell). Since then, this method was mentioned in about a dozen other CSSM posts.

By setting the Matlab root (handle 0)’s HideUndocumented property to ‘off’ (default=’on’), subsequent calls to the built-in get and set functions list the hidden properties in addition to the regular ones. Note that HideUndocumented is itself a hidden property, which is why Hans’ post is so important – he presented us with the loose end that then enabled us to untangle the hidden properties in any other HG object.

Here is a simple example, showing HideUndocumented‘s effect on the root (handle 0) object handle itself (hidden properties are highlighted):

% Display only regular properties
>> get(0)
	CallbackObject = []
	CommandWindowSize = [86 51]
	CurrentFigure = []
	Diary = off
	DiaryFile = diary
	Echo = off
	FixedWidthFontName = Courier New
	Format = longG
	FormatSpacing = compact
	Language = he_il
	MonitorPositions = [ (2 by 4) double array]
	More = off
	PointerLocation = [1084 590]
	RecursionLimit = [500]
	ScreenDepth = [32]
	ScreenPixelsPerInch = [96]
	ScreenSize = [1 1 1440 900]
	ShowHiddenHandles = off
	Units = pixels
 
	BeingDeleted = off
	ButtonDownFcn = 
	Children = []
	Clipping = on
	CreateFcn = 
	DeleteFcn = 
	BusyAction = queue
	HandleVisibility = on
	HitTest = on
	Interruptible = on
	Parent = []
	Selected = off
	SelectionHighlight = on
	Tag = 
	Type = root
	UIContextMenu = []
	UserData = []
	Visible = on
 
% Display ALL properties (including hidden ones, which are highlighted below)
>> set(0,'HideUndocumented','off')
>> get(0)
	BlackAndWhite = off	CallbackObject = []
	CommandWindowSize = [86 51]
	CurrentFigure = []
	Diary = off
	DiaryFile = diary
	Echo = off
	ErrorMessage = [ (1 by 79) char array]	FixedWidthFontName = Courier New
	Format = longG
	FormatSpacing = compact
	HideUndocumented = off	Language = he_il
	MonitorPositions = [ (2 by 4) double array]
	More = off
	PointerLocation = [1022 82]
	PointerWindow = [0]	RecursionLimit = [500]
	ScreenDepth = [32]
	ScreenPixelsPerInch = [96]
	ScreenSize = [1 1 1440 900]
	ShowHiddenHandles = off
	Units = pixels
	AutomaticFileUpdates = on 
	BeingDeleted = off
	PixelBounds = [0 0 0 0]	ButtonDownFcn = 
	Children = []
	Clipping = on
	CreateFcn = 
	DeleteFcn = 
	BusyAction = queue
	HandleVisibility = on
	HelpTopicKey = 	HitTest = on
	Interruptible = on
	Parent = []
	Selected = off
	SelectionHighlight = on
	Serializable = on	Tag = 
	Type = root
	UIContextMenu = []
	UserData = []
	ApplicationData = [ (1 by 1) struct array]	Behavior = [ (1 by 1) struct array]	Visible = on
	XLimInclude = on	YLimInclude = on	ZLimInclude = on	CLimInclude = on	ALimInclude = on	IncludeRenderer = on

Property definitions

An entirely different mechanism uses the schema.prop definitions that were presented here by Donn Scull at the beginning of 2011. The idea is to get the object’s classhandle reference, from it to get the list of properties definitions, and for each property look at its Visible meta-property: hidden properties will simply have Visible=’off’, whereas regular properties will have ‘on’.

It turns out that there is not always a full correspondence between these two mechanism. I can’t remember specific examples, and perhaps these were fixed in the latest Matlab releases. It doesn’t matter, because merging the list of hidden properties reported by these two methods is always safe to do. Which is exactly what my getundoc utility does:

getundoc utility

The getundoc utility is based on another utility by the same name, posted by Duane Hanselman to the Matlab File Exchange in 2006 (Duane has elected to remove all his submissions from FEX a year or two ago, but that’s an entirely separate [and extremely heated] topic for a different discussion). Duane’s original getundoc utility relied only on the first (HideUndocumented) mechanism.

I have since expanded this utility to include support for the second mechanism, as well as support for the upcoming HG2 (see below). The updated getundoc is now available for download on the File Exchange. Since it’s a very short utility, I will digress from my norm and simply present its code, in its present form, here:

function c = getundoc(arg)
%GETUNDOC Get Undocumented Object Properties.
% GETUNDOC('OBJECT') or GETUNDOC(H) returns a structure of
% undocumented properties (names & values) for the object having handle
% H or indentified by the string 'OBJECT'.
%
% For example, GETUNDOC('axes') or GETUNDOC(gca) returns undocumented
% property names and values for the axes object.
 
% Extension of Duane Hanselman's original utility (which is no longer
% available on the File Exchange):
% D.C. Hanselman, University of Maine, Orono, ME 04469
% MasteringMatlab@yahoo.com
% Mastering MATLAB 7
% 2006-01-06
 
if nargin~=1
   error('One Input Required.')
end
if ischar(arg) % GETUNDOC('OBJECT')
   switch lower(arg)
   case 'root'                                                       % root
      h=0;
      hf=0;
   case 'figure'                                                   % figure
      h=figure('Visible','off');
      hf=h;
   otherwise                          % some other string name of an object
      hf=figure('Visible','off');
      object=str2func(arg);
      try
         h=object('Parent',hf,'Visible','off');
      catch
         error('Unknown Object Type String Provided.')         
      end
   end
elseif ishandle(arg) % GETUNDOC(H)
   h=arg;
   hf=0;
else
   error('Unknown Object Handle Provided.')
end
 
wstate=warning;
warning off                                      % supress warnings about obsolete properties
try set(0,'HideUndocumented','off'); catch; end  % Fails in HG2
undocfnames=fieldnames(get(h));                  % get props including undocumented
try set(0,'HideUndocumented','on'); catch; end   % Fails in HG2
docfnames=fieldnames(get(h));                    % get props excluding undocumented
 
% Yair 18/3/2010 - add a few more undocs:
try
    % This works in HG1
    props = get(classhandle(handle(h)),'properties');
    undocfnames = [undocfnames; get(props(strcmp(get(props,'Visible'),'off')),'Name')];
catch
    % Yair 18/9/2011: In HG2, the above fails, so use the following workaround:
    try
        prop = findprop(handle(h),undocfnames{1});
        props = prop.DefiningClass.PropertyList;
        undocfnames = [undocfnames; {props.Name}'];   % {props([props.Hidden]).Name}
    catch
        % ignore...
    end
end
 
c = setdiff(undocfnames,docfnames);      % extract undocumented
 
% Get the values in struct format, if relevant
if ~isempty(c)
  s = struct;
  for fieldIdx = 1 : length(c)
      try
          fieldName = c{fieldIdx};
          s.(fieldName) = get(h,fieldName);
      catch
          s.(fieldName) = '???';
      end
  end
  c = s;
end
% Yair end
 
if hf~=0                     % delete hidden figure holding selected object
   delete(hf)
end
warning(wstate)

Usage of this utility is extremely simple:

>> getundoc(0)
ans = 
             ALimInclude: 'on'
         ApplicationData: [1x1 struct]
    AutomaticFileUpdates: 'on'
                Behavior: [1x1 struct]
           BlackAndWhite: 'off'
             CLimInclude: 'on'
            ErrorMessage: [1x79 char]
            HelpTopicKey: ''
        HideUndocumented: 'on'
         IncludeRenderer: 'on'
             PixelBounds: [0 0 0 0]
           PointerWindow: 0
            Serializable: 'on'
             XLimInclude: 'on'
             YLimInclude: 'on'
             ZLimInclude: 'on'
 
>> getundoc(gcf)
ans = 
               ALimInclude: 'on'
    ActivePositionProperty: 'position'
           ApplicationData: [1x1 struct]
              BackingStore: 'on'
                  Behavior: [1x1 struct]
               CLimInclude: 'on'
                CurrentKey: ''
           CurrentModifier: {1x0 cell}
                 Dithermap: [64x3 double]
             DithermapMode: 'manual'
              DoubleBuffer: 'on'
            ExportTemplate: []
               FixedColors: [3x3 double]
                   HelpFcn: ''
              HelpTopicKey: ''
              HelpTopicMap: ''
           IncludeRenderer: 'on'
                 JavaFrame: [1x1 com.mathworks.hg.peer.HG1FigurePeer]
               MinColormap: 64
             OuterPosition: [436 374 568 502]
               PixelBounds: [0 0 560 420]
             PrintTemplate: []
              Serializable: 'on'
                    UseHG2: 'off'
                WaitStatus: []
               XLimInclude: 'on'
               YLimInclude: 'on'
               ZLimInclude: 'on'

Fixes for HG2

Unfortunately, in the new HG2 (which is still not in production, but we must be prepared, mustn’t we?), the original mechanism above (HideUndocumented) fails completely (there is no such property in the new matlab.ui.root object), whereas the second mechanism (UDD property defs) needs to be modified: Apparently, classhandle fails for HG2 object handles. Instead, we use the workaround of using findprop to get the property definition for any regular property, then get its parent (the requested class definition), and then down again to list all available properties. Note that in HG2, the relevant meta-property is Hidden which holds logical (true/false) values, as opposed to Visible and ‘off’/'on’ values for HG1 above.

All of these fixes are now incorporated in the getundoc code that is listed above.

When comparing the list of hidden properties in the existing HG1 and the new HG2, we see many interesting differences. And yes: the figure’s JavaFrame property was indeed removed in HG2. Bummer! (don’t worry – there are several workarounds up my sleeve…)

Do you have any favorite hidden property that you use in your code? If so, please tell us about it in a comment below.

p.s. – For all the numerous good people telling me about cprintf – Yes: I am aware that the latest R2011b release has broken cprintf‘s functionality. I plan to release a workaround sometime soon, when I have some spare time. I’ll keep everybody posted of course. Please be patient. (if you can’t wait, you can always hire me to fix it sooner; otherwise I need to give priority to my paying clients…)

1. Undocumented cursorbar object Matlab's internal undocumented graphics.cursorbar object can be used to present dynamic data-tip cross-hairs...
2. UDD Properties UDD provides a very convenient way to add customizable properties to existing Matlab object handles...
3. Plot LimInclude properties The plot objects' XLimInclude, YLimInclude, ZLimInclude, ALimInclude and CLimInclude properties are an important feature, that has both functional and performance implications....
4. Displaying hidden handle properties I present two ways of checking undocumented hidden properties in Matlab Handle Graphics (HG) handles...

plot data tips

A client has recently asked me to automatically display an attached data-tip to the last data point of a plotted time series of values. The idea was to immediately see what the latest value of the data series is.

Unfortunately, the official documentation clearly says that:

You place data tips only by clicking data objects on graphs. You cannot place them programmatically (by executing code to position a data cursor).

Well, this has never stopped us before, has it?

Creating new data tips

Under the hood, data tips use a data-cursor mode, which shares many similarities in behavior and programming code with the other plot modes (zoom, pan, data-brushing, etc.). At any one time, only a single such mode can be active in any figure window (this is a known limitation of the design). The code itself it actually quite complex and handles numerous edge-cases. Understanding it by simply reading the code (under %matlabroot%\toolbox\matlab\graphics\) is actually pretty difficult. A much easier way to understand the programming flow is to liberally distribute breakpoints (start in datacursormode.m) and interactively activate the functionality, then debug the code step-by-step.

Luckily, it turns out that the code to create a new data-tip is actually quite simple: first get the data-cursor mode object, then create a new data tip using the mode’s createDatatip() method, update some data-tip properties and finally update the data-tip’s position:

% First plot the data
hLine = plot(xdata, ydata);
 
% First get the figure's data-cursor mode, activate it, and set some of its properties
cursorMode = datacursormode(gcf);
set(cursorMode, 'enable','on', 'UpdateFcn',@setDataTipTxt, 'NewDataCursorOnClick',false);
% Note: the optional @setDataTipTxt is used to customize the data-tip's appearance
 
% Note: the following code was adapted from %matlabroot%\toolbox\matlab\graphics\datacursormode.m
% Create a new data tip
hTarget = handle(hLine);
hDatatip = cursorMode.createDatatip(hTarget);
 
% Create a copy of the context menu for the datatip:
set(hDatatip,'UIContextMenu',get(cursorMode,'UIContextMenu'));
set(hDatatip,'HandleVisibility','off');
set(hDatatip,'Host',hTarget);
set(hDatatip,'ViewStyle','datatip');
 
% Set the data-tip orientation to top-right rather than auto
set(hDatatip,'OrientationMode','manual');
set(hDatatip,'Orientation','top-right');
 
% Update the datatip marker appearance
set(hDatatip, 'MarkerSize',5, 'MarkerFaceColor','none', ...
              'MarkerEdgeColor','k', 'Marker','o', 'HitTest','off');
 
% Move the datatip to the right-most data vertex point
position = [xdata(end),ydata(end),1; xdata(end),ydata(end),-1];
update(hDatatip, position);

Updating an existing data tip

To modify the appearance of a data-tip, we first need to get access to the hDatatip object that we created earlier, either programmatically, or interactively (or both). Since we can access pre-stored handles only of programmatically-created (not interactively-created) data-tips, we need to use a different method. There are actually two ways to do this:

The basic way is to search the relevant axes for objects that have Tag=’DataTipMarker’. For each data-tip, we will get two such handles: one for the marker (Type=’line’) and the other for the text box tooltip (Type=’text’). We can use these to update (for example) the marker size, color and style; and the text’s font, border and colors.

A better way is to access the graphics.datatip object itself. This can be done using two hidden properties of the datacursormode object:

% Get the list of all data-tips in the current figure
>> cursorMode = datacursormode(gcf)
cursorMode =
	graphics.datacursormanager
 
>> cursorMode.DataCursors
ans =
	graphics.datatip: 2-by-1
 
>> cursorMode.CurrentDataCursor
ans =
	graphics.datatip
 
>> cursorMode.CurrentDataCursor.get
            Annotation: [1x1 hg.Annotation]
           DisplayName: ''
           HitTestArea: 'off'
          BeingDeleted: 'off'
         ButtonDownFcn: []
              Children: [2x1 double]
              Clipping: 'on'
             CreateFcn: []
             DeleteFcn: []
            BusyAction: 'queue'
      HandleVisibility: 'off'
               HitTest: 'off'
         Interruptible: 'on'
                Parent: 492.005493164063
    SelectionHighlight: 'on'
                   Tag: ''
                  Type: 'hggroup'
              UserData: []
              Selected: 'off'
             FontAngle: 'normal'
              FontName: 'Helvetica'
              FontSize: 8
             FontUnits: 'points'
            FontWeight: 'normal'
             EdgeColor: [0.8 0.8 0.8]
       BackgroundColor: [1 1 0.933333333333333]
             TextColor: [0 0 0]
                Marker: 'o'
            MarkerSize: 5
       MarkerEdgeColor: 'k'
       MarkerFaceColor: 'none'
       MarkerEraseMode: 'normal'
             Draggable: 'on'
                String: {'Date: 01/09/11'  'Value: 573.24'}
               Visible: 'on'
             StringFcn: []
             UpdateFcn: []
         UIContextMenu: [1x1 uicontextmenu]
                  Host: [1x1 graph2d.lineseries]
           Interpolate: 'off'

We can see that the returned graphics.datatip object includes properties of both the text-box and the marker, making it easy to modify. Moreover, we can use its aforementioned update method to move the datatip to a different plot position (see example in the code above). In addition, we can also use the self-explanatory getCursorInfo(), getaxes(), makeCurrent(), movetofront() methods, and a few others.

Cursor mode and data-tip properties

The graphics.datacursormanager and the graphics.datatip objects have several public properties that we can use:

>> cursorMode.get
              Enable: 'off'
    SnapToDataVertex: 'on'
        DisplayStyle: 'datatip'
           UpdateFcn: @setDataTipTxt
              Figure: [1x1 figure]
 
>> cursorMode.CurrentDataCursor.get
            Annotation: [1x1 hg.Annotation]
           DisplayName: ''
           HitTestArea: 'off'
      ... % See the list above

Both these objects have plenty of additional hidden properties. You can inspect them using my uiinspect utility. Here is a brief list for reference (R2011b):

graphics.datacursormanager:

• CurrentDataCursor
• DataCursors
• Debug
• DefaultExportVarName
• DefaultPanelPosition
• EnableAxesStacking
• EnableZStacking
• ExternalListeners
• HiddenUpdateFcn
• NewDataCursorOnClick
• OriginalRenderer
• OriginalRendererMode
• PanelDatatipHandle
• PanelHandle
• PanelTextHandle
• UIContextMenu
• UIState
• ZStackMinimum

graphics.datatip:

• ALimInclude
• ApplicationData
• Behavior
• CLimInclude
• DataCursorHandle
• DataManagerHandle
• Debug
• DoThrowStartDragEvent
• EmptyArgUpdateFcn
• EnableAxesStacking
• EnableZStacking
• EraseMode
• EventObject
• ExternalListenerHandles
• HelpTopicKey
• HostAxes
• HostListenerHandles
• IncludeRenderer
• Invalid
• IsDeserializing
• MarkerHandle
• MarkerHandleButtonDownFcn
• Orientation
• OrientationMode
• OrientationPropertyListener
• OriginalDoubleBufferState
• PixelBounds
• PointsOffset
• Position
• SelfListenerHandles
• Serializable
• TextBoxHandle
• TextBoxHandleButtonDownFcn
• Version
• ViewStyle
• XLimInclude
• YLimInclude
• ZLimInclude
• ZStackMinimum
• uistate

As can be seen, if we really want, we can always use the MarkerHandle or TextBoxHandle directly.

Deleting data tips

To delete a specific data-tip, simply call the cursor mode’s removeDataCursor() method; to delete all data-tips, call its removeAllDataCursors() method:

% Delete the current data-tip
cursorMode.removeDataCursor(cursorMode.CurrentDataCursor)
 
% Delete all data-tips
cursorMode.removeAllDataCursors()

Have you used plot data-tips in some nifty way? If so, please share your experience in a comment below.

p.s. – did you notice that Java was not mentioned anywhere above? Mode managers use pure-Matlab functionality.

1. Accessing plot brushed data Plot data brushing can be accessed programmatically using very simple pure-Matlab code...
2. Plot LineSmoothing property LineSmoothing is a hidden and undocumented plot line property that creates anti-aliased (smooth unpixelized) lines in Matlab plots...
3. Plot LimInclude properties The plot objects' XLimInclude, YLimInclude, ZLimInclude, ALimInclude and CLimInclude properties are an important feature, that has both functional and performance implications....
4. COM/ActiveX tips & tricks This article describes several little-known tips useful for COM / ActiveX programming in Matlab...