Undocumented function – Undocumented Matlab https://undocumentedmatlab.com Charting Matlab's unsupported hidden underbelly Fri, 20 Oct 2017 09:57:44 +0000 en-US hourly 1 https://wordpress.org/?v=4.4.1 Runtime code instrumentationhttps://undocumentedmatlab.com/blog/runtime-code-instrumentation https://undocumentedmatlab.com/blog/runtime-code-instrumentation#comments Thu, 28 Sep 2017 13:36:17 +0000 http://undocumentedmatlab.com/?p=7063
 
Related posts:
  1. Creating a simple UDD class This article explains how to create and test custom UDD packages, classes and objects...
  2. Customizing print setup Matlab figures print-setup can be customized to automatically prepare the figure for printing in a specific configuration...
  3. UDD Events and Listeners UDD event listeners can be used to listen to property value changes and other important events of Matlab objects...
  4. Trapping warnings efficiently warnings can be trapped in a try-catch block just like regular errors. This articles explains how. ...
 
]]>
I regularly follow the MathWorks Pick-of-the-Week (POTW) blog. In a recent post, Jiro Doke highlighted Per Isakson’s tracer4m utility. Per is an accomplished Matlab programmer, who has a solid reputation in the Matlab user community for many years. His utility uses temporary conditional breakpoints to enable users to trace calls to their Matlab functions and class methods. This uses a little-known trick that I wish to highlight in this post.

tracer4m utility uses conditional breakpoints that evaluate but never become live

tracer4m utility uses conditional breakpoints that evaluate but never become live

Matlab breakpoints are documented and supported functionality, and yet their documented use is typically focused at interactive programming in the Matlab editor, or as interactive commands that are entered in the Matlab console using the set of db* functions: dbstop, dbclear, dbstatus, dbstack etc. However, nothing prevents us from using these db* functions directly within our code.

For example, the dbstack function can help us diagnose the calling tree for the current function, in order to do action A if one of the calling ancestors was FunctionX, or to do action B otherwise (for example, to avoid nested recursions).

Similarly, we could add a programmatic call to dbstop in order to stop at a certain code location downstream (for debugging), if a certain condition happens upstream.

Per extended this idea very cleverly in tracer4m: conditional breakpoints evaluate a string in run-time: if the result is true (non-zero) then the code run is stopped at that location, but if it’s false (or zero) then the code run continues normally. To instrument calls to specific functions, Per created a function tracer() that logs the function call (using dbstack) and always returns the value false. He then dynamically created a string that contains a call to this new function and used the dbstop function to create a conditional breakpoint based on this function, something similar to this:

dbstop('in', filename, 'at', location, 'if', 'tracer()');

We can use this same technique for other purposes. For example, if we want to do some action (not necessarily log – perhaps do something else) when a certain code point is reached. The benefit here is that we don’t need to modify the code at all – we’re adding ad-hoc code pieces using the conditional breakpoint mechanism without affecting the source code. This is particularly useful when we do not have access to the source code (such as when it’s compiled or write-protected). All you need to do is to ensure that the instrumentation function always returns false so that the breakpoint does not become live and for code execution to continue normally.

The tracer4m utility is quite sophisticated in the sense that it uses mlint and smart regexp to parse the code and know which functions/methods occur on which line numbers and have which type (more details). In this sense, Per used undocumented functionality. I’m certain that Jiro was not aware of the dependency on undocumented features when he posted about the utility, so please don’t take this to mean that Jiro or MathWorks officially support this or any other undocumented functionality. Undocumented aspects are often needed to achieve top functionality, and I’m happy that the POTW blog highlights utilities based on their importance and merit, even if they do happen to use some undocumented aspect.

tracer4m‘s code also contains references to the undocumented profiler option -history, but this is not in fact used by the code itself, only in comments. I use this feature in my profile_history utility, which displays the function call/timing history in an interactive GUI window. This utility complements tracer4m by providing a lot more information, but this can result in a huge amount of information for large and/or long-running programs. In addition, tracer4m has the benefit of only logging those functions/methods that the user finds useful, rather than all the function call, which enables easier debugging when the relevant code area is known. In short, I wish I had known about tracer4m when I created profile_history. Now that I know about it, maybe I’ll incorporate some of its ideas into profile_history in order to make it more useful. Perhaps another moral of this is that we should actively monitor the POTW blog, because true gems are quite often highlighted there.

Function call timeline profiling
Function call timeline profiling

For anyone who missed the announcement in my previous post, I’m hosting a series of live webinars on advanced Matlab topics in the upcoming 2 weeks – I’ll be happy if you would join.

]]>
https://undocumentedmatlab.com/blog/runtime-code-instrumentation/feed 3
MathWorks-solicited Java surveyhttps://undocumentedmatlab.com/blog/mathworks-solicited-java-survey https://undocumentedmatlab.com/blog/mathworks-solicited-java-survey#comments Wed, 22 Mar 2017 22:05:34 +0000 http://undocumentedmatlab.com/?p=6866
 
Related posts:
  1. Minimize/maximize figure window Matlab figure windows can easily be maximized, minimized and restored using a bit of undocumented magic powder...
  2. Types of undocumented Matlab aspects This article lists the different types of undocumented/unsupported/hidden aspects in Matlab...
  3. FindJObj – find a Matlab component’s underlying Java object The FindJObj utility can be used to access and display the internal components of Matlab controls and containers. This article explains its uses and inner mechanism....
  4. Matlab callbacks for Java events Events raised in Java code can be caught and handled in Matlab callback functions - this article explains how...
 
]]>
Over the years I’ve reported numerous uses for integrating Java components and functionality in Matlab. As I’ve also recently reported, MathWorks is apparently making a gradual shift away from standalone Java-based figures, toward browser-based web-enabled figures. As I surmised a few months ago, MathWorks has created dedicated surveys to solicit user feedbacks on the most important (and undocumented) non-compatible aspects of this paradigm change: one regarding users’ use of the javacomponent function, the other regarding the use of the figure’s JavaFrame property:

In MathWorks’ words:

In order to extend your ability to build MATLAB apps, we understand you sometimes need to make use of undocumented Java UI technologies, such as the JavaFrame property. In response to your needs, we are working to develop documented alternatives that address gaps in our app building offerings.

To help inform our work and plans, we would like to understand how you are using the JavaFrame property. Based on your understanding of how it is being used within your app, please take a moment to fill out the following survey. The survey will take approximately 1-2 minutes to finish.

I urge anyone who uses one or both of these features to let MathWorks know how you’re using them, so that they could incorporate that functionality into the core (documented) Matlab. The surveys are really short and to the point. If you wish to send additional information, please email George.Caia at mathworks.com.

The more feedback responses that MathWorks will get, the better it will be able to prioritize its R&D efforts for the benefit of all users, and the more likely are certain features to get a documented solution at some future release. If you don’t take the time now to tell MathWorks how you use these features in your code, don’t complain if and when they break in the future…

My personal uses of these features

  • Functionality:
    • Figure: maximize/minimize/restore, enable/disable, always-on-top, toolbar controls, menu customizations (icons, tooltips, font, shortcuts, colors)
    • Table: sorting, filtering, grouping, column auto-sizing, cell-specific behavior (tooltip, context menu, context-sensitive editor, merging cells)
    • Tree control
    • Listbox: cell-specific behavior (tooltip, context menu)
    • Tri-state checkbox
    • uicontrols in general: various event callbacks (e.g. mouse hover/unhover, focus gained/lost)
    • Ability to add Java controls e.g. color/font/date/file selector panel or dropdown, spinner, slider, search box, password field
    • Ability to add 3rd-party components e.g. JFreeCharts, JIDE controls/panels

  • Appearance:
    • Figure: undecorated (frameless), other figure frame aspects
    • Table: column/cell-specific rendering (alignment, icons, font, fg/bg color, string formatting)
    • Listbox: auto-hide vertical scrollbar as needed, cell-specific renderer (icon, font, alignment, fg/bg color)
    • Button/checkbox/radio: icons, text alignment, border customization, Look & Feel
    • Right-aligned checkbox (button to the right of label)
    • Panel: border customization (rounded/matte/…)

You can find descriptions/explanations of many of these in posts I made on this website over the years.

]]>
https://undocumentedmatlab.com/blog/mathworks-solicited-java-survey/feed 2
Checking status of warning messages in MEXhttps://undocumentedmatlab.com/blog/checking-status-of-warning-messages-in-mex https://undocumentedmatlab.com/blog/checking-status-of-warning-messages-in-mex#respond Wed, 21 Dec 2016 15:24:06 +0000 http://undocumentedmatlab.com/?p=6797
 
Related posts:
  1. Inactive Control Tooltips & Event Chaining Inactive Matlab uicontrols cannot normally display their tooltips. This article shows how to do this with a combination of undocumented Matlab and Java hacks....
  2. Introduction to UDD UDD classes underlie many of Matlab's handle-graphics objects and functionality. This article introduces these classes....
  3. Creating a simple UDD class This article explains how to create and test custom UDD packages, classes and objects...
  4. Undocumented Matlab MEX API Matlab's MEX API contains numerous undocumented functions, that can be extremely useful. ...
 
]]>
Once again I would like to welcome guest blogger Pavel Holoborodko, the developer of the Advanpix Multiprecision Computing Toolbox. Pavel has already posted here as a guest blogger about undocumented Matlab MEX functions. Today he will discuss another little-known aspect of advanced MEX programming with Matlab, a repost of an article that was originally posted on his own blog. Happy holidays everybody!

Matlab allows flexible adjustment of visibility of warning messages. Some, or even all, messages can be disabled from showing on the screen by warning command.

The little known fact is that status of some warnings may be used to change the execution path in algorithms. For example, if warning 'Matlab:nearlySingularMatrix' is disabled, then the linear system solver (mldivide operator) might skip estimation of reciprocal condition number which is used exactly for the purpose of detection of nearly singular matrices. If the trick is used, it allows 20%-50% boost in solver performance, since rcond estimation is a time consuming process.

Therefore it is important to be able to retrieve status of warnings in Matlab. Especially in MEX libraries targeted for improved performance. Unfortunately Matlab provides no simple way to check status of warning message from MEX module.

Today’s article outlines two workarounds for the issue:

  1. Using mexCallMATLABWithTrap (documented)
  2. Using utGetWarningStatus (undocumented)

Using mexCallMATLABWithTrap (documented)

The first idea is to use documented mexCallMATLABWithTrap function to execute warning(‘query’,…) command using Matlab’s interpreter and then parse the returned result:

bool mxIsWarningEnabled(const char* warningId)
{
    bool enabled = true;
 
    if (NULL != warningId)
    {
        mxArray *mxCommandResponse = NULL, *mxException = NULL;
        mxArray *args[2];
 
        /* warning('query', warningId); */
        args[0] = mxCreateString("query");
        args[1] = mxCreateString(warningId);
        mxException = mexCallMATLABWithTrap(1,&mxCommandResponse,2,args,"warning");
        if (NULL == mxException && NULL != mxCommandResponse)
        {
            if (mxIsStruct(mxCommandResponse))
            {
                const mxArray* state_field = mxGetField(mxCommandResponse, 0, "state");
                if (mxIsChar(state_field))
                {
                    char state_value[8] = {0};
                    enabled = (0 == mxGetString(state_field, state_value, 8)) &&
                              (0 == strcmp(state_value,"on"));
                }
            }
            mxDestroyArray(mxCommandResponse);
        }
        else
        {
            /* 'warning' returned with error */
            mxDestroyArray(mxException);
        }
        mxDestroyArray(args[0]);
        mxDestroyArray(args[1]);
    }
    return enabled;
}

This approach is slow, but works fine in most standard situations. See the bottom of this post for a usage example.

However, this approach has an important drawback – we should be careful with recursive calls to the Matlab interpreter (Matlab -> MEX -> Matlab) and with handling Matlab errors in MEX. It is safe only if we use identical standard libraries and compiler to build both MEX and Matlab.

In other cases, for example when MEX is targeted to work with different versions of Matlab, or was built with a different standard library and compiler, etc. – cross boundary handling of errors (which are just C++ exceptions) might lead to unpredictable results, most likely segfaults.

Using utGetWarningStatus (undocumented)

To avoid all the overhead of calling Matlab interpreter and unsafe error handling, we can use some undocumented internal Matlab functions:

/* Link with libut library to pick-up undocumented functions: */
extern "C" void* utGetWarningManagerContext(void);
extern "C" bool  utIsValidMessageIdentifier(const char *warningId);
extern "C" bool  utGetWarningStatus(void* context, const char *warningId);
 
/* 
   Returns true if warning with warningId enabled 
   Matlab versions supported/tested: R2008b - R2016b
*/
bool mxIsWarningEnabled(const char *warningId)
{
    bool enabled = true;
 
    if (NULL != warningId && utIsValidMessageIdentifier(warningId))
    {
        void* context = utGetWarningManagerContext();
        enabled = (NULL != context) && utGetWarningStatus(context, warningId);
    }
    return enabled;
}

Now the code is clean, fast and safe – we bypass the interpreter and work directly with Matlab kernel. All the undocumented functions involved are present in Matlab for at least 10 years and do simple logical checks under the hood.

The standard function mexWarnMsgIdAndTxt uses similar code to check if it should display the warning or just suppress it, and that code remains unchanged since R2008b. This is a good indication of code stability and makes us believe that it will not be changed in future versions of Matlab.

For both workarounds, usage is simple:

if (mxIsWarningEnabled("Matlab:nearlySingularMatrix"))
{
   /* compute rcond */
}
else
{
   /* do something else */
}
]]>
https://undocumentedmatlab.com/blog/checking-status-of-warning-messages-in-mex/feed 0
A few parfor tipshttps://undocumentedmatlab.com/blog/a-few-parfor-tips https://undocumentedmatlab.com/blog/a-few-parfor-tips#comments Wed, 06 Jul 2016 16:29:21 +0000 http://undocumentedmatlab.com/?p=6516
 
Related posts:
  1. ismembc – undocumented helper function Matlab has several undocumented internal helper functions that can be useful on their own in some cases. This post presents the ismembc function....
  2. Datenum performance The performance of the built-in Matlab function datenum can be significantly improved by using an undocumented internal help function...
  3. sprintfc – undocumented helper function The built-in sprintfc function can be used to quickly generate a cell-array of formatted strings. ...
  4. Profiling Matlab memory usage mtic and mtoc were a couple of undocumented features that enabled users of past Matlab releases to easily profile memory usage. ...
 
]]>
Matlab Expo 2016 keynote presentation

Matlab Expo 2016 keynote presentation

A few days ago, MathWorks uploaded a video recording of my recent keynote presentation at the Matlab Expo 2016 in Munich, Germany. During the presentation, I skimmed over a few tips for improving performance of parallel-processing (parfor) loops. In today’s post I plan to expand on these tips, as well as provide a few others that for lack of space and time I did not mention in the presentation.

The overall effect can be dramatic: The performance (speed) difference between a sub-optimal and optimized parfor‘ed code can be up to a full order of magnitude, depending on the specific situation. Naturally, to use any of today’s tips, you need to have MathWorks’ Parallel Computing Toolbox (PCT).

Before diving into the technical details, let me say that MathWorks has extensive documentation on PCT. In today’s post I will try not to reiterate the official tips, but rather those that I have not found mentioned elsewhere, and/or are not well-known (my apologies in advance if I missed an official mention of one or more of the following). Furthermore, I limit myself only to parfor in this post: much can be said about spmd, GPU and other parallel constructs, but not today.

parpool(numCores)

The first tip is to not [always] use the default number of workers created by parpool (or matlabpool in R2013a or earlier). By default, Matlab creates as many workers as logical CPU cores. On Intel CPUs, the OS reports two logical cores per each physical core due to hyper-threading, for a total of 4 workers on a dual-core machine. However, in many situations, hyperthreading does not improve the performance of a program and may even degrade it (I deliberately wish to avoid the heated debate over this: you can find endless discussions about it online and decide for yourself). Coupled with the non-negligible overhead of starting, coordinating and communicating with twice as many Matlab instances (workers are headless [=GUI-less] Matlab processes after all), we reach a conclusion that it may actually be better in many cases to use only as many workers as physical (not logical) cores.

I know the documentation and configuration panel seem to imply that parpool uses the number of physical cores by default, but in my tests I have seen otherwise (namely, logical cores). Maybe this is system-dependent, and maybe there is a switch somewhere that controls this, I don’t know. I just know that in many cases I found it beneficial to reduce the number of workers to the actual number of physical cores:

p = parpool;     % use as many workers as logical CPUs (4 on my poor laptop...)
p = parpool(2);  % use only 2 parallel workers

Of course, this can vary greatly across programs and platforms, so you should test carefully on your specific setup. I suspect that for the majority of Matlab programs it would turn out that using the number of physical cores is better.

It would of course be better to dynamically retrieve the number of physical cores, rather than hard-coding a constant value (number of workers) into our program. We can get this value in Matlab using the undocumented feature(‘numcores’) function:

numCores = feature('numcores');
p = parpool(numCores);

Running feature(‘numcores’) without assigning its output displays some general debugging information:

>> feature('numcores')
MATLAB detected: 2 physical cores.
MATLAB detected: 4 logical cores.
MATLAB was assigned: 4 logical cores by the OS.
MATLAB is using: 2 logical cores.
MATLAB is not using all logical cores because hyper-threading is enabled.
ans =
     2

Naturally, this specific tip is equally valid for both parfor loops and spmd blocks, since both of them use the pool of workers started by parpool.

Running separate code in parfor loops

The conventional wisdom is that parfor loops (and loops in general) can only run a single code segment over all its iterations. Of course, we can always use conditional constructs (such as if or switch) based on the data. But what if we wanted some workers to run a different code path than the other workers? In spmd blocks we could use a conditional based on the labindex value, but unfortunately labindex is always set to the same value 1 within parfor loops. So how can we let worker A run a different code path than worker B?

An obvious answer is to create a parfor loop having as many elements as there are separate code paths, and use a switch-case mechanism to run the separate paths, as follows:

% Naive implementation example - do NOT use!
parfor idx = 1 : 3
   switch idx
      case 1,  result{1} = fun1(data1, data2);
      case 2,  result{2} = fun2(data3, data4, data5);
      case 3,  result{3} = fun3(data6);
   end
end

There are several problems with this naive implementation. First, it unnecessarily broadcasts all the input data to all workers (more about this issue below). Secondly, it appears clunky and too verbose. A very nice extension of this mechanism, posted by StackOverflow heavyweight Jonas, uses indexed arrays of function handles and input args, thereby solving both problems:

funcList = {@fun1, @fun2, @fun3};
dataList = {data1, data2, data3};  %# or pass file names 
parfor idx = 1 : length(funcList)
    result{idx} = funcList{idx}(dataList{idx});
end

Reduce the amount of broadcast data

It is often easy, too-easy, to convert for loops into parfor loops. In many cases, all we need to do is to add the “par” prefix to the for keyword and we’re done (assuming we have no incompatibly-used variables that should be converted into sliced variables etc.). This transformation was intentionally made simple by MathWorks (which is great!). On the other hand, it also hides a lot under the hood. One of the things that is often overlooked in such simple loop transformations is that a large part of the data used within the loop needs to be copied (broadcast) to each of the workers separately. This means that each of the data items needs to be serialized (i.e., copied in memory), packaged, communicated to and accepted by each of the workers. This can mean a lot of memory, networking bandwidth and time-consuming. It can even mean thrashing to hard-disk in case the number of workers times the amount of transferred data exceeds the available RAM. For example, if we have 10GB available RAM and try to communicate 3GB to 4 workers, we will not have enough RAM and the OS will start swapping to hard-disk. This will kill performance and Matlab will appear “hung” and will need to be hard-killed.

You might think that it would be very difficult to reach the RAM limit, but in fact it can be far too easy when you consider the multiplication by the number of workers, and the fact that each worker uses 1+GB of memory just for its MATLAB process, even before the data, and all this in addition to the parent (client) Matlab process. That’s a lot of GBs flying around…

Moreover, it’s enough for one small part of a Matlab struct or array to be used within the parfor loop for the entire Matlab construct to be broadcast to all workers. For example, a very common use-case is to store program data, both raw and processed, within a simple Matlab struct. Let’s say that we have data.raw and data.processed and within the loop we only need data.processed – the entire data variable (which might include many GBs due to the raw data) is broadcast, although the loop’s code only needs data.processed. In such cases, it makes sense to separate the broadcast data into standalone variables, and only use them within the loop:

data.raw = ...
data.processed = ...
 
% Inefficient variant:
parfor idx = 1 : N
   % do something with data.processed
end
 
% This is better:
processedData = data.processed;
parfor idx = 1 : N
   % do something with processedData
end

Moreover, if you can convert a broadcast variable into a sliced one, this would be even better: in this case each worker will only be communicated its small share (“slice”) of the entire data, rather than a full copy of the entire data.

All this would of course be much simpler if Matlab’s computational engine was multi-threaded, since then PCT could be implemented using lightweight threads rather than heavyweight processes. The memory and communication overheads would then be drastically reduced and performance would improve significantly. Unfortunately, Matlab’s computational engine is [still] single-threaded, preventing this. Hopefully Matlab’s new engine (which debuted in R2015b) will enable true multithreading at some future release. PCT will still need to retain an option of using headless worker processes to run on multiple machines (i.e., distributed/grid/cloud computing), but single-machine parallelization should employ multithreading instead.

Additional speedup tips can be found in my book “Accelerating MATLAB Performance“.

Do you have some other important parfor tips that you found useful? If so, please post them in a comment below.

]]>
https://undocumentedmatlab.com/blog/a-few-parfor-tips/feed 6
MEX ctrl-c interrupthttps://undocumentedmatlab.com/blog/mex-ctrl-c-interrupt https://undocumentedmatlab.com/blog/mex-ctrl-c-interrupt#comments Wed, 15 Jun 2016 17:00:40 +0000 http://undocumentedmatlab.com/?p=6454
 
Related posts:
  1. Undocumented Matlab MEX API Matlab's MEX API contains numerous undocumented functions, that can be extremely useful. ...
  2. Serializing/deserializing Matlab data Matlab does not provide a documented manner to serialize data into a byte stream, but we can do this with some undocumented functionality. ...
  3. Undocumented feature list A list of undocumented MATLAB features can be retrieved. Here's how... ...
  4. Matlab’s internal memory representation Matlab's internal memory structure is explored and discussed. ...
 
]]>
I recently became aware of a very nice hack by Wotao Yin (while at Rice in 2010; currently teaching at UCLA). The core problem is that unlike m-files that can be interrupted in mid-run using ctrl-c, MEX functions cannot be interrupted in the same way. Well, not officially, that is.

Interrupts are very important for long-running user-facing operations. They can even benefit performance by avoiding the need to periodically poll some external state. Interrupts are registered asynchronously, and the program can query the interrupt buffer at its convenience, in special locations of its code, and/or at specific times depending on the required responsiveness.

Yin reported that the libut library that ships with Matlab contain a large set of undocumented functions, including utIsInterruptPending() that can be used to detect ctrl-c interrupt events. The original report of this feature seems to be by Matlab old hand Peter Boettcher back in 2002 (with a Fortran wrapper reported in 2013). The importance of Yin’s post is that he clearly explained the use of this feature, with detailed coding and compilation instructions. Except for Peter’s original report, Yin’s post and the Fortran wrapper, precious few mentions can be found online (oddly enough, yours truly mentioned it in the very same CSSM newsletter post in which I outed this blog back in 2009). Apparently, this feature was supposed to have been made documented in R12.1, but for some reason it was not and people just moved on and forgot about it.

The relevant functions seem to be:

// Most important functions (C):
bool utIsInterruptEnabled(void)
bool utIsInterruptPending(void)
bool utWasInterruptHandled(void)
 
bool utSetInterruptHandled(bool)
bool utSetInterruptEnabled(bool)
bool utSetInterruptPending(bool)
 
// Related functions (C, signature unknown):
? utHandlePendingInterrupt(?)
? utRestoreInterruptEnabled(?)
? utLongjmpIfInterruptPending(?)
 
// utInterruptMode class (C++):
utInterruptMode::utInterruptMode(enum utInterruptMode::Mode)  // constructor
utInterruptMode::~utInterruptMode(void)  // destructor
bool utInterruptMode::isInterruptEnabled(void)
enum utInterruptMode::Mode utInterruptMode::CurrentMode
enum utInterruptMode::Mode utInterruptMode::GetCurrentMode(void)
enum utInterruptMode::Mode utInterruptMode::GetOriginalMode(void)
enum utInterruptMode::Mode utInterruptMode::SetMode(enum utInterruptMode::Mode)
 
// utInterruptState class (C++):
class utInterruptState::AtomicPendingFlags utInterruptState::flags_pending
void utInterruptState::HandlePeekMsgPending(void)
bool utInterruptState::HandlePendingInterrupt(void)
bool utInterruptState::interrupt_handled
bool utInterruptState::IsInterruptPending(void)
bool utInterruptState::IsPauseMsgPending(void)
class utInterruptState & utInterruptState::operator=(class utInterruptState const &)
void utInterruptState::PeekMessageIfPending(void)
bool utInterruptState::SetInterruptHandled(bool)
bool utInterruptState::SetInterruptPending(bool)
bool utInterruptState::SetIqmInterruptPending(bool)
bool utInterruptState::SetPauseMsgPending(bool)
bool utInterruptState::SetPeekMsgPending(bool)
void utInterruptState::ThrowIfInterruptPending(void)
bool utInterruptState::WasInterruptHandled(void)
unsigned int const utInterruptState::FLAG_PENDING_CTRLC
unsigned int const utInterruptState::FLAG_PENDING_INTERRUPT_MASK
unsigned int const utInterruptState::FLAG_PENDING_IQM_INTERRUPT
unsigned int const utInterruptState::FLAG_PENDING_PAUSE
unsigned int const utInterruptState::FLAG_PENDING_PEEKMSG

Of all these functions, we can make do with just utIsInterruptPending, as shown by Yin (complete with compilation instructions):

/* A demo of Ctrl-C detection in mex-file by Wotao Yin. Jan 29, 2010. */
 
#include "mex.h"
 
#if defined (_WIN32)
    #include <windows.h>
#elif defined (__linux__)
    #include <unistd.h>
#endif
 
#ifdef __cplusplus 
    extern "C" bool utIsInterruptPending();
#else
    extern bool utIsInterruptPending();
#endif
 
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]) {
    int count = 0;    
    while(1) {
        #if defined(_WIN32)
            Sleep(1000);        /* Sleep one second */
        #elif defined(__linux__)
            usleep(1000*1000);  /* Sleep one second */
        #endif
 
        mexPrintf("Count = %d\n", count++);  /* print count and increase it by 1 */
        mexEvalString("drawnow;");           /* flush screen output */
 
        if (utIsInterruptPending()) {        /* check for a Ctrl-C event */
            mexPrintf("Ctrl-C Detected. END\n\n");
            return;
        }
        if (count == 10) {
            mexPrintf("Count Reached 10. END\n\n");
            return;
        }
    }
}

An elaboration of this idea was created by Ramon Casero (Oxford) for the Gerardus project. Ramon wrapped Yin’s code in C/C++ #define to create an easy-to-use pre-processor function ctrlcCheckPoint(fileName,lineNumber):

...
ctrlcCheckPoint(__FILE__, __LINE__);  // exit if user pressed Ctrl+C
...

Here’s the code for the preprocessor header file (GerardusCommon.h) that #defines ctrlcCheckPoint() (naturally, the __FILE__ and __LINE__ parts could also be made part of the #define, for even simpler usage):

 /*
  * Author: Ramon Casero <rcasero@gmail.com>
  * Copyright © 2011-2013 University of Oxford
  * Version: 0.10.2
  *
  * University of Oxford means the Chancellor, Masters and Scholars of
  * the University of Oxford, having an administrative office at
  * Wellington Square, Oxford OX1 2JD, UK. 
  *
  * This file is part of Gerardus.
  *
  * This program is free software: you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
  * the Free Software Foundation, either version 3 of the License, or
  * (at your option) any later version.
  *
  * This program is distributed in the hope that it will be useful,
  * but WITHOUT ANY WARRANTY; without even the implied warranty of
  * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  * GNU General Public License for more details. The offer of this
  * program under the terms of the License is subject to the License
  * being interpreted in accordance with English Law and subject to any
  * action against the University of Oxford being under the jurisdiction
  * of the English Courts.
  *
  * You should have received a copy of the GNU General Public License
  * along with this program.  If not, see
  * <http://www.gnu.org/licenses/>.
  */
 
#ifndef GERARDUSCOMMON_H
#define GERARDUSCOMMON_H
 
/* mex headers */
#include <mex.h>
 
/* C++ headers */
#include <iostream>
#include <algorithm>
#include <iterator>
#include <string>
#include <vector>
 
/* ITK headers */
#include "itkOffset.h"
 
/*
 * utIsInterruptPending(): "undocumented MATLAB API implemented in
 * libut.so, libut.dll, and included in the import library
 * libut.lib. To use utIsInterruptPending in a mex-file, one must
 * manually declare bool utIsInterruptPending() because this function
 * is not included in any header files shipped with MATLAB. Since
 * libut.lib, by default, is not linked by mex, one must explicitly
 * tell mex to use libut.lib." -- Wotao Yin, 
 * http://www.caam.rice.edu/~wy1/links/mex_ctrl_c_trick/
 *
 */
#ifdef __cplusplus 
    extern "C" bool utIsInterruptPending();
#else
    extern bool utIsInterruptPending();
#endif
 
/*
 * ctrlcCheckPoint(): function to check whether the user has pressed
 * Ctrl+C, and if so, terminate execution returning an error message
 * with a hyperlink to the offending function's help, and a hyperlink
 * to the line in the source code file this function was called from
 *
 * It is implemented as a C++ macro to check for the CTRL+C flag, and
 * a call to function ctrlcErrMsgTxt() inside, to throw the error. The
 * reason is that if ctrlcCheckPoint() were a function instead of a
 * macro, this would introduce a function call at every iteration of
 * the loop, which is very expensive. But then we don't want to put
 * the whole error message part inside a macro, it's bug-prone and bad
 * programming practice. And once the CTRL+C has been detected,
 * whether the error message is generated a bit faster or not is not
 * important.
 *
 * In practice, to use this function put a call like this e.g. inside
 * loops that may take for a very long time:
 *
 *    // exit if user pressed Ctrl+C
 *    ctrlcCheckPoint(__FILE__, __LINE__);
 *
 * sourceFile: full path and name of the C++ file that calls this
 *             function. This should usually be the preprocessor
 *             directive __FILE__
 *
 * lineNumber: line number where this function is called from. This
 *             should usually be the preprocessor directive __LINE__
 *
 */
inline
void ctrlcErrMsgTxt(std::string sourceFile, int lineNumber) {
 
  // run from here the following code in the Matlab side:
  //
  // >> path = mfilename('fullpath')
  //
  // this provides the full path and function name of the function
  // that called ctrlcCheckPoint()
  int nlhs = 1; // number of output arguments we expect
  mxArray *plhs[1]; // to store the output argument
  int nrhs = 1; // number of input arguments we are going to pass
  mxArray *prhs[1]; // to store the input argument we are going to pass
  prhs[0] = mxCreateString("fullpath"); // input argument to pass
  if (mexCallMATLAB(nlhs, plhs, nrhs, prhs, "mfilename")) { // run mfilename('fullpath')
    mexErrMsgTxt("ctrlcCheckPoint(): mfilename('fullpath') returned error");
  }
  if (plhs == NULL) {
    mexErrMsgTxt("ctrlcCheckPoint(): mfilename('fullpath') returned NULL array of outputs");
  }
  if (plhs[0] == NULL) {
    mexErrMsgTxt("ctrlcCheckPoint(): mfilename('fullpath') returned NULL output instead of valid path");
  }
 
  // get full path to current function, including function's name
  // (without the file extension)
  char *pathAndName = mxArrayToString(plhs[0]);
  if (pathAndName == NULL) {
    mexErrMsgTxt("ctrlcCheckPoint(): mfilename('fullpath') output cannot be converted to string");
  }
 
  // for some reason, using mexErrMsgTxt() to give this output
  // doesn't work. Instead, we have to give the output to the
  // standar error, and then call mexErrMsgTxt() to terminate
  // execution of the program
  std::cerr << "Operation terminated by user during "
	    << "<a href=\"matlab:helpUtils.errorDocCallback('"
	    << mexFunctionName()
	    << "', '" << pathAndName << ".m', " << lineNumber << ")\">"
	    << mexFunctionName()
	    << "</a> (<a href=\"matlab:opentoline('"
	    << sourceFile
	    << "'," << lineNumber << ",0)\">line " << lineNumber
	    << "</a>)"
	    << std::endl;
  mexErrMsgTxt("");
}
 
#define ctrlcCheckPoint(sourceFile, lineNumber)		\
  if (utIsInterruptPending()) {				\
    ctrlcErrMsgTxt(sourceFile, lineNumber);		\
  }

This feature has remained as-is since at least 2002 (when Peter first reported it), and apparently works to this day. Why then did I categorize this as “High risk for breaking in a future Matlab versions”? The reason is that internal undocumented MEX functions are prone to break in new Matlab releases (example). Hopefully my report today will prompt MathWorks to make this feature documented, rather than to remove it from a future release :-)

By the way, if anyone knows any use for the other interrupt-related functions in libut that I listed above, and/or the missing signatures, please leave a note below and I will update here accordingly.

Addendum July 2, 2016: Pavel Holoborodko just posted an asynchronous version of this mechanism, which is an improvement of the synchronous code above. Pavel uses a separate Windows thread to check the Ctrl-C interrupt state. Readers can extend this idea to use threads for other asynchronous (multi-threaded) computations or I/O. In chapter 7 of my book “Accelerating MATLAB Performance” I explain how we can use Posix threads (pthreads) or OpenMP threads for similar multithreading in MEX (unlike Windows threads, pthreads and OpenMP are cross platform). Users can also use other multithreading solutions, such as the open-source Boost library (bundled with Matlab) or Intel’s commercial TBB.

]]>
https://undocumentedmatlab.com/blog/mex-ctrl-c-interrupt/feed 5
Enabling user callbacks during zoom/panhttps://undocumentedmatlab.com/blog/enabling-user-callbacks-during-zoom-pan https://undocumentedmatlab.com/blog/enabling-user-callbacks-during-zoom-pan#comments Wed, 28 Oct 2015 12:42:03 +0000 http://undocumentedmatlab.com/?p=6043
 
Related posts:
  1. Introduction to UDD UDD classes underlie many of Matlab's handle-graphics objects and functionality. This article introduces these classes....
  2. Multi-column (grid) legend This article explains how to use undocumented axes listeners for implementing multi-column plot legends...
  3. Pinning annotations to graphs Annotation object can be programmatically set at, and pinned-to, plot axes data points. ...
  4. uiundo – Matlab’s undocumented undo/redo manager The built-in uiundo function provides easy yet undocumented access to Matlab's powerful undo/redo functionality. This article explains its usage....
 
]]>
An annoying side-effect of Matlab figure uimodes (zoom, pan and rotate3d) is that they disable the user’s figure-wide callbacks (KeyPressFcn, KeyReleaseFcn, WindowKeyPressFcn, WindowKeyReleaseFcn, WindowButtonUpFcn, WindowButtonDownFcn and WindowScrollWheelFcn). In most cases, users will indeed expect the mouse and keyboard behavior to change when these modes are active (selecting a zoom region with the mouse for instance). However, in certain cases we may need to retain our custom callback behavior even when the modes are active. This is particularly relevant for the keyboard callbacks, which are not typically used to control the modes and yet may be very important for our figure’s interactive functionality. Unfortunately, Matlab’s mode manager installs property listeners that prevent users from modifying these callback when any mode is active:

>> hFig=figure; plot(1:5); zoom on
>> set(hFig, 'KeyPressFcn', @myKeyPressCallback)
Warning: Setting the "KeyPressFcn" property is not permitted while this mode is active.
(Type "warning off MATLAB:modes:mode:InvalidPropertySet" to suppress this warning.)
 
> In matlab.uitools.internal.uimodemanager>localModeWarn (line 211)
  In matlab.uitools.internaluimodemanager>@(obj,evd)(localModeWarn(obj,evd,hThis)) (line 86) 
 
>> get(hFig, 'KeyPressFcn')  % the KeyPressFcn callback set by the mode manager
ans = 
    @localModeKeyPressFcn               
    [1x1 matlab.uitools.internal.uimode]
    {1x2 cell                          }

The question of how to override this limitation has appeared multiple times over the years in the CSSM newsgroup (example1, example2) and the Answers forum, most recently yesterday, so I decided to dedicate today’s article to this issue.

In Matlab GUI, a “mode” (aka uimode) is a set of defined behaviors that may be set for any figure window and activated/deactivated via the figure’s menu, toolbar or programmatically. Examples of predefined modes are plot edit, zoom, pan, rotate3d and data-cursor. Only one mode can be active in a figure at any one time – this is managed via a figure’s ModeManager. Activating a figure mode automatically deactivates any other active mode, as can be seen when switching between plot edit, zoom, pan, rotate3d and data-cursor modes.

This mode manager is implemented in the uimodemanager class in %matlabroot%/toolbox/matlab/uitools/+matlab/+uitools/internal/@uimodemanager/uimodemanager.m and its sibling functions in the same folder. Until recently it used to be implemented in Matlab’s old class system, but was recently converted to the new MCOS (classdef) format, without an apparent major overhaul of the underlying logic (which I think is a pity, but refactoring naturally has lower priority than fixing bugs or adding new functionality).

Anyway, until HG2 (R2014b), the solution for bypassing the mode-manager’s callbacks hijacking was as follows (credit: Walter Roberson):

% This will only work in HG1 (R2014a or earlier)
hManager = uigetmodemanager(hFig);
set(hManager.WindowListenerHandles, 'Enable', 'off');
set(hFig, 'WindowKeyPressFcn', []);
set(hFig, 'KeyPressFcn', @myKeyPressCallback);

In HG2 (R2014b), the interface of WindowListenerHandles changed and the above code no longer works:

>> set(hManager.WindowListenerHandles, 'Enable', 'off');
Error using set
Cannot find 'set' method for event.proplistener class.

Luckily, in MathWorks’ refactoring to MCOS, the property name WindowListenerHandles was not changed, nor its status as a public read-only hidden property. So we can still access it directly. The only thing that changed is its meta-properties, and this is due not to any change in the mode-manager’s code, but rather to a much more general change in the implementation of the event.proplistener class:

>> hManager.WindowListenerHandles(1)
ans = 
  proplistener with properties:
 
       Object: {[1x1 Figure]}
       Source: {7x1 cell}
    EventName: 'PreSet'
     Callback: @(obj,evd)(localModeWarn(obj,evd,hThis))
      Enabled: 0
    Recursive: 0

In the new proplistener, the Enabled meta-property is now a boolean (logical) value rather than a string, and was renamed Enabled in place of the original Enable. Also, the new proplistener doesn’t inherit hgsetget, so it does not have set and get methods, which is why we got an error above; instead, we should use direct dot-notation.

In summary, the code that should now be used, and is backward compatible with old HG1 releases as well as new HG2 releases, is as follows:

% This should work in both HG1 and HG2:
hManager = uigetmodemanager(hFig);
try
    set(hManager.WindowListenerHandles, 'Enable', 'off');  % HG1
catch
    [hManager.WindowListenerHandles.Enabled] = deal(false);  % HG2
end
set(hFig, 'WindowKeyPressFcn', []);
set(hFig, 'KeyPressFcn', @myKeyPressCallback);

During an active mode, the mode-manager disables user-configured mouse and keyboard callbacks, in order to prevent interference with the mode’s functionality. For example, mouse clicking during zoom mode has a very specific meaning, and user-configured callbacks might interfere with it. Understanding this and taking the proper precautions (for example: chaining the default mode’s callback at the beginning or end of our own callback), we can override this default behavior, as shown above.

Note that the entire mechanism of mode-manager (and the related scribe objects) is undocumented and might change without notice in future Matlab releases. We were fortunate in the current case that the change was small enough that a simple workaround could be found, but this may possibly not be the case next time around. It’s impossible to guess when such features will eventually break: it might happen in the very next release, or 20 years in the future. Since there are no real alternatives, we have little choice other than to use these features, and document the risk (the fact that they use undocumented aspects). In your documentation/comments, add a link back here so that if something ever breaks you’d be able to check if I posted any fix or workaround.

For the sake of completeness, here is a listing of the accessible properties (regular and hidden) of both the mode-manager as well as the zoom uimode, in R2015b:

>> warning('off', 'MATLAB:structOnObject');  % suppress warning about using structs (yes we know it's not good for us...)
>> struct(hManager)
ans = 
                    CurrentMode: [1x1 matlab.uitools.internal.uimode]
                  DefaultUIMode: ''
                       Blocking: 0
                   FigureHandle: [1x1 Figure]
          WindowListenerHandles: [1x2 event.proplistener]
    WindowMotionListenerHandles: [1x2 event.proplistener]
                 DeleteListener: [1x1 event.listener]
            PreviousWindowState: []
                RegisteredModes: [1x1 matlab.uitools.internal.uimode]
 
>> struct(hManager.CurrentMode)
ans = 
    WindowButtonMotionFcnInterrupt: 0
                UIControlInterrupt: 0
                   ShowContextMenu: 1
                         IsOneShot: 0
                    UseContextMenu: 'on'
                          Blocking: 0
               WindowJavaListeners: []
                    DeleteListener: [1x1 event.listener]
              FigureDeleteListener: [1x1 event.listener]
                    BusyActivating: 0
                              Name: 'Exploration.Zoom'
                      FigureHandle: [1x1 Figure]
             WindowListenerHandles: [1x2 event.proplistener]
                   RegisteredModes: []
                       CurrentMode: []
               ModeListenerHandles: []
               WindowButtonDownFcn: {@localWindowButtonDownFcn  [1x1 matlab.uitools.internal.uimode]}
                 WindowButtonUpFcn: []
             WindowButtonMotionFcn: {@localMotionFcn  [1x1 matlab.uitools.internal.uimode]}
                 WindowKeyPressFcn: []
               WindowKeyReleaseFcn: []
              WindowScrollWheelFcn: {@localButtonWheelFcn  [1x1 matlab.uitools.internal.uimode]}
                     KeyReleaseFcn: []
                       KeyPressFcn: {@localKeyPressFcn  [1x1 matlab.uitools.internal.uimode]}
                      ModeStartFcn: {@localStartZoom  [1x1 matlab.uitools.internal.uimode]}
                       ModeStopFcn: {@localStopZoom  [1x1 matlab.uitools.internal.uimode]}
                  ButtonDownFilter: []
                     UIContextMenu: []
                     ModeStateData: [1x1 struct]
                WindowFocusLostFcn: []
          UIControlSuspendListener: [1x1 event.listener]
      UIControlSuspendJavaListener: [1x1 handle.listener]
                   UserButtonUpFcn: []
               PreviousWindowState: []
                 ActionPreCallback: []
                ActionPostCallback: []
           WindowMotionFcnListener: [1x1 event.listener]
                       FigureState: [1x1 struct]
                        ParentMode: []
                     DefaultUIMode: ''
             CanvasContainerHandle: []
                            Enable: 'on'
]]>
https://undocumentedmatlab.com/blog/enabling-user-callbacks-during-zoom-pan/feed 9
Hyperlink text labelshttps://undocumentedmatlab.com/blog/hyperlink-text-labels https://undocumentedmatlab.com/blog/hyperlink-text-labels#comments Wed, 21 Oct 2015 14:42:48 +0000 http://undocumentedmatlab.com/?p=6033
 
Related posts:
  1. Customizing Matlab labels Matlab's text uicontrol is not very customizable, and does not support HTML or Tex formatting. This article shows how to display HTML labels in Matlab and some undocumented customizations...
  2. GUI integrated HTML panel Simple HTML can be presented in a Java component integrated in Matlab GUI, without requiring the heavy browser control....
  3. Matlab-Java interface using a static control The switchyard function design pattern can be very useful when setting Matlab callbacks to Java GUI controls. This article explains why and how....
  4. ScreenCapture utility The ScreenCapture utility uses purely-documented Matlab for capturing a screen region as an image from within Matlab. ...
 
]]>
It is often useful to include hyperlinked text labels in GUIs. Such labels provide single-click access to important functionality, improve branding, and are non-intrusive action controls having a lower visual impact than a full-blown button. There are several ways that we can display such hyperlinks in Matlab GUIs, and today I will show one of my favorites.

The basic idea is to create a Java text label control whose label displays an HTML link. The control is modified to change the mouse cursor when it hovers over the hyperlink, and a mouse-click callback is set to open the hyperlink target in the system browser:

hyperlink text label

hyperlink text label

% Create and display the text label
url = 'UndocumentedMatlab.com';
labelStr = ['<html>More info: <a href="">' url '</a></html>'];
jLabel = javaObjectEDT('javax.swing.JLabel', labelStr);
[hjLabel,hContainer] = javacomponent(jLabel, [10,10,250,20], gcf);
 
% Modify the mouse cursor when hovering on the label
hjLabel.setCursor(java.awt.Cursor.getPredefinedCursor(java.awt.Cursor.HAND_CURSOR));
 
% Set the label's tooltip
hjLabel.setToolTipText(['Visit the ' url ' website']);
 
% Set the mouse-click callback
set(hjLabel, 'MouseClickedCallback', @(h,e)web(['http://' url], '-browser'))

Note the visual illusion here: we do not directly click the hyperlink (note that its href is empty), but rather the label control. The end-result is the same.

Also note that this technique could be used to easily display clickable icons/images, including animated and transparent GIFs, by simply setting the label’s HTML string to display the relevant image. I have already shown how to do this in another post. Uses could range from clickable logo images to clickable help icons.

We could also use a flat (borderless) button. I have already posted related articles about button customizations in Matlab GUIs (here, here and here). In fact, I have already shown how we can use borderless buttons in Matlab axes to display a non-obtrusive control for controlling plot properties. The code would be very similar to the above, except that we would use a JButton rather than a JLabel, and also need to setBorder([]) and similar button-specific modifications. Buttons have a bit more functionality and customizability than simple text labels; the appearance would be the same, but the extra customizability may be handy in very special cases, although not for most use-cases.

One of the benefits of using either JLabels or JButtons is that they enable multiple callbacks (e.g., FocusGained,FocusLost) and properties (e.g., VerticalAlignment,ToolTipText) that the standard Matlab text uicontrol does not provide (not to mention their builtin ability to display HTML, which Matlab’s uicontrol text labels do not posses).

]]>
https://undocumentedmatlab.com/blog/hyperlink-text-labels/feed 5
Adding dynamic properties to graphic handleshttps://undocumentedmatlab.com/blog/adding-dynamic-properties-to-graphic-handles https://undocumentedmatlab.com/blog/adding-dynamic-properties-to-graphic-handles#comments Wed, 16 Sep 2015 17:26:44 +0000 http://undocumentedmatlab.com/?p=6006
 
Related posts:
  1. New information on HG2 More information on Matlab's new HG2 object-oriented handle-graphics system...
  2. Performance: accessing handle properties Handle object property access (get/set) performance can be significantly improved using dot-notation. ...
  3. uiundo – Matlab’s undocumented undo/redo manager The built-in uiundo function provides easy yet undocumented access to Matlab's powerful undo/redo functionality. This article explains its usage....
  4. Matlab’s HG2 mechanism HG2 is presumably the next generation of Matlab graphics. This article tries to explore its features....
 
]]>
A client recently asked me to extend one of Matlab’s built-in graphic containers (uiflowcontainer in this specific case) with automatic scrollbars that would enable the container to act as a scroll-panel. The basic idea would be to dynamically monitor the container’s contents and when it is determined that they overflow the container’s boundaries, then attach horizontal/vertical scrollbars to enable scrolling the contents into view:

Scrollable Matlab container

Scrollable Matlab container

This may sound simple, but there are actually quite a few undocumented hacks that make this possible, including listening to ObjectChildAdded/ObjectChildRemoved events, location/size/visibility events, layout changes etc. Maybe I’ll blog about it in some future article.

Today’s post is focused on a specific aspect of this project, attaching dynamic properties to the builtin uiflowcontainer, that would enable users to modify the container’s properties directly, as well as control aspects of the scrolling using the new properties: handles to the parent container, as well as the horizontal and vertical scrollbars, and even a new refresh() method.

The “textbook” approach to this would naturally be to create a new class that extends (inherits) uiflowcontainer and includes these new properties and methods. Unfortunately, for some reason that escapes my understanding, MathWorks saw fit to make all of its end-use graphic object classes Sealed, such that they cannot be extended by users. I did ask for this to be changed long ago, but the powers that be apparently decided that it’s better this way.

So the fallback would be to create our own dedicated class having all the new properties as well as those of the original container, and ensure that all the property values are synchronized in both directions. This is probably achievable, if you have a spare few days and a masochistic state of mind. Being the lazy bum and authority-rebel that I am, I decided to take an alternate approach that would simply add my new properties to the built-in container handle. The secret lies in the undocumented function schema.prop (for HG1, R2014a and older) and the fully-documented addprop function (for HG2, R2014b and newer).

In the examples below I use a panel, but this mechanism works equally well on any Matlab HG object: axes, lines, uicontrols, figures, etc.

HG2 – addprop function

The addprop function is actually a public method of the dynamicprops class. Both the dynamicprops class as well as its addprop function are fully documented. What is NOT documented, as far as I could tell, is that all of Matlab’s builtin handle graphics objects indirectly inherit dynamicprops, via matlab.graphics.Graphics, which is a high-level superclass for all HG objects. The bottom line is that we can dynamically add run-time properties to any HG object, without affecting any other object. In other words, the new properties will only be added to the handles that we specifically request, and not to any others. This suits me just fine:

hProp = addprop(hPanel, 'hHorizontalScrollBar');
hPanel.hHorizontalScrollBar = hMyScrollbar;
hProp.SetAccess = 'private';  % make this property read-only

The new property hHorizontalScrollBar is now added to the hPanel handle, and can be accessed just like any other read-only property. For example:

>> get(hPanel, 'hHorizontalScrollBar')
ans = 
    JavaWrapper

>> hPanel.hHorizontalScrollBar
ans = 
    JavaWrapper

>> hPanel.hHorizontalScrollBar = 123
You cannot set the read-only property 'hHorizontalScrollBar' of UIFlowContainer.

Adding new methods is more tricky, since we do not have a corresponding addmethod function. The trick I used was to create a new property having the requested new method’s name, and set its read-only value to a handle of the requested function. For example:

hProp = addprop(hPanel, 'refresh');
hPanel.refresh = @myRefreshFunc;
hProp.SetAccess = 'private';  % make this property read-only

We can then invoke the new refresh “method” using the familiar dot-notation:

hPanel.refresh();

Note: if you ever need to modify the initial value in your code, you should revert the property’s SetAccess meta-property to 'public' before Matlab will enable you to modify the value:

try
    % This will raise an exception if the property already exists
    hProp = addprop(hPanel, propName);
catch
    % Property already exists - find it and set its access to public
    hProp = findprop(hPanel, propName);
    hProp.SetAccess = 'public';
end
hPanel.(propName) = newValue;

HG1 – schema.prop function

In HG1 (R2014a and earlier), we can use the undocumented schema.prop function to add a new property to any HG handle (which is a numeric value in HG1). Donn Shull wrote about schema.prop back in 2011, as part of his series of articles on UDD (Unified Data Dictionary, MCOS’s precursor). In fact, schema.prop is so useful that it has its own blog tag here and appears in no less than 15 separate articles (excluding today). With HG2’s debut 2 years ago, MathWorks tried very hard to rid the Matlab code corpus of all the legacy schema-based, replacing most major functionalities with MCOS-based HG2 code. But so far it has proven impossible to get rid of schema completely, and so schema code is still used extensively in Matlab to this day (R2015b). Search your Matlab path for “schema.prop” and see for yourself.

Anyway, the basic syntax is this:

hProp = schema.prop(hPanel, propName, 'mxArray');

The 'mxArray' specifies that the new property can accept any data type. We can limit the property to only accept certain types of data by specifying a less-generic data type, among those recognized by UDD (details).

Note that the meta-properties of the returned hProp are somewhat different from those of HG2’s hProp. Taking this into account, here is a unified function that adds/updates a new property (with optional initial value) to any HG1/HG2 object:

function addProp(hObject, propName, initialValue, isReadOnly)
    try
        hProp = addprop(hObject, propName);  % HG2
    catch
        try
            hProp = schema.prop(hObject, propName, 'mxArray');  % HG1
        catch
            hProp = findprop(hObject, propName);
        end
    end
    if nargin > 2
        try
            hProp.SetAccess = 'public';  % HG2
        catch
            hProp.AccessFlags.PublicSet = 'on';  % HG1
        end
        hObject.(propName) = initialValue;
    end
    if nargin > 3 && isReadOnly
        try
            % Set the property as read-only
            hProp.SetAccess = 'private';  % HG2
        catch
            hProp.AccessFlags.PublicSet = 'off';  % HG1
        end
    end
end
]]>
https://undocumentedmatlab.com/blog/adding-dynamic-properties-to-graphic-handles/feed 4
General-use object copyhttps://undocumentedmatlab.com/blog/general-use-object-copy https://undocumentedmatlab.com/blog/general-use-object-copy#comments Wed, 06 May 2015 13:30:23 +0000 http://undocumentedmatlab.com/?p=5782
 
Related posts:
  1. Types of undocumented Matlab aspects This article lists the different types of undocumented/unsupported/hidden aspects in Matlab...
  2. uiundo – Matlab’s undocumented undo/redo manager The built-in uiundo function provides easy yet undocumented access to Matlab's powerful undo/redo functionality. This article explains its usage....
  3. Matlab’s HG2 mechanism HG2 is presumably the next generation of Matlab graphics. This article tries to explore its features....
  4. Introduction to UDD UDD classes underlie many of Matlab's handle-graphics objects and functionality. This article introduces these classes....
 
]]>
When using Matlab objects, either a Matlab class (MCOS) or any other (e.g., Java, COM, C# etc.), it is often useful to create a copy of the original object, complete with all internal property values. This enables modification of the new copy without affecting the original object. This is not important for MCOS value-class objects, since value objects use the COW (Copy-on-Write/Update, a.k.a. Lazy Copy) and this is handled automatically by the Matlab interpreter when it detects that a change is made to the copy reference. However, it is very important for handle objects, where modifying any property of the copied object also modifies the original object.

Most OOP languages include some sort of a copy constructor, which enables programmers to duplicate a handle/reference object, internal properties included, such that it becomes entirely separate from the original object. Unfortunately, Matlab did not include such a copy constructor until R2011a (matlab.mixin.Copyable.copy()).

On Matlab R2010b and older, as well as on newer releases, we do not have a readily-available solution for handle object copy. Until now, that is.

There are several ways by which we can create such a copy function. We might call the main constructor to create a default object and then override its properties by iterating over the original object’s properties. This might work in some cases, but not if there is no default constructor for the object, or if there are side-effects to object property modifications. If we wanted to implement a deep (rather than shallow) copy, we’d need to recursively iterate over all the properties of the internal objects as well.

A simpler solution might be to save the object to a temporary file (tempname, then load from that file (which creates a copy), and finally delete the temp file. This is nice and clean, but the extra I/O could be relatively slow compared to in-memory processing.

Which leads us to today’s chosen solution, where we use Matlab’s builtin functions getByteStreamFromArray and getArrayFromByteStream, which I discussed last year as a way to easily serialize and deserialize Matlab data of any type. Specifically, getArrayFromByteStream has the side-effect of creating a duplicate of the serialized data, which is perfect for our needs here (note that these pair of function are only available on R2010b or newer; on R2010a or older we can still serialize via a temp file):

% Copy function - replacement for matlab.mixin.Copyable.copy() to create object copies
function newObj = copy(obj)
    try
        % R2010b or newer - directly in memory (faster)
        objByteArray = getByteStreamFromArray(obj);
        newObj = getArrayFromByteStream(objByteArray);
    catch
        % R2010a or earlier - serialize via temp file (slower)
        fname = [tempname '.mat'];
        save(fname, 'obj');
        newObj = load(fname);
        newObj = newObj.obj;
        delete(fname);
    end
end

This function can be placed anywhere on the Matlab path and will work on all recent Matlab releases (including R2010b and older), any type of Matlab data (including value or handle objects, UDD objects, structs, arrays etc.), as well as external objects (Java, C#, COM). In short, it works on anything that can be assigned to a Matlab variable:

obj1 = ... % anything really!
 
obj2 = obj1.copy();  % alternative #1
obj2 = copy(obj1);   % alternative #2

Alternative #1 may look “nicer” to a computer scientist, but alternative #2 is preferable because it also handles the case of non-object data (e.g., [] or ‘abc’ or magic(5) or a struct or cell array), whereas alternative #1 would error in such cases.

In any case, using either alternatives, we no longer need to worry about inheriting our MCOS class from matlab.mixin.Copyable, or backward compatibility with R2010b and older (I may possibly be bashed for this statement, but in my eyes future compatibility is less important than backward compatibility). This is not such a wild edge-case. In fact, I came across the idea for this post last week, when I developed an MCOS project for a consulting client that uses both R2010a and R2012a, and the same code needed to run on both Matlab releases.

Using the serialization functions also solves the case of creating copies for Java/C#/COM objects, which currently have no other solution, except if these objects happen to contain their own copy method.

In summary, using Matlab’s undocumented builtin serialization functions enables easy implementation of a very efficient (in-memory) copy constructor, which is expected to work across all Matlab types and many releases, without requiring any changes to existing code – just placing the above copy function on the Matlab path. This is expected to continue working properly until Matlab decides to remove the serialization functions (which should hopefully never happen, as they are so useful).

Sometimes, the best solutions lie not in sophisticated new features (e.g., matlab.mixin.Copyable), but by using plain ol’ existing building blocks. There’s a good lesson to be learned here I think.

p.s. – I do realize that matlab.mixin.Copyable provides the nice feature of enabling users to control the copy process, including implementing deep or shallow or selective copy. If that’s your need and you have R2011a or newer then good for you, go ahead and inherit Copyable. Today’s post was meant for the regular Joe who doesn’t need this fancy feature, but does need to support R2010b, and/or a simple way to clone Java/C#/COM objects.

]]>
https://undocumentedmatlab.com/blog/general-use-object-copy/feed 15
savezip utilityhttps://undocumentedmatlab.com/blog/savezip-utility https://undocumentedmatlab.com/blog/savezip-utility#comments Thu, 04 Sep 2014 18:14:12 +0000 http://undocumentedmatlab.com/?p=4982
 
Related posts:
  1. Matlab callbacks for Java events Events raised in Java code can be caught and handled in Matlab callback functions - this article explains how...
  2. UDD and Java UDD provides built-in convenience methods to facilitate the integration of Matlab UDD objects with Java code - this article explains how...
  3. Types of undocumented Matlab aspects This article lists the different types of undocumented/unsupported/hidden aspects in Matlab...
  4. Profiling Matlab memory usage mtic and mtoc were a couple of undocumented features that enabled users of past Matlab releases to easily profile memory usage. ...
 
]]>
A few months ago I wrote about Matlab’s undocumented serialization/deserialization functions, getByteStreamFromArray and getArrayFromByteStream. This could be very useful for both sending Matlab data across a network (thus avoiding the need to use a shared file), as well as for much faster data-save using the -V6 MAT format (save -v6 …).

As a followup to that article, in some cases it might be useful to use ZIP/GZIP compression, rather than Matlab’s proprietary MAT format or an uncompressed byte-stream.

Unfortunately, Matlab’s compression functions zip, gzip and tar do not really help run-time performance, but rather hurt it. The reason is that we would be paying the I/O costs three times: first to write the original (uncompressed) file, then to have zip or its counterparts read it, and finally to save the compressed file. tar is worst in this respect, since it does both a GZIP compression and a simple tar concatenation to get a standard tar.gz file. Using zip/gzip/tar only makes sense if we need to pass the data file to some external program on some remote server, whereby compressing the file could save transfer time. But as far as our Matlab program’s performance is concerned, these functions bring little value.

In contrast to file-system compression, which is what zip/gzip/tar do, on-the-fly (memory) compression makes more sense and can indeed help performance. In this case, we are compressing the data in memory, and directly saving to file the resulting (compressed) binary data. The following example compresses int8 data, such as the output of our getByteStreamFromArray serialization:

% Serialize the data into a 1D array of uint8 bytes
dataInBytes = getByteStreamFromArray(data);
 
% Parse the requested output filename (full path name)
[fpath,fname,fext] = fileparts(filepath);
 
% Compress in memory and save to the requested file in ZIP format
fos = java.io.FileOutputStream(filepath);
%fos = java.io.BufferedOutputStream(fos, 8*1024);  % not really important (*ZipOutputStream are already buffered), but doesn't hurt
if strcmpi(fext,'.gz')
    % Gzip variant:
    zos = java.util.zip.GZIPOutputStream(fos);  % note the capitalization
else
    % Zip variant:
    zos = java.util.zip.ZipOutputStream(fos);  % or: org.apache.tools.zip.ZipOutputStream as used by Matlab's zip.m
    ze  = java.util.zip.ZipEntry('data.dat');  % or: org.apache.tools.zip.ZipEntry as used by Matlab's zip.m
    ze.setSize(originalDataSizeInBytes);
    zos.setLevel(9);  % set the compression level (0=none, 9=max)
    zos.putNextEntry(ze);
end
zos.write(dataInBytes, 0, numel(dataInBytes));
zos.finish;
zos.close;

This will directly create a zip archive file in the current folder. The archive will contain a single entry (data.dat) that contains our original data. Note that data.dat is entirely virtual: it was never actually created, saving us its associated I/O costs. In fact we could have called it simply data, or whatever other valid file name.

Saving to a gzip file is even simpler, since GZIP files have single file entries. There is no use for a ZipEntry as in zip archives that may contain multiple file entries.

Note that while the resulting ZIP/GZIP file is often smaller in size than the corresponding MAT file generated by Matlab’s save, it is not necessarily faster. In fact, except on slow disks or network drives, save may well outperform this mechanism. However, in some cases, the reduced file size may save enough I/O to offset the extra processing time. Moreover, GZIP is typically much faster than either ZIP or Matlab’s save.

Loading data from ZIP/GZIP

Similar logic applies to reading compressed data: We could indeed use unzip/gunzip/untar, but these would increase the I/O costs by reading the compressed file, saving the uncompressed version, and then reading that uncompressed file into Matlab.

A better solution would be to read the compressed file directly into Matlab. Unfortunately, the corresponding input-stream classes do not have a read() method that returns a byte array. We therefore use a small hack to copy the input stream into a ByteArrayOutputStream, using Matlab’s own stream-copier class that is used within all of Matlab’s compression and decompression functions:

% Parse the requested output filename (full path name)
[fpath,fname,fext] = fileparts(filepath);
 
% Get the serialized data
streamCopier = com.mathworks.mlwidgets.io.InterruptibleStreamCopier.getInterruptibleStreamCopier;
baos = java.io.ByteArrayOutputStream;
fis  = java.io.FileInputStream(filepath);
if strcmpi(fext,'.gz')
    % Gzip variant:
    zis  = java.util.zip.GZIPInputStream(fis);
else
    % Zip variant:
    zis  = java.util.zip.ZipInputStream(fis);
    % Note: although the ze & fileName variables are unused in the Matlab
    % ^^^^  code below, they are essential in order to read the ZIP!
    ze = zis.getNextEntry;
    fileName = char(ze.getName);  %#ok<NASGU> => 'data.dat' (virtual data file)
end
streamCopier.copyStream(zis,baos);
fis.close;
data = baos.toByteArray;  % array of Matlab int8
 
% Deserialize the data back into the original Matlab data format
% Note: the zipped data is int8 => need to convert into uint8:
% Note2: see discussion with Martin in the comments section below
if numel(data) < 1e5
    data = uint8(mod(int16(data),256))';
else
    data = typecast(data, 'uint8');
end
 
data = getArrayFromByteStream(data);

Note that when we deserialize, we have to convert the unzipped int8 byte-stream into a uint8 byte-stream that getArrayFromByteStream can process (we don’t need to do this during serialization).

The SAVEZIP utility

I have uploaded a utility called SAVEZIP to the Matlab File Exchange which includes the savezip and loadzip functions. These include the code above, plus some extra sanity checks and data processing. Usage is quite simple:

savezip('myData', magic(4))   %save data to myData.zip in current folder
savezip('myData', 'myVar')    %save myVar to myData.zip in current folder
savezip('myData.gz', 'myVar') %save data to myData.gz in current folder
savezip('data\myData', magic(4))    %save data to .\data\myData.zip
savezip('data\myData.gz', magic(4)) %save data to .\data\myData.gz
 
myData = loadzip('myData');
myData = loadzip('myData.zip');
myData = loadzip('data\myData');
myData = loadzip('data\myData.gz');

Jan Berling has written another variant of the idea of using getByteStreamFromArray and getArrayFromByteStream for saving/loading data from disk, in this case in an uncompressed manner. He put it all in his Bytestream Save Toolbox on the File Exchange.

Transmitting compressed data via the network

If instead of saving to a file we wish to transmit the compressed data to a remote process (or to save it ourselves later), we can simply wrap our ZipOutputStream with a ByteArrayOutputStream rather than a FileOutputStream. For example, on the way out:

baos = java.io.ByteArrayOutputStream;
if isGzipVarant
    zos = java.util.zip.GZIPOutputStream(baos);
else  % Zip variant
    zos = java.util.zip.ZipOutputStream(baos);
    ze  = java.util.zip.ZipEntry('data.dat');
    ze.setSize(numel(data));
    zos.setLevel(9);
    zos.putNextEntry(ze);
end
dataInBytes = int8(data);  % or: getByteStreamFromArray(data)
zos.write(dataInBytes,0,numel(dataInBytes));
zos.finish;
zos.close;
compressedDataArray = baos.toByteArray;  % array of Matlab int8

I leave it as an exercise to the reader to make the corresponding changes for the receiving end.

New introductory Matlab book

Matlab has a plethora of introductory books. But I have a special affection to one that was released only a few days ago: MATLAB Succinctly by Dmitri Nesteruk, for which I was a technical editor/reviewer. It’s a very readable and easy-to-follow book, and it’s totally free, so go ahead and download.

MATLAB Succinctly book by Dmitri Nesteruk This title adds to the large (and growing) set of free ~100-page introductory titles by Syncfusion, on a wide variety of programming languages and technologies. Go ahead and download these books as well. While you’re at it, take a look at Syncfusion’s set of professional components and spread the word. If Syncfusion gets enough income from such incoming traffic, they may continue to support their commendable project of similar free IT-related titles.

This may be a good place to update that my own [second] book, Accelerating MATLAB Performance, is nearing completing, and is currently in advanced copy-editing stage. It turned out that there was a lot more to Matlab performance than I initially realized. The book size (and writing time) doubled, and it turned out to be a hefty ~750 pages packed full with performance improvement tips. I’m still considering the cover image (ideas anyone?) and working on the index, but the end is now in sight. The book should be in your favorite bookstore this December. If you can’t wait until then, and/or if you’d rather use the real McCoy, consider inviting me for a consulting session…

Addendum December 16, 2014: I am pleased to announce that my book, Accelerating MATLAB Performance, is now published (additional information).

]]>
https://undocumentedmatlab.com/blog/savezip-utility/feed 18
Undocumented feature listhttps://undocumentedmatlab.com/blog/undocumented-feature-list https://undocumentedmatlab.com/blog/undocumented-feature-list#comments Wed, 19 Mar 2014 18:00:30 +0000 http://undocumentedmatlab.com/?p=4724
 
Related posts:
  1. Serializing/deserializing Matlab data Matlab does not provide a documented manner to serialize data into a byte stream, but we can do this with some undocumented functionality. ...
  2. uiundo – Matlab’s undocumented undo/redo manager The built-in uiundo function provides easy yet undocumented access to Matlab's powerful undo/redo functionality. This article explains its usage....
  3. Matlab’s internal memory representation Matlab's internal memory structure is explored and discussed. ...
  4. Pinning annotations to graphs Annotation object can be programmatically set at, and pinned-to, plot axes data points. ...
 
]]>
Three years ago I posted an article on Matlab’s undocumented feature function. feature is a Matlab function that enables access to undocumented internal Matlab functionality. Most of this functionality is not very useful, but in some cases it could indeed be very interesting. As sometimes happens, this innocent-enough article generated a lot of interest, both online and offline. Perhaps the reason was that in this article I listed the known list of supported features with a short explanation and references were available. At the time, this was the only comprehensive such listing, which I manually collected from numerous sources. For this reason I was delighted to receive Yves Piguet’s tip about the availability of a programmatic interface for a full listing of features:

>> list = feature('list')   % 260 features in R2013b
list = 
1x260 struct array with fields:
    name
    value
    has_callback
    has_builtin
    call_count

Which can be listed as follows:

for i = 1 : length(list)
   fprintf('%35s has_cb=%d has_bi=%d calls=%d val=%g\n', ...
      list(i).name, list(i).has_callback, list(i).has_builtin, list(i).call_count, list(i).value);
end
 
                    100 has_cb=0 has_bi=1 calls=0 val=1
                    102 has_cb=0 has_bi=1 calls=0 val=1
                     12 has_cb=0 has_bi=1 calls=0 val=1
                     14 has_cb=0 has_bi=1 calls=0 val=1
                     25 has_cb=0 has_bi=1 calls=0 val=1
                    300 has_cb=0 has_bi=0 calls=0 val=1
                    301 has_cb=0 has_bi=0 calls=0 val=1
                     44 has_cb=0 has_bi=1 calls=0 val=1
                     45 has_cb=0 has_bi=1 calls=0 val=1
                      7 has_cb=0 has_bi=0 calls=0 val=1
                      8 has_cb=0 has_bi=1 calls=0 val=1
                      9 has_cb=0 has_bi=0 calls=0 val=1
                  accel has_cb=0 has_bi=1 calls=0 val=0
         AccelBlockSize has_cb=0 has_bi=1 calls=0 val=0
          AccelMaxTemps has_cb=0 has_bi=1 calls=0 val=0
    AccelThreadBlockMin has_cb=0 has_bi=1 calls=0 val=0
              allCycles has_cb=0 has_bi=1 calls=0 val=0
 AllWarningsCanBeErrors has_cb=1 has_bi=0 calls=0 val=0
           ArrayEditing has_cb=0 has_bi=0 calls=0 val=1
       AutomationServer has_cb=0 has_bi=1 calls=0 val=0
              CachePath has_cb=0 has_bi=0 calls=0 val=1
     CaptureScreenCount has_cb=0 has_bi=0 calls=0 val=0
       CheckMallocClear has_cb=0 has_bi=0 calls=0 val=1
                    ... (etc. etc.)

Unfortunately, in the latest Matlab R2014a, which was released last week, this nice feature has been removed:

>> list = feature('list')
Error using feature
Feature list not found

Luckily, the list can still be retrieved programmatically, using an undocumented MEX library function. Place the following in a file called feature_list.cpp:

#include "mex.h"
void svListFeatures(int, mxArray_tag** const, int, mxArray_tag** const);
 
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
    svListFeatures(1,plhs,0,NULL);
}

Now compile this MEX file:

if isunix
   mex('feature_list.cpp',['-L',matlabroot,'/bin/',computer('arch')],'-lmwservices');
else
   mex('feature_list.cpp','-llibmwservices');
end

As you can see, all this MEX function does is just call the svListFeatures() function from the libmwservices dynamic/shared library.

We can now run this MEX function directly in Matlab:

>> list = feature_list
list = 
1x273 struct array with fields:
    name
    value
    has_callback
    has_builtin
    call_count

Running both the new feature_list and the previous feature(‘list’) on previous Matlab releases produces exactly the same result, showing that under the hood, feature(‘list’) was basically just calling libmwservices‘s svListFeatures() function.

There are 273 undocumented features in R2014a: 20 were added and 7 were removed compared to R2013b. For those interested, the modified features in the past two releases are:

  • R2013b:
    1. ConvertAllDoublesToHandles (added)
    2. DrawnowNowaitFeature (added)
    3. getsimd (added)
    4. CoreDump (removed)
  • R2014a:
    1. GPUAllowPartialSharedDataCopies (added)
    2. GPUAllowSharedDataCopies (added)
    3. GetOpenGLData (added)
    4. GetOpenGLInfo (added)
    5. HGUpdateErrorChecking (added)
    6. JVMShutdown (added)
    7. LoadHG1FigFileWithHG1Defaults (added)
    8. OpenGLBitmapZbufferBug (added)
    9. OpenGLClippedImageBug (added)
    10. OpenGLDockingBug (added)
    11. OpenGLEraseModeBug (added)
    12. OpenGLLineSmoothingBug (added)
    13. OpenGLPolygonOffsetBiasBug (added)
    14. OpenGLVerbose (added)
    15. OpenGLWobbleTesselatorBug (added)
    16. SaveFigFileWithHG1Defaults (added)
    17. ShutdownReportLevel (added)
    18. UseGenericOpenGL (added)
    19. crash_mode (added)
    20. isLightweightEval (added)
    21. EnableBangErrors (removed)
    22. EnableJavaAsGObject (removed)
    23. List (removed) – this is actually the reason for today’s article!
    24. hwtic (removed)
    25. hwtoc (removed)
    26. oldtictoc (removed)
    27. timing (removed)

Happy digging!

p.s. – Packt publishing, which publishes IT books, has a great deal on ebooks until March 26 – buy 1 get 1 free. Some of their books are Matlab-related. Check it out!


Addendum July 18, 2016: Since R2015a, feature(‘list’) returns an empty array. A reader of this blog, Mikhail P, reported the following workaround, by temporarily setting the MWE_INSTALL environment variable (that should then be reset, since it is used in other locations too):

% The following was run in R2016b
>> setenv('MWE_INSTALL','1'); list = feature('list'), setenv('MWE_INSTALL');
list = 
  1×266 struct array with fields:
    name
    value
    has_callback
    has_builtin
    call_count
]]>
https://undocumentedmatlab.com/blog/undocumented-feature-list/feed 4
Serializing/deserializing Matlab datahttps://undocumentedmatlab.com/blog/serializing-deserializing-matlab-data https://undocumentedmatlab.com/blog/serializing-deserializing-matlab-data#comments Wed, 22 Jan 2014 19:57:49 +0000 http://undocumentedmatlab.com/?p=4583
 
Related posts:
  1. Matlab’s HG2 mechanism HG2 is presumably the next generation of Matlab graphics. This article tries to explore its features....
  2. New information on HG2 More information on Matlab's new HG2 object-oriented handle-graphics system...
  3. Undocumented cursorbar object Matlab's internal undocumented graphics.cursorbar object can be used to present dynamic data-tip cross-hairs...
  4. Undocumented Matlab MEX API Matlab's MEX API contains numerous undocumented functions, that can be extremely useful. ...
 
]]>
Last year I wrote an article on improving the performance of the save function. The article discussed various ways by which we can store Matlab data on disk. However, in many cases we are interested in a byte-stream serialization, in order to transmit information to external processes.

The request to get a serialized byte-stream of Matlab data has been around for many years (example), but MathWorks has never released a documented way of serializing and unserializing data, except by storing onto a disk file and later loading it from file. Naturally, using a disk file significantly degrades performance. We could always use a RAM-disk or flash memory for improved performance, but in any case this seems like a major overkill to such a simple requirement.

In last year’s article, I presented a File Exchange utility for such generic serialization/deserialization. However, that utility is limited in the types of data that it supports, and while it is relatively fast, there is a much better, more generic and faster solution.

The solution appears to use the undocumented built-in functions getByteStreamFromArray and getArrayFromByteStream, which are apparently used internally by the save and load functions. The usage is very simple:

byteStream = getByteStreamFromArray(anyData);  % 1xN uint8 array
anyData = getArrayFromByteStream(byteStream);

Many Matlab functions, documented and undocumented alike, are defined in XML files within the %matlabroot%/bin/registry/ folder; our specific functions can be found in %matlabroot%/bin/registry/hgbuiltins.xml. While other functions include information about their location and number of input/output args, these functions do not. Their only XML attribute is type = ":all:", which seems to indicate that they accept all data types as input. Despite the fact that the functions are defined in hgbuiltins.xml, they are not limited to HG objects – we can serialize basically any Matlab data: structs, class objects, numeric/cell arrays, sparse data, Java handles, timers, etc. For example:

% Simple Matlab data
>> byteStream = getByteStreamFromArray(pi)  % 1x72 uint8 array
byteStream =
  Columns 1 through 19
    0    1   73   77    0    0    0    0   14    0    0    0   56    0    0    0    6    0    0
  Columns 20 through 38
    0    8    0    0    0    6    0    0    0    0    0    0    0    5    0    0    0    8    0
  Columns 39 through 57
    0    0    1    0    0    0    1    0    0    0    1    0    0    0    0    0    0    0    9
  Columns 58 through 72
    0    0    0    8    0    0    0   24   45   68   84  251   33    9   64
 
>> getArrayFromByteStream(byteStream)
ans =
          3.14159265358979
 
% A cell array of several data types
>> byteStream = getByteStreamFromArray({pi, 'abc', struct('a',5)});  % 1x312 uint8 array
>> getArrayFromByteStream(byteStream)
ans = 
    [3.14159265358979]    'abc'    [1x1 struct]
 
% A Java object
>> byteStream = getByteStreamFromArray(java.awt.Color.red);  % 1x408 uint8 array
>> getArrayFromByteStream(byteStream)
ans =
java.awt.Color[r=255,g=0,b=0]
 
% A Matlab timer
>> byteStream = getByteStreamFromArray(timer);  % 1x2160 uint8 array
>> getArrayFromByteStream(byteStream)
 
   Timer Object: timer-2
 
   Timer Settings
      ExecutionMode: singleShot
             Period: 1
           BusyMode: drop
            Running: off
 
   Callbacks
           TimerFcn: ''
           ErrorFcn: ''
           StartFcn: ''
            StopFcn: ''
 
% A Matlab class object
>> byteStream = getByteStreamFromArray(matlab.System);  % 1x1760 uint8 array
>> getArrayFromByteStream(byteStream)
ans = 
  System: matlab.System

Serializing HG objects

Of course, we can also serialize/deserialize also HG controls, plots/axes and even entire figures. When doing so, it is important to serialize the handle of the object, rather than its numeric handle, since we are interested in serializing the graphic object, not the scalar numeric value of the handle:

% Serializing a simple figure with toolbar and menubar takes almost 0.5 MB !
>> hFig = handle(figure);  % a new default Matlab figure
>> length(getByteStreamFromArray(hFig))
ans =
      479128
 
% Removing the menubar and toolbar removes much of this amount:
>> set(hFig, 'menuBar','none', 'toolbar','none')
>> length(getByteStreamFromArray(hFig))
ans =
       11848   %!!!
 
% Plot lines are not nearly as "expensive" as the toolbar/menubar
>> x=0:.01:5; hp=plot(x,sin(x));
>> byteStream = getByteStreamFromArray(hFig);
>> length(byteStream)
ans =
       33088
 
>> delete(hFig);
>> hFig2 = getArrayFromByteStream(byteStream)
hFig2 =
	figure

The interesting thing here is that when we deserialize a byte-stream of an HG object, it is automatically rendered onscreen. This could be very useful for persistence mechanisms of GUI applications. For example, we can save the figure handles in file so that if the application crashes and relaunches, it simply loads the file and we get exactly the same GUI state, complete with graphs and what-not, just as before the crash. Although the figure was deleted in the last example, deserializing the data caused the figure to reappear.

We do not need to serialize the entire figure. Instead, we could choose to serialize only a specific plot line or axes. For example:

>> x=0:0.01:5; hp=plot(x,sin(x));
>> byteStream = getByteStreamFromArray(handle(hp));  % 1x13080 uint8 array
>> hLine = getArrayFromByteStream(byteStream)
ans =
	graph2d.lineseries

This could also be used to easily clone (copy) any figure or other HG object, by simply calling getArrayFromByteStream (note the corresponding copyobj function, which I bet uses the same underlying mechanism).

Also note that unlike HG objects, deserialized timers are NOT automatically restarted; perhaps the Running property is labeled transient or dependent. Properties defined with these attributes are apparently not serialized.

Performance aspects

Using the builtin getByteStreamFromArray and getArrayFromByteStream functions can provide significant performance speedups when caching Matlab data. In fact, it could be used to store otherwise unsupported objects using the save -v6 or savefast alternatives, which I discussed in my save performance article. Robin Ince has shown how this can be used to reduce the combined caching/uncaching run-time from 115 secs with plain-vanilla save, to just 11 secs using savefast. Robin hasn’t tested this in his post, but since the serialized data is a simple uint8 array, it is intrinsically supported by the save -v6 option, which is the fastest alternative of all:

>> byteStream = getByteStreamFromArray(hFig);
>> tic, save('test.mat','-v6','byteStream'); toc
Elapsed time is 0.001924 seconds.
 
>> load('test.mat')
>> data = load('test.mat')
data = 
    byteStream: [1x33256 uint8]
>> getArrayFromByteStream(data.byteStream)
ans =
	figure

Moreover, we can now use java.util.Hashtable to store a cache map of any Matlab data, rather than use the much slower and more limited containers.Map class provided in Matlab.

Finally, note that as built-in functions, these functions could change without prior notice on any future Matlab release.

MEX interface – mxSerialize/mxDeserialize

To complete the picture, MEX includes a couple of undocumented functions mxSerialize and mxDeserialize, which correspond to the above functions. getByteStreamFromArray and getArrayFromByteStream apparently call them internally, since they provide the same results. Back in 2007, Brad Phelan wrote a MEX wrapper that could be used directly in Matlab (mxSerialize.c, mxDeserialize.c). The C interface was very simple, and so was the usage:

#include "mex.h"
 
EXTERN_C mxArray* mxSerialize(mxArray const *);
EXTERN_C mxArray* mxDeserialize(const void *, size_t);
 
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
    if (nlhs && nrhs) {
          plhs[0] = (mxArray *) mxSerialize(prhs[0]);
        //plhs[0] = (mxArray *) mxDeserialize(mxGetData(prhs[0]), mxGetNumberOfElements(prhs[0]));
    }
}

Unfortunately, MathWorks has removed the C interface for these functions from libmx in R2014a, keeping only their C++ interfaces:

mxArray* matrix::detail::noninlined::mx_array_api::mxSerialize(mxArray const *anyData)
mxArray* matrix::detail::noninlined::mx_array_api::mxDeserialize(void const *byteStream, unsigned __int64 numberOfBytes)
mxArray* matrix::detail::noninlined::mx_array_api::mxDeserializeWithTag(void const *byteStream, unsigned __int64 numberOfBytes, char const* *tagName)

These are not the only MEX functions that were removed from libmx in R2014a. Hundreds of other C functions were also removed with them, some of them quite important (e.g., mxCreateSharedDataCopy). A few hundred new C++ functions were added in their place, but I fear that these are not accessible to MEX users without a code change (see below). libmx has always changed between Matlab releases, but not so drastically for many years. If you rely on any undocumented MEX functions in your code, now would be a good time to recheck it, before R2014a is officially released.

Thanks to Bastian Ebeling, we can still use these interfaces in our MEX code by simply renaming the MEX file from .c to .cpp and modifying the code as follows:

#include "mex.h"
 
// MX_API_VER has unfortunately not changed between R2013b and R2014a,
// so we use the new MATRIX_DLL_EXPORT_SYM as an ugly hack instead
#if defined(__cplusplus) && defined(MATRIX_DLL_EXPORT_SYM)
    #define EXTERN_C extern
    namespace matrix{ namespace detail{ namespace noninlined{ namespace mx_array_api{
#endif
 
EXTERN_C mxArray* mxSerialize(mxArray const *);
EXTERN_C mxArray* mxDeserialize(const void *, size_t);
// and so on, for any other MEX C functions that migrated to C++ in R2014a
 
#if defined(__cplusplus) && defined(MATRIX_DLL_EXPORT_SYM)
    }}}}
    using namespace matrix::detail::noninlined::mx_array_api;
#endif
 
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
    if (nlhs && nrhs) {
        plhs[0] = (mxArray *) mxSerialize(prhs[0]);
      //plhs[0] = (mxArray *) mxDeserialize(mxGetData(prhs[0]), mxGetNumberOfElements(prhs[0]));
    }
}

Unfortunately, pre-R2014a code cannot coexist with R2014a code (since libmx is different), so separate MEX files need to be used depending on the Matlab version being used. This highlights the risk of using such unsupported functions.

The roundabout alternative is of course to use mexCallMATLAB to invoke getByteStreamFromArray and getArrayFromByteStream. This is actually rather silly, but it works…

p.s. – Happy 30th anniversary, MathWorks!

Addendum March 9, 2014

Now that the official R2014a has been released, I am happy to report that most of the important MEX functions that were removed in the pre-release have been restored in the official release. These include mxCreateSharedDataCopy, mxFastZeros, mxCreateUninitDoubleMatrix, mxCreateUninitNumericArray, mxCreateUninitNumericMatrix and mxGetPropertyShared. Unfortunately, mxSerialize and mxDeserialize remain among the functions that were left out, which is a real pity considering their usefulness, but we can use one of the workarounds mentioned above. At least those functions that were critical for in-place data manipulation and improved MATLAB performance have been restored, perhaps in some part due to lobbying by yours truly and by others.

MathWorks should be commended for their meaningful dialog with users and for making the fixes in such a short turn-around before the official release, despite the fact that they belong to the undocumented netherworld. MathWorks may appear superficially to be like any other corporate monolith, but when you scratch the surface you discover that there are people there who really care about users, not just the corporate bottom line. I must say that I really like this aspect of their corporate culture.

]]>
https://undocumentedmatlab.com/blog/serializing-deserializing-matlab-data/feed 16
sprintfc – undocumented helper functionhttps://undocumentedmatlab.com/blog/sprintfc-undocumented-helper-function https://undocumentedmatlab.com/blog/sprintfc-undocumented-helper-function#comments Wed, 27 Nov 2013 20:18:04 +0000 http://undocumentedmatlab.com/?p=4419
 
Related posts:
  1. ismembc – undocumented helper function Matlab has several undocumented internal helper functions that can be useful on their own in some cases. This post presents the ismembc function....
  2. Datenum performance The performance of the built-in Matlab function datenum can be significantly improved by using an undocumented internal help function...
  3. uisplittool & uitogglesplittool Matlab's undocumented uisplittool and uitogglesplittool are powerful controls that can easily be added to Matlab toolbars - this article explains how...
  4. Creating a simple UDD class This article explains how to create and test custom UDD packages, classes and objects...
 
]]>
Every now and then I discover a useful built-in Matlab function, often by bumping into them within the m-code of standard functions. Such was the case, for example, of the ismembc function, that I described here back in 2009, and the dtstr2dtnummx function that I described in 2011. Today I describe another such function, sprintfc, based on a tip I received from Dirk Engel.

The requirement

The use-case of generating a cell-array of formatted strings is often encountered. Unfortunately, the standard sprintf function generates a single string, rather than a cell-array of strings. So, for example,

>> data = pi * (0:.5:2)
data =
         0    1.5708    3.1416    4.7124    6.2832
 
>> sprintf('%.3f  ',data)
ans =
0.000  1.571  3.142  4.712  6.283

This is nice, but not exactly what we need, which is a cell array that looks like this:

>> c  % c is a 1x5 cell array
c = 
    '0.000'    '1.571'    '3.142'    '4.712'    '6.283'

The standard solutions

There are several standard solution for getting the required cell array. A simple for-loop is one of the fastest of these, due to JIT optimizations in recent Matlab releases. Some of the other alternatives may seem very strange, but they perform faster than the simpler variants. Note that the following code should be run within a testing function, to remove extraneous influences on the timing (JIT compilation etc.):

% 1) Simple for-loop with sprintf
tic
for idx = 1 : 1000
    c = {};  %initialize
    % note: reallocations are not really a factor here, so don't bother preallocating
    for dataElement = data
        c{end+1} = sprintf('%.3f',dataElement);
    end
end
toc
% Elapsed time is 0.076002 seconds.
 
% 2) Using arrayfun + num2str
tic
for idx = 1 : 1000
    c = arrayfun(@(c) num2str(c,'%.3f'), data, 'uniform',false);
end
toc
% Elapsed time is 0.557993 seconds.
 
% 3) Using cellfun + num2str + num2cell
tic
for idx = 1 : 1000
    c = cellfun(@(c) num2str(c,'%.3f'), num2cell(data), 'uniform',false);
end
toc
% Elapsed time is 0.566924 seconds.
 
% 4) Using num2cell + num2str
tic
for idx = 1 : 1000
    c = num2cell(num2str(data','%.3f'),2)';
end
toc
% Elapsed time is 0.173026 seconds.
 
% 5) Using cellstr + num2str
tic
for idx = 1 : 1000
    c = reshape(cellstr(num2str(data(:),'%.3f')),size(data));
end
toc
% Elapsed time is 0.175769 seconds.
 
% 6) Using num2cell + num2str (after Urs Schwartz & Jiro Doke)
tic
for idx = 1 : 1000
    c = reshape(num2cell(num2str(data(:),'%.3f'),2),size(data));
end
toc
% Elapsed time is 0.170247 seconds.
 
% 7) Using strread + sprintf
tic
for idx = 1 : 1000
    c = reshape(strread(sprintf(['%.3f','$'], data),'%s', 'delimiter','$'),size(data));
end
toc
% Elapsed time is 0.059457 seconds.

There are also some File Exchange contributions that do a similar thing, including num2cellstr or mtx2charcell, which basically implement the last two variants.

sprintfc

It turns out that the standard num2str function uses an undocumented built-in function sprintfc that already does this for us in one fell swoop – this is both the simplest and the fastest solution:

% 8) fastest and simplest - sprintfc
tic
for idx = 1 : 1000
    c = sprintfc('%.3f',data);
end
toc
% Elapsed time is 0.015714 seconds.

This is a 35x speedup compared to arrayfun/cellfun, 5x speedup compared to the JIT-optimized for-loop, and 4x speedup compared to the fastest other variant. Useful indeed.

The syntax of the sprintfc function can be seen in the num2str.m function:

[cells, errorMsg, isLeft] = sprintfc(format, data, isImaginary);

where isImaginary is an optional flag (default=false) indicating that data should be formatted as imaginary values with the specified magnitude:

>> data = pi * (-1:.5:1)
data =
   -3.1416   -1.5708         0    1.5708    3.1416
 
>> sprintfc('%.3f', data, false)
ans = 
    '-3.142'    '-1.571'    '0.000'    '1.571'    '3.142'
 
>> sprintfc('%.3f', data, true)
ans = 
    '-3.142i'    '-1.571i'    '+0.000i'    '+1.571i'    '+3.142i'

The second returned argument is an errorMsg string that is returned when sprintfc fails, typically due to an invalid format:

>> [c,errMsg] = sprintfc('%r', data)
c = 
    {''}
errMsg =
Invalid format.

The third returned value, isLeft, is apparently used to indicate whether the output cell-string is left-justified; namely, if the ‘-‘ flag is used within the format (thanks Yaroslav!). For example,

>> [cells, ~, isLeft] = sprintfc('%-8.5g', sqrt(2), false)
cells = 
    '1.4142  '
isLeft =
    1
 
>> [cells, ~, isLeft] = sprintfc('%8.5g', sqrt(2), false)
cells = 
    '  1.4142'
isLeft =
    0

sprintfc has apparently remained in its current stable state since it was first introduced in the last decade (sometime between Matlab release 7.1 [in 2005] and R2008a). I do not know why MathWorks chose to keep sprintfc as an internal undocumented and unsupported function, despite its obvious usefulness.

If anyone finds any other similar useful built-ins, please do let me know.

London training course – Feb/Mar 2014

I am planning a week of advanced Matlab training workshops/seminars in London in Feb/Mar 2014. I plan two separate courses:

This is a unique opportunity to enhance your Matlab skills in a few days, at an affordable cost, by an established expert. This training is not provided anywhere else. If you have a specific topic that you would like me to discuss, then I would be happy to do it. In other words, I can customize the training to your specific needs, within the provided framework. More details on my training can be found here.

If you are interested in the London courses or a private dedicated course, please Email me (altmany at gmail dot com) for details.

Happy Hanukkah/Thanksgiving everyone :-)

]]>
https://undocumentedmatlab.com/blog/sprintfc-undocumented-helper-function/feed 5
Performance: accessing handle propertieshttps://undocumentedmatlab.com/blog/performance-accessing-handle-properties https://undocumentedmatlab.com/blog/performance-accessing-handle-properties#comments Wed, 20 Nov 2013 18:00:45 +0000 http://undocumentedmatlab.com/?p=4403
 
Related posts:
  1. Adding dynamic properties to graphic handles It is easy and very useful to attach dynamic properties to Matlab graphics objects in run-time. ...
  2. HG2 update HG2 appears to be nearing release. It is now a stable mature system. ...
  3. Customizing axes part 2 Matlab HG2 axes can be customized in many different ways. This article explains some of the undocumented aspects. ...
  4. New information on HG2 More information on Matlab's new HG2 object-oriented handle-graphics system...
 
]]>
Graphic handle properties

There are several ways to access (read or update) Handle Graphics object properties. The simplest (and documented) way is to use the built-in get and set functions on the HG object’s handle. However, it is not the fastest: a significant speedup is possible (see below).

Accessing individual properties is so fast that this speedup may not seem important. Individual properties are accessed in tens or hundreds of microseconds, which is a very short time for most Matlab programs. Indeed, if our application seldom accesses properties, it is probably not worth the effort to optimize this particular aspect of the program:

% Individual property access is extremely fast:
hFig = figure;
tic, figName = get(hFig,'name'); toc  % Elapsed time is 0.000229 seconds. 
 
tic, set(hFig,'name','testing'); toc  % Elapsed time is 0.000270 seconds.

But if we have thousands of reads and/or updates, this could possibly become an important factor (drawnow calls are intentionally omitted to illustrate the effect):

% Using the standard set() function
tic, for idx=1:10000, set(hFig,'name','testing'); end, toc  % Elapsed time is 0.229772 seconds. 
 
% Using the HG handle() wrapper is about 25% faster (or 1.3x speedup):
tic
hFig = handle(hFig);
for idx=1:10000, set(hFig,'name','testing'); end
toc  % Elapsed time is 0.170205 seconds. 
 
% Using the HG handle() wrapper with dot notation is even faster (65% faster, or 2.7x speedup):
tic
hFig = handle(hFig);
for idx=1:10000, hFig.name='testing'; end
toc  % Elapsed time is 0.083762 seconds.

Similar results occur when trying to get() a property value:

% Using the standard set() function
tic, for idx=1:10000, s=get(hFig,'name'); end, toc   % Elapsed time is 0.140865 seconds.
 
% Using the HG handle() wrapper is about 25% faster (or 1.3x speedup):
tic, hFig=handle(hFig); for idx=1:10000, s=get(hFig,'name'); end, toc  % Elapsed time is 0.108543 seconds.
 
% Using the HG handle() wrapper with dot notation is even faster (62% faster, or 2.6x speedup):
tic, hFig=handle(hFig); for idx=1:10000, s=hFig.name; end, toc   % Elapsed time is 0.053423 seconds.

We learn from this that using the HG handle() wrapper function is useful for improving performance. This has the benefit of improving our code performance with minimal changes to our code, by simply updating our handle to be a handle() wrapper:

hFig = handle(hFig);

Using the handle’d handle enables further performance benefits if we use the dot notation (handle.propertyName), rather than use the familiar get/set functions.

Note that both the handle function and its beneficial effects on performance are undocumented.

Java reference properties

The same conclusions also hold true for Java objects, where it turns out that using handle() and the simple dot notation is 2-6 times as fast as using the Java set<PropName> and get<PropName> functions, due to Matlab-Java interface aspects:

jb = javax.swing.JButton;
 
% Using the standard set() function
tic, for idx=1:10000, set(jb,'Text','testing'); end, toc  % Elapsed time is 0.278516 seconds.
 
% Using the HG handle() wrapper is about 35% faster (or 1.6x speedup):
jhb = handle(jb);
tic, for idx=1:10000, set(jhb,'Text','testing'); end, toc   % Elapsed time is 0.175018 seconds.
 
% Using the HG handle() wrapper with dot notation is even faster (65% faster, or 2.8x speedup):
tic, for idx=1:10000, jhb.text='testing'; end, toc  % Elapsed time is 0.100239 seconds.
 
% Using the Java setText() function, is actually slower (faster with the handle() wrapper, but still slower than dot-notation):
tic, for idx=1:10000, jb.setText('testing'); end, toc  % Elapsed time is 0.587543 seconds.
tic, for idx=1:10000, jhb.setText('testing'); end, toc  % Elapsed time is 0.201635 seconds.

The same holds true also for retrieving property values via the get function.

User handle class properties

Exactly the same conclusions also apply to non-graphical user classes that derive from the base handle class regarding property access. In this case, we derive the hgsetget built-in class, which provides a handle class with the standard get/set functions. Note that this is documented – it is only the performance implications that are undocumented in this case.

We first create a simple class for testing:

% Define a simple handle class
classdef TestClass < hgsetget 
    properties
        name
    end
end

Now let’s test both sides of the property access (get/set) – first let’s set the property value:

obj = TestClass;
 
% Using the standard set() function
tic, for idx=1:10000, set(obj,'name','testing'); end, toc  % Elapsed time is 0.138276 seconds.
 
% Using class.propName notation - 72x faster!
tic, for idx=1:10000, obj.name='testing'; end, toc  % Elapsed time is 0.001906 seconds.

And similarly for retrieving a property value:

% Using the standard set() function
tic, for idx=1:10000, a=get(obj,'name'); end, toc  % Elapsed time is 0.105168 seconds.
 
% Using class.propName notation - 6.5x faster
tic, for idx=1:10000, a=obj.name; end, toc  % Elapsed time is 0.016179 seconds.

Conclusions

The general conclusion is that we should always strive to use the dot notation (handle.propertyName), rather than use the familiar get/set functions, in performance hotspots. Naturally, doing this in non-hotspot locations is futile, since accessing individual properties is relatively fast.

In the upcoming HG2, when all of Matlab’s GUI and graphic objects will use Matlab classes, this conclusion will be more important than ever. Initial tests on the HG2 alpha (that anyone can access) show that they hold true for HG2 just as they do in the current HG1.

The handle() function wrapper produces a UDD (schema) object, so while I have not specifically tested non-HG schema objects, I would be very surprised if the conclusions do not hold true for all UDD objects in general.

I have also not [yet] tested other types of handles (ActiveX/COM, Dot-Net etc.) but again, I would be very surprised if the conclusions would be any different.

If this performance tip has piqued your interest, then you might be interested in the several hundred other tips in my upcoming book “MATLAB Performance Tuning” (CRC Press, 2014). I’ll post a note on this blog letting everyone know when it’s officially available. In the meantime, enjoy my other performance-related articles – there are already several dozen of them by now, and the list will continue to grow…

Editorial note: In the past month I’ve gone to quite a lot of effort to improve the performance of this website. Most users should now see pages loading about twice as fast as one month ago. The effect should be especially noticeable on mobile devices connected over mobile, which is much slower than wifi/cable. I still have a few additional tweak ideas, so I expect performance to slightly improve even further in the upcoming weeks. Enjoy reading!

]]>
https://undocumentedmatlab.com/blog/performance-accessing-handle-properties/feed 13
Customizing the standard figure toolbar, menubarhttps://undocumentedmatlab.com/blog/customizing-standard-figure-toolbar-menubar https://undocumentedmatlab.com/blog/customizing-standard-figure-toolbar-menubar#comments Wed, 09 Jan 2013 18:00:12 +0000 http://undocumentedmatlab.com/?p=3461
 
Related posts:
  1. uisplittool & uitogglesplittool callbacks Matlab's undocumented uisplittool and uitogglesplittool are powerful toolbar controls - this article explains how to customize their behavior...
  2. Figure toolbar components Matlab's toolbars can be customized using a combination of undocumented Matlab and Java hacks. This article describes how to access existing toolbar icons and how to add non-button toolbar components....
  3. uisplittool & uitogglesplittool Matlab's undocumented uisplittool and uitogglesplittool are powerful controls that can easily be added to Matlab toolbars - this article explains how...
  4. Customizing figure toolbar background Setting the figure toolbar's background color can easily be done using just a tiny bit of Java magic powder. This article explains how. ...
 
]]>
A few days ago, a client asked me to integrate an MRU (most-recently-used) file list in a Matlab GUI window. The naive approach was to add a new “Recent files” main menu, but this would look bad. Today I explain how to integrate the MRU list into the toolbar’s standard “Open File” button, as well as into the standard “File” main menu.

Note: this relies on earlier articles about customizing the figure menubar, so if you are not comfortable with this topic you might benefit from reading these earlier articles.

Customizing the standard “File” main menu is easy. First, let’s find the “File” main menu’s handle, and add an MRU sub-menu directly to it:

hFileMenu = findall(gcf, 'tag', 'figMenuFile');
hMruMenu = uimenu('Label','Recent files', 'Parent',hFileMenu);

Our new MRU menu item is created at the end (in this case, at the bottom of the “File” main menu list). Let’s move it upward, between “New” and “Open…”, by reordering hFileMenu‘s Children menu items:

hAllMenuItems = allchild(hFileMenu);
set(hFileMenu, 'Children',fliplr(hAllMenuItems([2:end-1,1,end])));  % place in 2nd position, just above the "Open" item

Now let’s fix the “Open…” menu item’s callback to point to our custom openFile() function (unfortunately, the “Open…” menu item has no tag, so we must rely on its label to get its handle):

hOpenMenu = findall(hFileMenu, 'Label', '&Open...');
set(hOpenMenu, 'Callback',@openFile);

Finally, let’s add the MRU list as a sub-menu of the new hMruMenu. I assume that we have a getMRU() function in our code, which returns a cell-array of filenames:

% Add the list of recent files, one item at a time
filenames = getMRU();
for fileIdx = 1 : length(filenames)
    uimenu(hMruMenu, 'Label',filenames{fileIdx}, 'Callback',{@openFile,filenames{fileIdx}});
end

Modified standard figure menu-bar

Modified standard figure menu-bar


Clicking the main “Open…” menu item calls our openFile() function without the optional filename input argument, while selecting one of the MRU files will call the openFile() function with that specific filename as input. The openFile() function could be implemented something like this:

% Callback for the open-file functionality (toolbar and menubar)
function openFile(hObject,eventData,filename)
    % If no filename specified, ask for it from the user
    if nargin < 3
        filename = uigetfile({'*.csv','Data files (*.csv)'}, 'Open data file');
        if isempty(filename) || isequal(filename,0)
            return;
        end
    end
 
    % Open the selected file and read the data
    data = readDataFile(filename);
 
    % Update the display
    updateGUI(data)
end

We can take this idea even further by employing HTML formatting, as I have shown in my first article of the menubar mini-series:

HTML-rendered menu items

HTML-rendered menu items

Customizing the standard toolbar’s “Open File” button

Note: this relies on earlier articles about customizing the figure toolbar, so if you are not comfortable with this topic you might benefit from reading these earlier articles.

The basic idea here is to replace the standard toolbar’s “Open File” pushbutton with a new uisplittool button that will contain the MRU list in its picker-menu.

The first step is to get the handle of the toolbar’s “Open File” button:

hOpen = findall(gcf, 'tooltipstring', 'Open File');
hOpen = findall(gcf, 'tag', 'Standard.FileOpen');  % Alternative

The second alternative is better for non-English Matlab installations where the tooltip text may be different, or in cases where we might have another GUI control with this specific tooltip. On the other hand, the 'Standard.FileOpen' tag may be different in different Matlab releases. So choose whichever option is best for your specific needs.

Assuming we have a valid (non-empty) hOpen handle, we get its properties data for later use:

open_data = get(hOpen);
hToolbar = open_data.Parent;

We have all the information we need, so we can now simply delete the existing toolbar open button, and create a new split-button with the properties data that we just got:

delete(hOpen);
hNewOpen = uisplittool('Parent',hToolbar, ...
                       'CData',open_data.CData, ...
                       'Tooltip',open_data.TooltipString, ...
                       'ClickedCallback',@openFile);

As with the menubar, the button is now created, but it appears on the toolbar’s right edge. Let’s move it to the far left. We could theoretically reorder hToolbar‘s Children as for the menubar above, but Matlab has an internal bug that causes some toolbar buttons to misbehave upon rearranging. Using Java solves this:

drawnow;  % this innocent drawnow is *very* important, otherwise Matlab might crash!
jToolbar = get(get(hToolbar,'JavaContainer'),'ComponentPeer');
jButtons = jToolbar.getComponents;
jToolbar.setComponentZOrder(jButtons(end),2);  % Move to the position between "New" and "Save"
jToolbar.revalidate;  % update the toolbar's appearance (drawnow will not do this)

Finally, let’s add the list of recent files to the new split-button’s picker menu:

% (Use the same list of filenames as for the menubar's MRU list)
jNewOpen = get(hNewOpen,'JavaContainer');
jNewOpenMenu = jNewOpen.getMenuComponent;
for fileIdx = 1 : length(filenames)
    jMenuItem = handle(jNewOpenMenu.add(filenames{fileIdx}),'CallbackProperties');
    set(jMenuItem,'ActionPerformedCallback',{@openFile,filenames{fileIdx}});
end

Modified standard figure toolbar

Modified standard figure toolbar

Clicking the main Open button calls our openFile() function without the optional filename input argument, while clicking the picker button (to the right of the main Open button) and selecting one of the files will call the openFile() function with that specific filename as input.

That’s it. Quite painless in fact.

]]>
https://undocumentedmatlab.com/blog/customizing-standard-figure-toolbar-menubar/feed 12
Pinning annotations to graphshttps://undocumentedmatlab.com/blog/pinning-annotations-to-graphs https://undocumentedmatlab.com/blog/pinning-annotations-to-graphs#comments Wed, 12 Dec 2012 18:00:51 +0000 http://undocumentedmatlab.com/?p=3398
 
Related posts:
  1. uiundo – Matlab’s undocumented undo/redo manager The built-in uiundo function provides easy yet undocumented access to Matlab's powerful undo/redo functionality. This article explains its usage....
  2. Matlab’s HG2 mechanism HG2 is presumably the next generation of Matlab graphics. This article tries to explore its features....
  3. New information on HG2 More information on Matlab's new HG2 object-oriented handle-graphics system...
  4. Adding dynamic properties to graphic handles It is easy and very useful to attach dynamic properties to Matlab graphics objects in run-time. ...
 
]]>
Many Matlab users are aware of Matlab’s annotation functionality, which enables us to assign graphic elements such as arrows, lines, ellipses and text labels to Matlab figures. Matlab has a corresponding built-in function, annotation, that enables creation of annotation objects. Through the handle returned by annotation we can customize the annotation’s appearance (for example, line width/style or text font properties).

Limitations of Matlab annotations

Unfortunately, annotation has several major deficiencies, that are in fact related:

A Matlab text-arrow annotation (unpinned)

A Matlab text-arrow annotation (unpinned)

  1. annotation requires us to specify the annotation’s position in normalized figure units. Often, we are interested in an annotation on a plot axes that does NOT span the entire figure’s content area. To correctly convert the position from plot axes data coordinates to figure coordinates requires non-trivial calculations.
  2. The created annotation is NOT pinned to the plot axes by default. This means that the annotation retains its relative position in the figure when the plot is zoomed, panned or rotated. This results in unintelligible and misleading annotations. We can indeed pin the annotation to the graph, but this requires delicate manual interaction (click the Edit Plot toolbar icon, then right-click the relevant annotation end-point, then select “Pin to Axes” from context menu). Unfortunately, the annotation handle does not provide a documented way to do this programmatically.
  3. Finally, the annotation objects are only displayed on top of plot axes – they are obscured by any GUI uicontrols that may happen to be present in the figure.

All of these limitations originate from the underlying implementation of annotation objects in Matlab. This is based on a transparent hidden axes that spans the entire figure’s content area, on which the annotations are being drawn (also called the scribe layer). The annotations may appear to be connected to the plot axes, but this is merely a visual illusion. In fact, they are located in a separate axes layer. For this reason, annotation requires figure position – in fact, the annotation has no information about the axes beneath it. Since plot axes are always obscured by uicontrols, so too is the annotation layer.

Matlab’s implementation of annotation is an attempt to replicate Java’s standard glass-pane mechanism. But whereas the Java glass-pane is a true transparent layer, on top of all other window components (examples), Matlab’s implementation only works for axes.

Oh well, it’s better than nothing, I guess. But still, it would be nice if we could specify the annotation in graph (plot axes) data units, and have it pinned automatically without requiring manual user interaction.

Debugging the problem

The obvious first place to start debugging this issue is to go to the annotation handle’s context-menu (accessible via the UIContextMenu property), drill down to the “Pin” menu item and take a look at its callback. We could then use the hgfeval function to execute this callback programmatically. Unfortunately, this does not work well, because the context-menu is empty when the annotation is first created. A context-menu is only assigned to the annotation after the Edit Plot toolbar button and then the annotation object are clicked.

Being too lazy in nature to debug this all the way through, I opted for an easier route: I started the Profiler just prior to clicking the context-menu’s “Pin to Axes”, and stopped it immediately afterwards. This showed me the code path (beneath %matlabroot%/toolbox/matlab/scribe/), and placing breakpoints in key code lines enabled me to debug the process step-by-step. This in turn enabled me to take the essence of the pinning code and implement it in my stand-alone application code.

Believe me when I say that the scribe code is complex (anyone say convoluted?). So I’ll spare you the gruesome details and skip right to the chase.

The solution

Positioning the annotation in axes data units

The first step is to ensure that the initial annotation position is within the figure bounds. Otherwise, the annotation function will shout. Note that it is ok to move the annotation outside the figure bounds later on (via panning/zooming) – it is only the initial annotation creation that must be within the figure bounds (i.e., between 0.0-1.0 in normalized X and Y units):

% Prepare the annotation's X position
% Note: we need 2 X values: one for the annotation's head, another for the tail
x = [xValue, xValue];
xlim = get(hAxes,'XLim');
 
% Prepare the annotation's Y position
% Note: we need 2 Y values: one for the annotation's head, another for the tail
% Note: we use a static Y position here, spanning the center of the axes.
% ^^^^  We could have used some other Y data value for this
yLim = get(hAxes,'YLim');
y = yLim(1) + 0*sum(yLim) + [0.1,0]*diff(ylim);  % TODO: handle reverse, log Y axes
 
% Ensure that the annotation fits in the window by enlarging
% the axes limits as required
if xValue < xlim(1) || xValue > xlim(2)
    hold(hAxes,'on');
    plot(hAxes,xValue,y(2),'-w');
    drawnow;
 
    % YLim may have changed, so recalculate y
    yLim = get(hAxes,'YLim');
    y = yLim(1) + 0*sum(yLim) + [0.1,0]*diff(ylim);  % TODO: handle reverse, log Y-axes
end

Next, we convert our plot data units, in order to get the annotation’s requested position in the expected figure units. For this we use %matlabroot%/toolbox/matlab/scribe/@scribe/@scribepin/topixels.m. This is an internal method of the scribepin UDD class, so in order to use it we need to create a dummy scribepin object. topixels then converts the dummy object’s position from axes data units to pixel units. We then use the undocumented hgconvertunits function to convert from pixel units into normalized figure units:

% Convert axes data position to figure normalized position
% uses %matlabroot%/toolbox/matlab/scribe/@scribe/@scribepin/topixels.m
scribepin = scribe.scribepin('parent',hAxes,'DataAxes',hAxes,'DataPosition',[x;y;[0,0]]');
figPixelPos = scribepin.topixels;
hFig = ancestor(hAxes,'figure');
figPos = getpixelposition(hFig);
figPixelPos(:,2) = figPos(4) - figPixelPos([2,1],2);
figNormPos = hgconvertunits(hFig,[figPixelPos(1,1:2),diff(figPixelPos)],'pixels','norm',hFig);
annotationX = figNormPos([1,1]);
annotationY = figNormPos([2,2]) + figNormPos(4)*[1,0];
Pinning the annotation to the axes data

Finally, we use the annotation handle’s pinAtAffordance() method and set the Pin.DataPosition property to the requested X,Y values (we need to do both of these, otherwise the annotation will jump around when we zoom/pan):

A Matlab text-arrow annotation (pinned)

A Matlab text-arrow annotation (pinned)

% Ensure that the annotation is within the axes bounds, then display it
if any([annotationX,annotationY] < 0) || any([annotationX,annotationY] > 1)
    % Annotation position is outside axes boundaries, so bail out without drawing
    hAnnotation = handle([]);
elseif ~isempty(annotationObj)
    % Create a text-arrow annotation with the requested string at the requested position
    hAnnotation = handle(annotation('textarrow', annotationX, annotationY, ...
                                    'String',annotationStr, 'TextColor','b', 'Tag','annotation'));
 
    % Example for setting annotation properties
    hAnnotation.TextEdgeColor = [.8,.8,.8];
 
    % Pin the annotation object to the required axes position
    % Note: some of the following could fail in certain cases - never mind
    try
        hAnnotation.pinAtAffordance(1);
        hAnnotation.pinAtAffordance(2);
        hAnnotation.Pin(1).DataPosition = [xValue, y(1), 0];
        hAnnotation.Pin(2).DataPosition = [xValue, y(2), 0];
    catch
        % never mind - ignore (no error)
    end
end

p.s. Notice that all this relies on pure Matlab code (i.e., no mention of the dreaded J-word…). In fact, practically the entire scribe code is available in m-file format in the base Matlab installation. Masochistic readers may find many hours of pleasure sifting through the scribe code functionality for interesting nuggets such as the one above. If you ever find any interesting items, please drop me an email, or post a comment below.

Undocumented annotation properties

Annotation objects have a huge number of undocumented properties. In fact, they have more undocumented properties than documented ones. You can see this using my uiinspect or getundoc utilities. Here is the list for a simple text-arrow annotation, such as the one that we used above:

>> getundoc(hAnnotation)
ans = 
              ALimInclude: 'on'
                   Afsize: 6
          ApplicationData: [1x1 struct]
                 Behavior: [1x1 struct]
              CLimInclude: 'on'
               ColorProps: {5x1 cell}
     EdgeColorDescription: 'Color'
        EdgeColorProperty: 'Color'
                  Editing: 'off'
                EraseMode: 'normal'
     FaceColorDescription: 'Head Color'
        FaceColorProperty: 'HeadColor'
             FigureResize: 0
            HeadBackDepth: 0.35
                HeadColor: [0 0 0]
            HeadColorMode: 'auto'
            HeadEdgeColor: [0 0 0]
            HeadFaceAlpha: 1
            HeadFaceColor: [0 0 0]
               HeadHandle: [1x1 patch]
         HeadHypocycloidN: 3
            HeadLineStyle: '-'
            HeadLineWidth: 0.5
               HeadRosePQ: 2
                 HeadSize: 10
             HelpTopicKey: ''
          IncludeRenderer: 'on'
                 MoveMode: 'mouseover'
                    NormX: [0.2 0.4]
                    NormY: [0.5 0.7]
                      Pin: [0x1 double]
                   PinAff: [1 2]
           PinContextMenu: [2x1 uimenu]
                PinExists: [0 0]
              PixelBounds: [0 0 0 0]
        PropertyListeners: [8x1 handle.listener]
        ScribeContextMenu: [9x1 uimenu]
                 Selected: 'off'
             Serializable: 'on'
                ShapeType: 'textarrow'
                    Srect: [2x1 line]
           StoredPosition: []
                TailColor: [0 0 0]
               TailHandle: [1x1 line]
            TailLineStyle: '-'
            TailLineWidth: 0.5
     TextColorDescription: 'Text Color'
            TextColorMode: 'auto'
        TextColorProperty: 'TextColor'
        TextEdgeColorMode: 'manual'
            TextEraseMode: 'normal'
               TextHandle: [1x1 text]
         UpdateInProgress: 0
    VerticalAlignmentMode: 'auto'
              XLimInclude: 'on'
              YLimInclude: 'on'
              ZLimInclude: 'on'
]]>
https://undocumentedmatlab.com/blog/pinning-annotations-to-graphs/feed 13
Setting axes tick labels formathttps://undocumentedmatlab.com/blog/setting-axes-tick-labels-format https://undocumentedmatlab.com/blog/setting-axes-tick-labels-format#comments Wed, 18 Apr 2012 18:00:17 +0000 http://undocumentedmatlab.com/?p=2856
 
Related posts:
  1. Introduction to UDD UDD classes underlie many of Matlab's handle-graphics objects and functionality. This article introduces these classes....
  2. Inactive Control Tooltips & Event Chaining Inactive Matlab uicontrols cannot normally display their tooltips. This article shows how to do this with a combination of undocumented Matlab and Java hacks....
  3. Multi-column (grid) legend This article explains how to use undocumented axes listeners for implementing multi-column plot legends...
  4. Controlling plot data-tips Data-tips are an extremely useful plotting tool that can easily be controlled programmatically....
 
]]>
Have you ever tried to customize the way in which tick labels appear in Matlab plot axes?

For example, setting the numerical precision of the labels, or adding some short descriptive text (for example, the units)? If you have, then I bet that you have encountered the following dilemma: Once we modify the tick labels (for discussion sake, let’s assume the Y axis, so this is done by updating the YTickLabel property), then the corresponding YTickLabelMode property changes from ‘auto’ to ‘manual’ and loses its relationship to the tick values (YTick). So, if we now zoom or pan the plot, our new labels remain unchanged although the tick values have changed, causing much panic and frustration… If we also set the tick values manually, this solves that problem but leaves us with another: now, when we zoom or pan, we no longer see any ticks or tick labels at all!

Original plot Manual labels, auto ticks Manual labels, manual ticks

Original plot (left)
Manual labels, auto ticks (center)
Manual labels, manual ticks (right)

Of course, we can always trap the zoom and pan callback functions to update the tick labels dynamically while keeping the tick values automatically. This will work for these cases, but we need to do it separately for zoom and pan. Also, if we modify the axes limits explicitly (via the corresponding YLim property) or indirectly (by modifying the displayed plot data), then the callbacks are not called and the labels are not updated.

The solution – using a property change listener

A better way to solve this problem is to simply trap changes to the displayed tick values, and whenever these occur to call our dedicated function to update the labels according to the new tick values. This can be done by using UDD, or more precisely the ability to trap update events on any property (in our case, YTick). Such a mechanism was already demonstrated here in 2010, as one way to achieve continuous slider feedback. The idea is to use the built-in handle.listener function with the PropertyPostSet event, as follows:

hhAxes = handle(hAxes);  % hAxes is the Matlab handle of our axes
hProp = findprop(hhAxes,'YTick');  % a schema.prop object
hListener = handle.listener(hhAxes, hProp, 'PropertyPostSet', @myCallbackFunction);
setappdata(hAxes, 'YTickListener', hListener);

Note that we have used setappdata to store the hListener handle in the axes. This ensures that the listener exists for exactly as long as the axes does. If we had not stored this listener handle somewhere, then Matlab would have immediately deleted the listener hook and our callback function would not have been called upon tick value updates. Forgetting to store listener handles is a common pitfall when using them. If you take a look at the addlistener function’s code, you will see that it also uses setappdata after creating the listener, for exactly this reason. Unfortunately, addlistsner cannot always be used, and I keep forgetting under which circumstances, so I generally use handle.listener directly as above: It’s simple enough to use that I find I never really need to use the simple addlistener wrapper, but you are welcome to try it yourself.

That’s all there is to it: Whenever YTick changes its value(s), our callback function (myCallbackFunction) will automatically be called. It is quite simple to set up. While we cannot use TeX in tick labels yet (this will change in the upcoming HG2), using sprintf formatting does enable quite a bit of flexibility in formatting the labels. For example, let’s say I want my tick labels to have the format ‘%.1fV’ (i.e., always one decimal, plus the Volts units):

function myCallbackFunction(hProp,eventData)    %#ok - hProp is unused
   hAxes = eventData.AffectedObject;
   tickValues = get(hAxes,'YTick');
   newLabels = arrayfun(@(value)(sprintf('%.1fV',value)), tickValues, 'UniformOutput',false);
   set(hAxes, 'YTickLabel', newLabels);
end  % myCallbackFunction

Manual labels, automatically updated Manual labels, automatically updated

Manual labels, automatically updated

Handling duplicate tick labels

Of course, ‘%.1fV’ may not be a good format when we zoom in to such a degree that the values differ by less than 0.1 – in this case all the labels will be the same. So let’s modify our callback function to add extra decimals until the labels become distinct:

function myCallbackFunction(hProp,eventData)    %#ok - hProp is unused
   hAxes = eventData.AffectedObject;
   tickValues = get(hAxes,'YTick');
 
   %newLabels = arrayfun(@(value)(sprintf('%.1fV',value)), tickValues, 'UniformOutput',false);
   digits = 0;
   labelsOverlap = true;
   while labelsOverlap
      % Add another decimal digit to the format until the labels become distinct
      digits = digits + 1;
      format = sprintf('%%.%dfV',digits);
      newLabels = arrayfun(@(value)(sprintf(format,value)), tickValues, 'UniformOutput',false);
      labelsOverlap = (length(newLabels) > length(unique(newLabels)));
 
      % prevent endless loop if the tick values themselves are non-unique
      if labelsOverlap && max(diff(tickValues))< 16*eps
         break;
      end
   end
 
   set(hAxes, 'YTickLabel', newLabels);
end  % myCallbackFunction

non-distinct labels distinct labels

Non-distinct labels                   distinct labels

ticklabelformat

Based on a file that I received from an anonymous reader a few years ago, I have prepared a utility called ticklabelformat that automates much of the set-up above. Feel free to download this utility and modify it for your needs – it’s quite simple to read and follow. The usage syntax is as follows:

ticklabelformat(gca,'y','%.6g V')  % sets y axis on current axes to display 6 significant digits
ticklabelformat(gca,'xy','%.2f')   % sets x & y axes on current axes to display 2 decimal digits
ticklabelformat(gca,'z',@myCbFcn)  % sets a function to update the Z tick labels on current axes
ticklabelformat(gca,'z',{@myCbFcn,extraData})  % sets an update function as above, with extra data
]]>
https://undocumentedmatlab.com/blog/setting-axes-tick-labels-format/feed 13
Profiling Matlab memory usagehttps://undocumentedmatlab.com/blog/profiling-matlab-memory-usage https://undocumentedmatlab.com/blog/profiling-matlab-memory-usage#comments Thu, 01 Mar 2012 00:13:04 +0000 http://undocumentedmatlab.com/?p=2768
 
Related posts:
  1. Undocumented profiler options The Matlab profiler has some undocumented options that facilitate debugging memory bottlenecks and JIT (Just-In-Time Java compilation) problems....
  2. Datenum performance The performance of the built-in Matlab function datenum can be significantly improved by using an undocumented internal help function...
  3. Undocumented Profiler options part 3 An undocumented feature of the Matlab Profiler can report call history timeline - part 3 of series. ...
  4. Undocumented Profiler options part 4 Several undocumented features of the Matlab Profiler can make it much more useful - part 4 of series. ...
 
]]>
Anyone who has had experience with real-life applications knows that Memory usage can have a significant impact on the application’s usability, in aspects such as performance, interactivity, and even (on some lousy memory-management Operating Systems) crashes/hangs.

In Matlab releases of the past few years, this has been addressed by expanding the information reported by the built-in memory function. In addition, an undocumented feature was added to the Matlab Profiler that enables monitoring memory usage.


Profile report with memory & JIT infoProfile report with memory & JIT info

Profile report with memory & JIT info

Profile report with memory & JIT info

In Matlab release R2008a (but not on newer releases) we could also use a nifty parameter of the undocumented feature function:

>> feature mtic; a=ones(100); feature mtoc
ans = 
      TotalAllocated: 84216
          TotalFreed: 2584
    LargestAllocated: 80000
           NumAllocs: 56
            NumFrees: 43
                Peak: 81640

As can easily be seen in this example, allocating 1002 doubles requires 80000 bytes of allocation, plus some 4KB others that were allocated (and 2KB freed) within the function ones. Running the same code line again gives a very similar result, but now there are 80000 more bytes freed when the matrix a is overwritten:

>> feature mtic; a=ones(100); feature mtoc
ans = 
      TotalAllocated: 84120
          TotalFreed: 82760
    LargestAllocated: 80000
           NumAllocs: 54
            NumFrees: 49
                Peak: 81328

This is pretty informative and very handy for debugging memory bottlenecks. Unfortunately, starting in R2008b, features mtic and mtoc are no longer supported “under the current memory manager. Sometime around 2010 the mtic and mtoc features were completely removed. Users of R2008b and newer releases therefore need to use the internal structs returned by the memory function, and/or use the profiler’s memory-monitoring feature. If you ask me, using mtic/mtoc was much simpler and easier. I for one miss these features.


In a related matter, if we wish to monitor Java’s memory used within Matlab, we are in a bind, because there are no built-in tools to help us. there are several JVM switches that can be turned on in the java.opts file: -Xrunhprof[:help]|[:option=value,…], -Xprof, -Xrunprof, -XX:+PrintClassHistogram and so on. There are several memory-monitoring (so-called “heap-walking”) tools: the standard JDK jconsole, jmap, jhat and jvisualvm (with its useful plugins) provide good basic coverage. MathWorks has posted a tutorial on using jconsole with Matlab. There are a number of other third-party tools such as JMP (for JVMs 1.5 and earlier) or TIJMP (for JVM 1.6). Within Matlab, we can use utilities such as Classmexer to estimate a particular object’s size (both shallow and deep referencing), or use java.lang.Runtime.getRuntime()‘s methods (maxMemory(), freeMemory() and totalMemory()) to monitor overall Java memory (sample usage).

We can monitor the Java memory (which is part of the overall Matlab process memory) usage using Java’s built-in Runtime class:

>> r=java.lang.Runtime.getRuntime
r =
java.lang.Runtime@5fb3b54
 
>> r.freeMemory
ans =
    86147768
 
>> r.totalMemory
ans =
   268304384
 
>> usedMemory = r.totalMemory - r.freeMemory;

Specifically in R2011b (but in no other release), we can also use a built-in Java memory monitor. Unfortunately, this simple and yet useful memory monitor was removed in R2012a (or maybe it was just moved to another package and I haven’t found out where… yet…):

com.mathworks.xwidgets.JavaMemoryMonitor.invoke

Matlab R2011b's Java memory monitor

Matlab R2011b's Java memory monitor

As I have already noted quite often, using undocumented Matlab features and functions carries the risk that they will not be supported in some future Matlab release. Today’s article is a case in point.

]]>
https://undocumentedmatlab.com/blog/profiling-matlab-memory-usage/feed 8
Converting Java vectors to Matlab arrayshttps://undocumentedmatlab.com/blog/converting-java-vectors-to-matlab-arrays https://undocumentedmatlab.com/blog/converting-java-vectors-to-matlab-arrays#comments Wed, 14 Dec 2011 18:00:47 +0000 http://undocumentedmatlab.com/?p=2613
 
Related posts:
  1. Types of undocumented Matlab aspects This article lists the different types of undocumented/unsupported/hidden aspects in Matlab...
  2. Matlab-Java memory leaks, performance Internal fields of Java objects may leak memory - this article explains how to avoid this without sacrificing performance. ...
  3. ScreenCapture utility The ScreenCapture utility uses purely-documented Matlab for capturing a screen region as an image from within Matlab. ...
  4. Pause for the better Java's thread sleep() function is much more accurate than Matlab's pause() function. ...
 
]]>
Matlab includes built-in support for automatic conversion of Matlab cell arrays into Java arrays. This is important in cases when we need to pass information to a Java function that expects an array (e.g., String[]).

Numeric data array

In some cases, namely Java numeric arrays, Matlab also automatically converts the Java array into Matlab arrays. This is actually inconvenient when we would like to access the original Java reference in order to modify some value – since the Java reference is inaccessible from Matlab in this case, the data is immutable.

>> jColor = java.awt.Color.red
jColor =
java.awt.Color[r=255,g=0,b=0]
 
>> matlabData = jColor.getColorComponents([])
matlabData =
     1
     0     % < = immutable array of numbers, not a reference to int[]
     0

Non-numeric array

Very often we encounter cases in Java where the information is stored in an array of non-numeric data. In such cases we need to apply a non-automatic conversion from Java into Matlab.

If the objects are of exactly the same type, then we could store them in a simple Matlab array; otherwise (as can be seen in the example below), we could store them in either a simple array of handles, or in a simple cell array:

>> jFrames = java.awt.Frame.getFrames
jFrames =
java.awt.Frame[]:
    [javax.swing.SwingUtilities$SharedOwnerFrame ]
    [com.mathworks.mde.desk.MLMainFrame          ]
    [com.mathworks.mde.desk.MLMultipleClientFrame]
    [com.mathworks.mwswing.MJFrame               ]
 
% Alternative #1 - use a loop
>> mFrames = handle([]); for idx = 1 : length(jFrames); mFrames(idx)=handle(jFrames(idx)); end
>> mFrames
mFrames =
	handle: 1-by-4
>> mFrames(1)
ans =
	javahandle.javax.swing.SwingUtilities$SharedOwnerFrame
>> mFrames(2)
ans =
	javahandle.com.mathworks.mde.desk.MLMainFrame
 
% Alternative #2a - convert into a Matlab cell array
>> mFrames = jFrames.cell
mFrames = 
    [1x1 javax.swing.SwingUtilities$SharedOwnerFrame ]
    [1x1 com.mathworks.mde.desk.MLMainFrame          ]
    [1x1 com.mathworks.mde.desk.MLMultipleClientFrame]
    [1x1 com.mathworks.mwswing.MJFrame               ]
 
% Alternative #2b - convert to a cell array (equivalent variant of alternative 2a)
>> mFrames = cell(jFrames);

Note that if we only need to access a particular item in the Java vector or array, we could do that directly, without needing to convert the entire data into Matlab first. Simply use jFrames(1) to directly access the first item in the jFrames array, for example.

(note: Java Frames are discussed in chapters 7 and 8 of my Matlab-Java book).

Vectors and other Collections

Very often we encounter cases in Java where the information is stored in a Java Collection rather than in a simple Java array. The basic mechanism for the conversion in this case is to first convert the Java data into a simple Java array (in cases it was not so in the first place), and then to convert this into a Matlab array using either the automated conversion (if the data is numeric), or using a for loop (ugly and slow!), or into a cell array using the cell function, as explained above.

Different Collections have different manners of converting into a Java array: some Collections return an Iterator/Enumerator that can be processed in a loop (be careful not to reset the iterator reference by re-reading it within the loop):

% Wrong way - causes an infinite loop
idx = 1;
props = java.lang.System.getProperties;
while props.elements.hasMoreElements
    mPropValue{idx} = props.elements.nextElement;
end
 
% Right way
idx = 1;
propValues = java.lang.System.getProperties.elements;  % Enumerator
while propValues.hasMoreElements
    mPropValue{idx} = propValues.nextElement;
end

(note: system properties are discussed in section 1.9 of my Matlab-Java book; Collections are discussed in section 2.1)

Other Collections, such as java.util.Vector, have a toArray() method that directly converts into a Java array, and we can process from there as described above:

>> jVector = java.util.Vector;
>> jVector.add(1); jVector.add(2); jVector.add(3);
>> jVector.addAll(jv); jVector.addAll(jv);
>> jVector
jVector =
[1.0, 2.0, 3.0, 1.0, 2.0, 3.0, 1.0, 2.0, 3.0, 1.0, 2.0, 3.0]
 
% Now convert into a Matlab cell array via a Java simple array
>> mCellArray = jVector.toArray.cell
mCellArray = 
    [1]
    [2]
    [3]
    [1]
    [2]
    [3]
    [1]
    [2]
    [3]
    [1]
    [2]
    [3]

Performance

It so happens, that the undocumented built-in feature function (or its near-synonym system_dependent) enables improved performance in this conversion process. feature(44) accepts a java.util.Vector and converts it directly into a Matlab cell-array, in one third to one-half the time that it would take the equivalent toArray.cell() (the third input argument is the number of columns in the result – the reshaping is done automatically):

>> mCellArray = feature(44,jVector,jVector.size)   % jVector.size = 12
mCellArray = 
    [1]    [2]    [3]    [1]    [2]    [3]    [1]    [2]    [3]    [1]    [2]    [3]
 
>> mCellArray = feature(44,jVector,4)
mCellArray = 
    [1]    [1]    [1]    [1]
    [2]    [2]    [2]    [2]
    [3]    [3]    [3]    [3]

The conversion process is pretty efficient: On my system, the regular toArray.cell() takes 0.45 seconds for a 100K vector, compared to 0.21 seconds for the feature alternative. However, this small difference could be important in cases where performance is crucial, for example in processing of highly-active Java events in Matlab callbacks, or when retrieving data from a database. And this latter case is indeed where a sample usage of this feature can be found, namely in the cursor.fetch.m function (where it appears as system_dependent(44)).

Please note that both feature and system_dependent are highly prone to change without prior warning in some future Matlab release. On the other hand, the conversion methods that I presented above, excluding feature, will probably still be valid in all Matlab releases in the near future.

]]>
https://undocumentedmatlab.com/blog/converting-java-vectors-to-matlab-arrays/feed 1
Types of undocumented Matlab aspectshttps://undocumentedmatlab.com/blog/types-of-undocumented-matlab-aspects https://undocumentedmatlab.com/blog/types-of-undocumented-matlab-aspects#respond Thu, 24 Nov 2011 18:00:36 +0000 http://undocumentedmatlab.com/?p=2534
 
Related posts:
  1. Matlab callbacks for Java events Events raised in Java code can be caught and handled in Matlab callback functions - this article explains how...
  2. Uitable sorting Matlab's uitables can be sortable using simple undocumented features...
  3. Matlab’s HG2 mechanism HG2 is presumably the next generation of Matlab graphics. This article tries to explore its features....
  4. Minimize/maximize figure window Matlab figure windows can easily be maximized, minimized and restored using a bit of undocumented magic powder...
 
]]>
Why are there so many undocumented aspects in Matlab?

This is a great question, recently asked by a reader of this blog, so I wanted to expand on it in next week’s article. Before specifying the different reasons, let’s map the nature of undocumented aspects that we find in Matlab.

The term undocumented/unsupported (as opposed to mis-documentated or deprecated) actually refers to quite a large number of different types.
In the following list, the hyperlinks on the list-item titles lead to a list of corresponding articles on this website:

  • Undocumented functions
    Matlab functions which appears nowhere in the documentation, are usually built-in functions (do not have an m-file) and can only be inferred from online CSSM posts or usage within one of the Matlab m-functions installed with Matlab (the latter being the usual case). None of these functions is officially supported by MathWorks. MEX is an important source for such functions.

  • Semi-documented functions
    Matlab functionality which exists in Matlab m-functions installed with Matlab, but have their main comment separated from the H1 comment line, thereby hiding it from normal view (via Matlab’s help function). The H1 comment line itself is simply a standard warning that this function is not officially supported and may change in some future version. To see the actual help comment, simply edit the function (using Matlab’s edit function or any text editor) and place a comment sign (%) at the empty line between the H1 comment and the actual help section. The entire help section will then onward be visible via the help function:

            function [tree, container] = uitree(varargin)
            % WARNING: This feature is not supported in MATLAB
            % and the API and functionality may change in a future release.
    fix =>  %
            % UITREE creates a uitree component with hierarchical data in a figure window.
            %   UITREE creates an empty uitree object with default property values in
            %   a figure window.
            %...

    These functions are not documented in the full documentation (via Matlab’s doc function, or online). The odd thing is that some of these functions may appear in the category help output (for example, help(‘uitools’)), and in some cases may even have a fully-visible help section (e.g., help(‘setptr’)), but do not have any online help documentation (docsearch(‘setptr’) fails, and doc(‘setptr’) simply displays the readable help text).

    All these functions are officially unsupported by MathWorks, even when having a readable help section. The rule of thumb appears to be that a Matlab function is supported only if it has online documentation. Note, however, that in some rare cases a documentation discrepancy may be due to a MathWorks documentation error, not to unsupportability…

  • Helper functions
    Many fully-supported Matlab functions use helper functions that have a specific use in the main (documented) function(s). Often, these helper functions are tightly-coupled to their documented parents and are useless as stand-alone functions. But quite a few of them have quite useful stand-alone use, as I’ve already shown in some past articles.

  • Undocumented features and properties
    Features of otherwise-documented Matlab functions, which appear nowhere in the official documentation. You guessed it – these are also not supported by MathWorks… Like undocumented functions, you can only infer such features by the occasional CSSM post or a reference somewhere in Matlab’s m-code.

  • Semi-documented features
    Features of otherwise-documented Matlab functions, which are documented in a separate section beneath the main help section, and nowhere else (not in the full doc not the online documentation). If you did not know in advance that these features existed, you could only learn of them by manually looking at Matlab’s m-files (which is what I do in most cases…).

  • Undocumented properties
    Many Matlab objects have internal properties, which can be retrieved (via Matlab’s get function) and/or set (via the set function) programmatically. All these properties are fully documented. Many objects also possess hidden properties, some of which are very interesting and useful, but which are undocumented and (oh yes) unsupported. Like undocumented features, they can only be inferred from CSSM or existing code. In a recent article I described my getundoc utility, which lists these undocumented properties of specified objects.

  • Internal Matlab classes
    Matlab uses a vast array of specialized Java classes to handle everything from algorithms to GUI. These classes are (of course) undocumented/unsupported. They can often be accessed directly from the Matlab Command Window or user m-files. GUI classes can be inferred by inspecting the figure frame’s Java components, and non-GUI classes can often be inferred from references in Matlab’s m-files.

  • Matlab-Java integration
    Matlab’s GUI interface, as well as the Java-to-Matlab interface (JMI) is fully undocumented and unsupported. In addition to JMI, there are other mechanisms to run Matlab code from within Java (namely JMI, COM and DDE) – these are all unsupported and by-and-large undocumented.

  • Matlab’s UDD mechanism
    UDD (Unified Data Definition?) is used extensively in Matlab as the internal object-oriented mechanism for describing object properties and functionalities. We can use UDD for a wide variety of uses. UDD was described in a series of articles here in early 2011.

Next week I will list the reasons that cause MathWorks to decide whether a particular feature or property should be documented or not.

]]>
https://undocumentedmatlab.com/blog/types-of-undocumented-matlab-aspects/feed 0