My plan is to start the miniseries with a discussion of the built-in showcase examples, followed by a post on the built-in classes that make up the toolstrip building-blocks. Finally, I’ll describe how toolstrips can be added to figures, not just in client/tool groups.

Matlab’s internal showcase examples

I start the discussion with a description of built-in examples for the toolstrip functionality, located in %matlabroot%/toolbox/matlab/toolstrip/+matlab/+ui/+internal/+desktop/. The most important of these are showcaseToolGroup.m and showcaseMPCDesigner.m, both of which use Java-based (Swing) containers and controls. Readers who wish to integrate toolstrips into their app immediately, without waiting for my followup posts in this series, are welcome to dig into the examples’ source-code and replicate it in their programs:

1. showcaseToolGroup

h = matlab.ui.internal.desktop.showcaseToolGroup

2. showcaseMPCDesigner

>> h = matlab.ui.internal.desktop.showcaseMPCDesigner
 
h =
  showcaseMPCDesigner with properties:
 
    ToolGroup: [1×1 matlab.ui.internal.desktop.ToolGroup]
       Dialog: [1×1 toolpack.component.TSTearOffPopup]
      Figure1: [1×1 Figure]
      Figure2: [1×1 Figure]

3. showcaseHTML and showcaseCEF

In addition to these showcase examples, the folder also contains a showcaseHTML.m and showcaseCEF.m files, that are supposed to showcase the toolstrip functionality in JavaScript-based containers (browser webpage and uifigure apps, respectively). Unfortunately, on my system running these classes displays blank, although the toolstrip is indeed created, as seen below (if you find out how to make these classes work, please let me know):

>> h = matlab.ui.internal.desktop.showcaseHTML
building toolstrip hierarchy...
rendering toolstrip...
 
h = 
  Toolstrip with properties:
 
               SelectedTab: [1×1 matlab.ui.internal.toolstrip.Tab]
              DisplayState: 'expanded'
    DisplayStateChangedFcn: @PropertyChangedCallback
                       Tag: 'toolstrip'
 
>> hs = struct(h)
Warning: Calling STRUCT on an object prevents the object from hiding its implementation details and should thus be avoided.
Use DISP or DISPLAY to see the visible public details of an object. See 'help struct' for more information.
(Type "warning off MATLAB:structOnObject" to suppress this warning.)
 
hs = 
  struct with fields:
 
                         SelectedTab: [1×1 matlab.ui.internal.toolstrip.Tab]
                        DisplayState: 'expanded'
              DisplayStateChangedFcn: @PropertyChangedCallback
                 DisplayStatePrivate: 'expanded'
                        QABIdPrivate: '2741bf89'
               QuickAccessBarPrivate: [1×1 matlab.ui.internal.toolstrip.impl.QuickAccessBar]
       DisplayStateChangedFcnPrivate: @PropertyChangedCallback
         SelectedTabChangedListeners: [1×1 event.listener]
                                 Tag: 'toolstrip'
                                Type: 'Toolstrip'
                          TagPrivate: 'toolstrip'
    WidgetPropertyMap_FromMCOSToPeer: [3×1 containers.Map]
    WidgetPropertyMap_FromPeerToMCOS: [3×1 containers.Map]
                              Parent: [0×0 matlab.ui.internal.toolstrip.base.Node]
                            Children: [1×1 matlab.ui.internal.toolstrip.TabGroup]
                             Parent_: []
                           Children_: [1×1 matlab.ui.internal.toolstrip.TabGroup]
                                Peer: [1×1 com.mathworks.peermodel.impl.PeerNodeImpl]
                   PropertySetSource: [1 java.util.HashMap]
                    PeerModelChannel: '/ToolstripShowcaseChannel'
                   PeerEventListener: [1×1 handle.listener]
                 PropertySetListener: [1×1 handle.listener]
 
>> hs.Peer
ans =
PeerNodeImpl{id='4a1e4b08', type='Toolstrip', properties={displayState=expanded, hostId=ToolStripShowcaseDIV, tag=toolstrip, QABId=2741bf89}, parent=878b0e2b, children=[
    PeerNodeImpl{id='5bb9632c', type='TabGroup', properties={QAGroupId=ea9b628c, tag=, selectedTab=f90db10c}, parent=4a1e4b08, children=[
        PeerNodeImpl{id='f90db10c', type='Tab', properties={mnemonic=, tag=tab_buttons, title=BUTTONS}, parent=5bb9632c, children=[
            PeerNodeImpl{id='1ccc9246', type='Section', properties={collapsePriority=0.0, mnemonic=, tag=sec_push, title=PUSH BUTTON}, parent=f90db10c, children=[
                PeerNodeImpl{id='8323f06e', type='Column', properties={horizontalAlignment=left, width=0.0, tag=}, parent=1ccc9246, children=[
                    PeerNodeImpl{id='af368d7b', type='PushButton', properties={textOverride=, descriptionOverride=, mnemonic=, actionId=230d471b, iconOverride=, tag=pushV, iconPathOverride=}, parent=8323f06e, children=[]}]}
                PeerNodeImpl{id='a557a712', type='Column', properties={horizontalAlignment=left, width=0.0, tag=}, parent=1ccc9246, children=[
                    PeerNodeImpl{id='f0d6a9fc', type='EmptyControl', properties={tag=}, parent=a557a712, children=[]}
                    PeerNodeImpl{id='74bc4cd2', type='PushButton', properties={textOverride=, descriptionOverride=, mnemonic=, actionId=12d6a26a, iconOverride=, tag=pushH, iconPathOverride=}, parent=a557a712, children=[]}
                    PeerNodeImpl{id='bcb5a9d0', type='EmptyControl', properties={tag=}, parent=a557a712, children=[]}]}]}
            PeerNodeImpl{id='0e515319', type='Section', properties={collapsePriority=0.0, mnemonic=, tag=sec_dropdown, title=DROP DOWN BUTTON}, parent=f90db10c, children=[
                PeerNodeImpl{id='80482225', type='Column', properties={horizontalAlignment=left, width=0.0, tag=}, parent=0e515319, children=[
                    PeerNodeImpl{id='469f469a', type='DropDownButton', properties={textOverride=, descriptionOverride=, mnemonic=, actionId=c6ca7335, iconOverride=, tag=dropdownV, iconPathOverride=}, parent=80482225, children=[]}]}
                ...

Note: showcaseCEF has been removed in 2018, but is available in older Matlab releases.

Levels of toolstrip encapsulation

Matlab currently has several levels of encapsulation for toolstrip components:

• Top-level m-file classes for showcasing the toolstrip functionality and creating toolstrips in Java-based containers and web-based apps – these are located in %matlabroot%/toolbox/matlab/toolstrip/+matlab/+ui/+internal/+desktop/
• Mid-level m-file classes that contain the toolstrip building blocks (tabs, sections, controls) – these are located in %matlabroot%/toolbox/matlab/toolstrip/+matlab/+ui/+internal/+toolstrip/
• Low-level Java classes that implement the underlying user-interface for Java-based UI – these are located in %matlabroot%/java/jar/toolstrip.jar. I discussed this briefly in a post few years ago.

The top- and mid-level m-file classes are provided with full source code that is quite well-documented internally (as m-file source-code comments). However, note that it is not officially documented or supported (i.e., semi-documented in this blog’s parlance).

The low-level Java classes on the other hand are compiled without available source code – we can inspect these classes (e.g., using uiinspect or checkClass), but we cannot see their original source-code. Luckily, the higher-level m-file classes provide us with plenty of hints and usage examples that we could use to tailor the appearance, functionality and integration of toolstrip components into our app.

Robyn Jackey’s Widgets Toolbox

Users who hesitate to mess around with the built-in toolstrip functionality may find interest in MathWorker Robyn Jackey’s Toolstrip look-alike, which is part of his open-source Widgets Toolbox on the Matlab File Exchange. Unlike other parts of Robyn’s toolbox, which use undocumented functionality, his Toolstrip class seems to use documented components (panels, uicontrols etc.), with just a small reliance on undocumented functionality (matlab.ui.* for example). This has a fair chance to continue working well into future releases, even if Matlab’s built-in toolstrip functionality changes:

Strong caution

Over the years, Matlab’s internal toolstrip interface has changed somewhat, but not dramatically. This could change at any time, since the toolstrip uses deeply undocumented functionality. What I will demonstrate over the next few posts might stop working in R2019a, or in R2025b – nobody really knows, perhaps not even MathWorks at this stage. Something that we do know for a fact is that Matlab is slowly transitioning away from Java-based user interfaces to web-based (HTML/JavaScript/CSS) interfaces, and this could have a drastic effect on the toolstrip functionality/API. It seems reasonable to assume that even if MathWorks would one day open up the toolstrip functionality, this would only be for the new web-based uifigure apps (not legacy Java-based figures), and might well have a different API than the one that I’ll discuss in this miniseries. Still, users could use the unofficial/undocumented information that I present here in their own Java figures today and quite possibly also in near-term upcoming releases.

Despite the many unknowns regarding future supportability/roadmap of the built-in toolstrip API, I believe that my readers are smart enough to decide for themselves whether they want to take the associated risks to improve their Matlab programs today, or wait until a documented API will possibly be provided sometime in the future. The choice is yours, as it always is when using undocumented tips from my blog.

With this warning stated, let’s start having fun with Matlab’s built-in toolstrip!

I basically had two main alternatives:

• I could either create a dedicated string-parsing function that searches for a particular pattern within the XML string, or
• I could use a standard XML-parsing library to create the XML model and then parse its nodes

The first alternative is quite error-prone, since it relies on the exact format of the data in the XML. Since the same data can be represented in multiple equivalent XML ways, making the string-parsing function robust as well as efficient would be challenging. I was lazy expedient, so I chose the second alternative.

Unfortunately, Matlab’s xmlread function only accepts input filenames (of *.xml files), it cannot directly parse XML strings. Yummy!

The obvious and simple solution is to simply write the XML string into a temporary *.xml file, read it with xmlread, and then delete the temp file:

% Store the XML data in a temp *.xml file
filename = [tempname '.xml'];
fid = fopen(filename,'Wt');
fwrite(fid,xmlString);
fclose(fid);
 
% Read the file into an XML model object
xmlTreeObject = xmlread(filename);
 
% Delete the temp file
delete(filename);
 
% Parse the XML model object
...

This works well and we could move on with our short lives. But cases such as this, where a built-in function seems to have a silly limitation, really fire up the investigative reporter in me. I decided to drill into xmlread to discover why it couldn’t parse XML strings directly in memory, without requiring costly file I/O. It turns out that xmlread accepts not just file names as input, but also Java object references (specifically, java.io.File, java.io.InputStream or org.xml.sax.InputSource). In fact, there are quite a few other inputs that we could use, to specify a validation parser etc. – I wrote about this briefly back in 2009 (along with other similar semi-documented input altermatives in xmlwrite and xslt).

In our case, we could simply send xmlread as input a java.io.StringBufferInputStream(xmlString) object (which is an instance of java.io.InputStream) or org.xml.sax.InputSource(java.io.StringReader(xmlString)):

% Read the xml string directly into an XML model object
inputObject = java.io.StringBufferInputStream(xmlString);                % alternative #1
inputObject = org.xml.sax.InputSource(java.io.StringReader(xmlString));  % alternative #2
 
xmlTreeObject = xmlread(inputObject);
 
% Parse the XML model object
...

If we don’t want to depend on undocumented functionality (which might break in some future release, although it has remained unchanged for at least the past decade), and in order to improve performance even further by passing xmlread‘s internal validity checks and processing, we can use xmlread‘s core functionality to parse our XML string directly. We can add a fallback to the standard (fully-documented) functionality, just in case something goes wrong (which is good practice whenever using any undocumented functionality):

try
    % The following avoids the need for file I/O:
    inputObject = java.io.StringBufferInputStream(xmlString);  % or: org.xml.sax.InputSource(java.io.StringReader(xmlString))
    try
        % Parse the input data directly using xmlread's core functionality
        parserFactory = javaMethod('newInstance','javax.xml.parsers.DocumentBuilderFactory');
        p = javaMethod('newDocumentBuilder',parserFactory);
        xmlTreeObject = p.parse(inputObject);
    catch
        % Use xmlread's semi-documented inputObject input feature
        xmlTreeObject = xmlread(inputObject);
    end
catch
    % Fallback to standard xmlread usage, using a temporary XML file:
 
    % Store the XML data in a temp *.xml file
    filename = [tempname '.xml'];
    fid = fopen(filename,'Wt');
    fwrite(fid,xmlString);
    fclose(fid);
 
    % Read the file into an XML model object
    xmlTreeObject = xmlread(filename);
 
    % Delete the temp file
    delete(filename);
end
 
% Parse the XML model object
...

None of the information I’ll present today is really new – it was all there already if you just knew what to search for online. But hopefully today’s post will concentrate all these loose ends in a single place, so it may have some value:

• Using a secure mail server
• Emailing multiple recipients
• Sending text messages
• User configuration panel

Using a secure mail server

All modern mail servers use end-to-end TLS/SSL encryption. The sendmail function needs extra configuration to handle such connections, since it is configured for a non-encrypted connection by default. Here’s the code that does this for gmail, using SMTP server smtp.gmail.com and default port #465 (for other SMTP servers, see here):

setpref('Internet', 'E_mail', from_address);  % sender "from" address, typically same as username, e.g. 'xyz@gmail.com'
setpref('Internet', 'SMTP_Username', username);
setpref('Internet', 'SMTP_Password', password);
setpref('Internet', 'SMTP_Server',   'smtp.gmail.com');
 
props = java.lang.System.getProperties;
props.setProperty('mail.smtp.auth',                'true');  % Note: 'true' as a string, not a logical value!
props.setProperty('mail.smtp.starttls.enable',     'true');  % Note: 'true' as a string, not a logical value!
props.setProperty('mail.smtp.socketFactory.port',  '465');   % Note: '465'  as a string, not a numeric value!
props.setProperty('mail.smtp.socketFactory.class', 'javax.net.ssl.SSLSocketFactory');
 
sendmail(recipient, title, body, attachments);  % e.g., sendmail('recipient@gmail.com', 'Hello world', 'What a nice day!', 'C:\images\sun.jpg')

All this is not enough to enable Matlab to connect to gmail’s SMTP servers. In addition, we need to set the Google account to allow access from “less secure apps” (details, direct link). Without this, Google will not allow Matlab to relay emails. Other mail servers may require similar server-side account configurations to enable Matlab’s access.

Note: This code snippet uses a bit of Java as you can see. Under the hood, all networking code in Matlab relies on Java, and sendmail is no exception. For some reason that I don’t fully understand, MathWorks chose to label the feature of using sendmail with secure mail servers as a feature that relies on “undocumented commands” and is therefore not listed in sendmail‘s documentation. Considering the fact that all modern mail servers are secure, this seems to make sendmail rather useless without the undocumented extension. I assume that TMW are well aware of this, which is the reason they posted a partial documentation in the form of an official tech-support answer. I hope that one day MathWorks will incorporate it into sendmail as optional input args, so that using sendmail with secure servers would become fully documented and officially supported.

Emailing multiple recipients

To specify multiple email recipients, it is not enough to set sendmail‘s recipient input arg to a string with , or ; delimiters. Instead, we need to provide a cell array of individual recipient strings. For example:

sendmail({'recipient1@gmail.com','recipient2@gmail.com'}, 'Hello world', 'What a nice day!')

Note: this feature is actually fully documented in sendmail‘s doc-page, but for some reason I see that some users are not aware of it (to which it might be said: RTFM!).

Sending phone text (SMS) messages

With modern smartphones, text (SMS) messages have become rather outdated, as most users get push notifications of incoming emails. Still, for some users text messages may still be a useful. To send such messages, all we need is to determine our mobile carrier’s email gateway for SMS messages, and send a simple text message to that email address. For example, to send a text message to T-Mobile number 123-456-7890 in the US, simply email the message to 1234567890@tmomail.net (details).

Ke Feng posted a nice Matlab File Exchange utility that wraps this messaging for a wide variety of US carriers.

User configuration panel

Many GUI programs contain configuration panels/tabs/windows. Enabling the user to set up their own email provider is a typical use-case for such a configuration. Naturally, you’d want your config panel not to display plain-text password, nor non-integer port numbers. You’d also want the user to be able to test the email connection.

Here’s a sample implementation for such a panel that I implemented for a recent project – I plan to discuss the implementation details of the password and port (spinner) controls in my next post, so stay tuned:

User configuration of emails in Matlab GUI (click to zoom-in)

I was thrilled to see the answer in the recent Matlab Computational Finance Conference, London. I presented at last year’s conference, so I was excited to review this year’s presentations when I came across a nugget in Kevin Shea’s keynote presentation on new Matlab developments. Note that there were two separate Matlab computational-finance events in 2014 – in London (June 24) and NY (April 9); the interesting piece is from London. Unlike the NY conference, the London conference proceedings do not include a video recording, only the slides (perhaps the video will be added to the proceedings page later, after all it takes some time to process). The last slide of Kevin’s presentation shows a screenshot of a Chrome browser displaying what appears to be a full-fledged Matlab desktop (Workspace, Editor, Command Window, figures) at https://matlab.mathworks.com:

Matlab Online (click to zoom)

The benefits of using Matlab Online are potentially tremendous: excluding some limitations (e.g., limited GUI, run time and supported toolboxes), we could deploy our Matlab programs online with minimal to no additional dev effort.

Information on Matlab Online can be found at http://mathworks.com/products/matlab/online. It even includes detailed dedicated documentation for all the supported functions and features.

For some reason, MathWorks has chosen not to link this new (and in my humble opinion, great) product in its official list of products. Perhaps the reason is that MathWorks do not wish for Matlab Online to cannibalize its main offerings. This might also explain the limitations placed on the online version (some limitations are technical, the rest are business-related). Matlab Online requires a dedicated login access, provided by MathWorks. Unfortunately, it is still not fully released to the public and is only available to users with up to date student licenses.

Still, if you are an important enough client of MathWorks, you might ask for access. Even with its current limitations, Matlab Online may well be useful for your needs. If you press strongly enough, and if your account is large enough, perhaps MathWorks might even enable a dedicated subdomain for you (as in intel.mathworks.com or matlab.apple.com).

Here’s another screenshot, in case you’re not convinced yet:

Matlab Online (click to zoom)

A post by MathWorks on the Answers forum earlier this year lists Matlab Online along with some additional online-computing alternatives for Matlab.

And now for some guesswork/speculation: MathWorks placed a major bet on Java technology in the early 2000’s (actually the late 90’s, since R12 took a few years to develop). Java was indeed hot at that time, but then so were some other technologies over the years. Fortunately for MathWorks, Java proved agile, mature and portable enough to enable mobile and online porting. This could explain the lack of Simulink GUI support, since Simulink GUI is still C++-based to a large extent. It might also explain the extra work done in HG2 in the graphic infrastructure (previously C++-based). After all, the basic MCOS graphic classes were available years ago, so if HG2 was only about a transition from double-value handles to MCOS, and some nice beautifications (anti-aliasing, CVD-aware colors etc.) then HG2 could be released long ago. Working on the underlying engine to make it portable could well explain HG2’s belated arrival.

So what does all this mean about Matlab’s future? Well, it appears to me that MathWorks’ apparent move towards SaaS (software as a service) and cloud-based computing is slightly belated, but quite evidently follows a general industry trend. In my eyes it heralds a move by MathWorks from desktop to online services, perhaps even to pay-per-use computing (as by Techila for example). The Matlab desktop will still be MathWorks’ bread and butter for many years to come. But the ability of Matlab programs to work either locally (on a Matlab client, either thin or thick) or online would be an enormous productivity benefit, in essence being a “killer feature” over Matlab’s competitors.

Technically, Matlab’s online integration could enable closer integration of online content in Matlab programs (esp. GUI). I’ve already shown how active web content can be displayed in Matlab GUI, but a closer integration could mean this might all become fully documented and integrated (recall Windows’ failed Active Desktop on one hand, but smart-phones’ enormously-successful active widgets on the other). It could also enable closer integration with online graphing services such as Plotly.

I for one, can’t wait to see this dream being realized. It’s not too far down the road, it would seem. I just hope that licensing and cost considerations won’t keep us from using it. MathWork’s recent Home License scheme seems to indicate that this is well understood by MathWorks, so I am highly optimistic

Then again, all this might be pure baseless speculation. Time will tell.

Today’s post is about the well-known fwrite function, which is used to write binary data to file. In many cases, using fwrite provides the fastest alternative to saving data files (save(…,’-v6′) coming a close second). This function is in fact so low-level, and is used so often, that some readers may be surprised that its default speed can be improved. Today’s article applies equally to the fprintf function, which is used to save data in text format.

Apparently, there are things to be learned even with such standard low-level functions; there’s a deep moral here I guess.

Flushing and buffering

Unlike C/C++’s implementation, Matlab’s fprintf and fwrite automatically flush the output buffer whenever they are called, even when '\n' is not present in the output stream. This is not mentioned outright in the main documentation, but is stated loud and clear in the official technical support solution 1-PV371.

The only exception to this rule is when the file was fopen‘ed with the 'W' or 'A' specifiers (which for some inexplicable reason is NOT mentioned in the technical solution!), or when outputting to the MATLAB’s Command Window (more precisely, to STDOUT (fid=1) and STDERR (fid=2)). Writing data without buffering in this manner severely degrades I/O performance:

data = randi(250,1e6,1);  % 1M integer values between 1-250
 
% Standard unbuffered writing - slow
fid = fopen('test.dat', 'wb');
tic, for idx = 1:length(data), fwrite(fid,data(idx)); end, toc
fclose(fid);
  => Elapsed time is 14.006194 seconds.
 
% Buffered writing – x4 faster
fid = fopen('test.dat', 'Wb');
tic, for idx = 1:length(data), fwrite(fid,data(idx)); end, toc
fclose(fid);
  => Elapsed time is 3.471557 seconds.

If I were in a generous mood, I could say that we could infer this information from fopen‘s doc page, where it mentions using the 'W' and 'A' permission specifiers to prevent automatic flushing, although it qualifies this with the very misleading statement that these specifiers are “Used with tape drives“. So first of all, who ever uses tape drives with Matlab nowadays?! Secondly, these specifiers are very useful for regular buffered I/O on standard disks and other I/O interfaces. I really think this was a poor choice of words. At the very least some extra clarification about these specifiers could be added.

It was also (IMHO) a poor design choice by MathWorks in the first place to break consistency with the C/C++ implementation for 'w' and associate the functionality to 'W' (and similarly, 'a' vs. 'A'). C’s fopen was in widespread usage for a decade before Matlab was invented, so there is really no excuse, certainly when Matlab’s fopen was so clearly modeled after the C implementation. It would have been more reasonable (again – IMHO) to preserve consistency of 'w' and 'a' for a default of buffered I/O (which is faster!), while providing the non-buffered functionality in 'W' and 'A'.

The vast majority of Matlab users fopen their files using 'w' and not 'W'. Even Matlab’s own documentation always uses 'w' and not 'W'. So coupled with the poorly-worded qualification about the tape drives, and the unintuitive inconsistency with C’s implementation, Matlab users could well be excused for not taking advantage of this feature.

Chunking I/O

The idea of buffering, and the reason behind the speedup above, is that I/O is faster when writing full pages (typically 4KB, but this changes on different platforms) and when bunched together to remove the disk access time between adjacent writes. This idea can be extended by preparing the entire file data in memory, and then using a single fwrite to write everything at once:

fid = fopen('test.dat', 'wb');
tic, fwrite(fid,data); toc
fclose(fid);
  => Elapsed time is 0.014025 seconds.

In fact, assembling the entire data in memory, within a long numeric or char array, and then using a single fwrite to save this array to file, is almost as fast as we can expect to get (example). Further improvement lies in optimizing the array assembly (which is CPU and memory-intensive) rather than the I/O itself.

In this example, the I/O was so fast (14mS) that it makes sense to write everything at once. But for enormously large data files and slower disks (I use a local SSD; network hard disks are way slower), writing the entire file’s data in this manner might take long minutes. In such cases, it is advisable to deliberately break up the data into smaller chunks, and fwrite them separately in a loop, all the time providing feedback to the user about the I/O’s progress. This could help improve the operation’s perceived performance. Here’s a bare-bones example:

h = waitbar(0, 'Saving data...', 'Name','Saving data...');
cN = 100;  % number of steps/chunks
 
% Divide the data into chunks (last chunk is smaller than the rest)
dN = length(data);
dataIdx = [1 : round(dN/cN) : dN, dN+1];  % cN+1 chunk location indexes
 
% Save the data
fid = fopen('test.dat', 'Wb');
for chunkIdx = 0 : cN-1
   % Update the progress bar
   fraction = chunkIdx/cN;
   msg = sprintf('Saving data... (%d%% done)', round(100*fraction));
   waitbar(fraction, h, msg);
 
   % Save the next data chunk
   chunkData = data(dataIdx(chunkIdx+1) : dataIdx(chunkIdx+2)-1);
   fwrite(fid,chunkData);
end
 
fclose(fid);
close(h);

Of course, rather than using a plain-ol’ waitbar window, we could integrate a progress bar directly into our GUI. Using my statusbar utility is one way to do it, but there are of course many other possible ways to dynamically present progress:

Dynamically updating progress using the statusbar utility

Note: the entire article above applies equally well to fprintf in addition to fwrite. Storing and loading data in binary format (using fwrite/fread) is often faster than text format (using fprintf/fscanf/textscan), so we should generally use text format only if the file needs to be human-readable for any reason.

Do you know of any other trick to store data efficiently? If so, please share it in a comment.

Next week: some surprising performance aspects of Matlab’s save function.

The problem: “Matlab crashes” – now go figure…

A client of one of my Matlab programs recently complained that Matlab crashes after several hours of extensive use of the program. The problem looked like something that is memory related (messages such as Matlab’s out-of-memory error or Java’s heap-space error). Apparently this happens even on 64-bit systems having lots of memory, where memory should never be a problem.

Well, we know that this is only in theory, but in practice Matlab’s internal memory management has problems that occasionally lead to such crashes. This is one of the reasons, by the way, that recent Matlab releases have added the preference option of increasing the default Java heap space (the previous way to do this was a bit complex). Still, even with a high Java heap space setting and lots of RAM, Matlab crashed after using my program for several hours.

Not pleasant at all, even a bit of an embarrassment for me. I’m used to crashing Matlab, but only as a result of my playing around with the internals – I would hate it to happen to my clients.

Finding the leak

While we can do little with Matlab’s internal memory manager, I started searching for the exact location of the memory leak and then to find a way to overcome it. I’ll save readers the description about the grueling task of finding out exactly where the memory leak occurred in a program that has thousands of lines of code and where events get fired asynchronously on a constant basis. Matlab Profiler’s undocumented memory profiling option helped me quite a bit, as well as lots of intuition and trial-and-error. Detecting memory leaks is never easy, and I consider myself somewhat lucky this time to have both detected the leak source and a workaround.

It turned out that the leakage happens in a callback that gets invoked multiple times per second by a Java object (see related articles here and here). Each time the Matlab callback function is invoked, it reads the event information from the supplied Java event-data (the callback’s second input parameter). Apparently, about 1KB of memory gets leaked whenever this event-data is being read. This may appear a very small leak, but multiply this by some 50-100K callback invocations per hour and you get a leakage of 50-100MB/hour. Not a small leak at all; more of a flood you could say…

Using get()

The leakage culprit turned out to be the following code snippet:

% 160 uSecs per call, with memory leak
eventData  = get(hEventData,{'EventName','ParamNames','EventData'});
eventName  = eventData{1};
paramNames = eventData{2};
paramData  = eventData{3}.cell;

In this innocent-looking code, hEventData is a Java object that contains the EventName, ParamNames, EventData properties: EventName is a Java String, that is automatically converted by Matlab’s get() function into a Matlab string (char array); ParamNames is a Java array of Strings, that gets automatically converted into a Matlab cell-array of string; and EventData is a Java array of Objects that needs to be converted into a Matlab cell array using the built-in cell function, as described in one of my recent articles.

The code is indeed innocent, works really well and is actually extremely fast: each invocation of this code segment takes less than 0.2 millisecs. Unfortunately, because of the memory leak I needed to find a better alternative.

Using handle()

The first idea was to use the built-in handle() function, under the assumption that it would solve the memory leak, as reported here. In fact, MathWorks specifically advises to use handle() rather than to work with “naked” Java objects, when setting Java object callbacks. The official documentation of the set function says:

Do not use the set function on Java objects as it will cause a memory leak.

It stands to reason then that a similar memory leak happens with get and that a similar use of handle would solve this problem:

% 300 uSecs per call, with memory leak
s = get(handle(hEventData));
eventName  = s.EventName;
paramNames = s.ParamNames;
paramData  = cell(s.EventData);

Unfortunately, this variant, although working correctly, still leaks memory, and also performs almost twice as worse than the original version, taking some 0.3 milliseconds to execute per invocation. Looks like this is a dead end.

Using Java accessor methods

The next attempt was to use the Java object’s internal accessor methods for the requested properties. These are public methods of the form getXXX(), isXXX(), setXXX() that enable Matlab to treat XXX as a property by its get and set functions. In our case, we need to use the getter methods, as follows:

% 380 uSecs per call, no memory leak
eventName  = char(hEventData.getEventName);
paramNames = cell(hEventData.getParamNames);
paramData  = cell(hEventData.getEventData);

Here, the method getEventName() returns a Java String, that we convert into a Matlab string using the char function. In our previous two variants, the get function did this conversion for us automatically, but when we use the Java method directly we need to convert the results ourselves. Similarly, when we call getParamNames(), we need to use the cell function to convert the Java String[] array into a Matlab cell array.

This version at last doesn’t leak any memory. Unfortunately, it has an even worse performance: each invocation takes almost 0.4 milliseconds. The difference may seem insignificant. However, recall that this callback gets called dozens of times each second, so the total adds up quickly. It would be nice if there were a faster alternative that does not leak any memory.

Using struct()

Luckily, I found just such an alternative. At 0.24 millisecs per invocation, it is almost as fast as the leaky best-performance original get version. Best of all, it leaks no memory, at least none that I could detect.

The mechanism relies on the little-known fact that public fields of Java objects can be retrieved in Matlab using the built-in struct function. For example:

>> fields = struct(java.awt.Rectangle)
fields = 
             x: 0
             y: 0
         width: 0
        height: 0
      OUT_LEFT: 1
       OUT_TOP: 2
     OUT_RIGHT: 4
    OUT_BOTTOM: 8
 
>> fields = struct(java.awt.Dimension)
fields = 
     width: 0
    height: 0

Note that this useful mechanism is not mentioned in the main documentation page for accessing Java object fields, although it is indeed mentioned in another doc-page – I guess this is a documentation oversight.

In any case, I converted my Java object to use public (rather than private) fields, so that I could use this struct mechanism (Matlab can only access public fields). Yes I know that using private fields is a better programming practice and all that (I’ve programmed OOP for some 15 years…), but sometimes we need to do ugly things in the interest of performance. The latest version now looks like this:

% 240 uSecs per call, no memory leak
s = struct(hEventData);
eventName  = char(s.eventName);
paramNames = cell(s.paramNames);
paramData  = cell(s.eventData);

This solved the memory leakage issue for my client. I felt fortunate that I was not only able to detect Matlab’s memory leak but also find a working workaround without sacrificing performance or functionality.

In this particular case, I was lucky to have full control over my Java object, to be able to convert its fields to become public. Unfortunately, we do not always have similar control over the object that we use, because they were coded by a third party.

By the way, Matlab itself uses this struct mechanism in its code-base. For example, Matlab timers are implemented using Java objects (com.mathworks.timer.TimerTask). The timer callback in Matlab code converts the Java timer event data into a Matlab struct using the struct function, in %matlabroot%/toolbox/matlab/iofun/@timer/timercb.m. The users of the timer callbacks then get passed a simple Matlab EventData struct without ever knowing that the original data came from a Java object.

As an interesting corollary, this same struct mechanism can be used to detect internal properties of Matlab class objects. For example, in the timers again, we can get the underlying timer’s Java object as follows (note the highlighted warning, which I find a bit ironic given the context):

>> timerObj = timerfind
 
   Timer Object: timer-1
 
   Timer Settings
      ExecutionMode: singleShot
             Period: 1
           BusyMode: drop
            Running: off
 
   Callbacks
           TimerFcn: @myTimerFcn
           ErrorFcn: ''
           StartFcn: ''
            StopFcn: ''
 
>> timerFields = struct(timerObj)
Warning: Calling STRUCT on an object prevents the object from hiding its implementation details and should thus be avoided.Use DISP or DISPLAY to see the visible public details of an object. See 'help struct' for more information.(Type "warning off MATLAB:structOnObject" to suppress this warning.)timerFields = 
         ud: {}
    jobject: [1x1 javahandle.com.mathworks.timer.TimerTask]

This is a great question, recently asked by a reader of this blog, so I wanted to expand on it in next week’s article. Before specifying the different reasons, let’s map the nature of undocumented aspects that we find in Matlab.

The term undocumented/unsupported (as opposed to mis-documentated or deprecated) actually refers to quite a large number of different types.
In the following list, the hyperlinks on the list-item titles lead to a list of corresponding articles on this website:

• Undocumented functions
  Matlab functions which appears nowhere in the documentation, are usually built-in functions (do not have an m-file) and can only be inferred from online CSSM posts or usage within one of the Matlab m-functions installed with Matlab (the latter being the usual case). None of these functions is officially supported by MathWorks. MEX is an important source for such functions.

• Semi-documented functions
  Matlab functionality which exists in Matlab m-functions installed with Matlab, but have their main comment separated from the H1 comment line, thereby hiding it from normal view (via Matlab’s help function). The H1 comment line itself is simply a standard warning that this function is not officially supported and may change in some future version. To see the actual help comment, simply edit the function (using Matlab’s edit function or any text editor) and place a comment sign (%) at the empty line between the H1 comment and the actual help section. The entire help section will then onward be visible via the help function:

          function [tree, container] = uitree(varargin)
          % WARNING: This feature is not supported in MATLAB
          % and the API and functionality may change in a future release.
  fix =>  %
          % UITREE creates a uitree component with hierarchical data in a figure window.
          %   UITREE creates an empty uitree object with default property values in
          %   a figure window.
          %...

  These functions are not documented in the full documentation (via Matlab’s doc function, or online). The odd thing is that some of these functions may appear in the category help output (for example, help(‘uitools’)), and in some cases may even have a fully-visible help section (e.g., help(‘setptr’)), but do not have any online help documentation (docsearch(‘setptr’) fails, and doc(‘setptr’) simply displays the readable help text).

  All these functions are officially unsupported by MathWorks, even when having a readable help section. The rule of thumb appears to be that a Matlab function is supported only if it has online documentation. Note, however, that in some rare cases a documentation discrepancy may be due to a MathWorks documentation error, not to unsupportability…

• Helper functions
  Many fully-supported Matlab functions use helper functions that have a specific use in the main (documented) function(s). Often, these helper functions are tightly-coupled to their documented parents and are useless as stand-alone functions. But quite a few of them have quite useful stand-alone use, as I’ve already shown in some past articles.

• Undocumented features and properties
  Features of otherwise-documented Matlab functions, which appear nowhere in the official documentation. You guessed it – these are also not supported by MathWorks… Like undocumented functions, you can only infer such features by the occasional CSSM post or a reference somewhere in Matlab’s m-code.

• Semi-documented features
  Features of otherwise-documented Matlab functions, which are documented in a separate section beneath the main help section, and nowhere else (not in the full doc not the online documentation). If you did not know in advance that these features existed, you could only learn of them by manually looking at Matlab’s m-files (which is what I do in most cases…).

• Undocumented properties
  Many Matlab objects have internal properties, which can be retrieved (via Matlab’s get function) and/or set (via the set function) programmatically. All these properties are fully documented. Many objects also possess hidden properties, some of which are very interesting and useful, but which are undocumented and (oh yes) unsupported. Like undocumented features, they can only be inferred from CSSM or existing code. In a recent article I described my getundoc utility, which lists these undocumented properties of specified objects.

• Internal Matlab classes
  Matlab uses a vast array of specialized Java classes to handle everything from algorithms to GUI. These classes are (of course) undocumented/unsupported. They can often be accessed directly from the Matlab Command Window or user m-files. GUI classes can be inferred by inspecting the figure frame’s Java components, and non-GUI classes can often be inferred from references in Matlab’s m-files.

• Matlab-Java integration
  Matlab’s GUI interface, as well as the Java-to-Matlab interface (JMI) is fully undocumented and unsupported. In addition to JMI, there are other mechanisms to run Matlab code from within Java (namely JMI, COM and DDE) – these are all unsupported and by-and-large undocumented.

• Matlab’s UDD mechanism
  UDD (Unified Data Definition?) is used extensively in Matlab as the internal object-oriented mechanism for describing object properties and functionalities. We can use UDD for a wide variety of uses. UDD was described in a series of articles here in early 2011.

Next week I will list the reasons that cause MathWorks to decide whether a particular feature or property should be documented or not.

To start the discussion, let’s re-create last week’s simple uitree:

% Fruits
fruits = uitreenode('v0', 'Fruits', 'Fruits', [], false);
fruits.add(uitreenode('v0', 'Apple',  'Apple',  [], true));
fruits.add(uitreenode('v0', 'Pear',   'Pear',   [], true));
fruits.add(uitreenode('v0', 'Banana', 'Banana', [], true));
fruits.add(uitreenode('v0', 'Orange', 'Orange', [], true));
 
% Vegetables
veggies = uitreenode('v0', 'Veggies', 'Vegetables', [], false);
veggies.add(uitreenode('v0', 'Potato', 'Potato', [], true));
veggies.add(uitreenode('v0', 'Tomato', 'Tomato', [], true));
veggies.add(uitreenode('v0', 'Carrot', 'Carrot', [], true));
 
% Root node
root = uitreenode('v0', 'Food', 'Food', [], false);
root.add(veggies);
root.add(fruits);
 
% Tree
figure('pos',[300,300,150,150]);
mtree = uitree('v0', 'Root', root);

  

User-created tree

Labels

Node labels (descriptions) can be set using their Name property (the second uitreenode data argument). Note that the horizontal space allotted for displaying the node name will not change until the node is collapsed or expanded. So, if the new name requires more than the existing space, it will be displayed as something like “abc…”, until the node is expanded or collapsed.

Node names share the same HTML support feature as all Java Swing labels. Therefore, we can specify font size/face/color, bold, italic, underline, super-/sub-script etc.:

txt1 = '<html><b><u><i>abra</i></u>';
txt2 = '<font color="red"><sup>kadabra</html>';
node.setName([txt1,txt2]);

HTML-enriched tree nodes

Icons

Tree-node icons can be specified during node creation, as the third data argument to uitreenode, which accepts an icon-path (a string):

iconPath = fullfile(matlabroot,'/toolbox/matlab/icons/greenarrowicon.gif');
node = uitreenode('v0',value,name,iconPath,isLeaf);

Tree node icons can also be created or modified programmatically in run-time, using Matlab’s im2java function. Icons can also be loaded from existing files as follows (real-life programs should check and possibly update jImage’s size to 16 pixels, before setting the node icon, otherwise the icon might get badly cropped; also note the tree-refreshing action):

jImage = java.awt.Toolkit.getDefaultToolkit.createImage(iconPath);
veggies.setIcon(jImage);
veggies.setIcon(im2java(imread(iconPath)));  % an alternative
 
% refresh the veggies node (and all its children)
mtree.reloadNode(veggies);

Setting node icon

Behavior

Nodes can be modified from leaf (non-expandable) to parent behavior (=expandable) by setting their LeafNode property (a related property is AllowsChildren):

set(node,'LeafNode',false);  % =expandable
node.setLeafNode(0);  % an alternative

One of the questions I was asked was how to “disable” a specific tree node. One way would be to modify the tree’s ExpandFcn callback. Another way is to use a combination of HTML rendering and the node’s AllowsChildren property:

label = char(veggies.getName);
veggies.setName(['<html><font color="gray">' label]);
veggies.setAllowsChildren(false);
t.reloadNode(veggies);

Disabled node

Another possible behavioral customization is adding a context-menu to a uitree. We can set node-specific tooltips using similar means.

Answering a reader’s request from last week, tree nodes icons can be used to present checkboxes, radio buttons and other similar node-specific controls. This can actually be done in several ways, that will be explored in next week’s article.

There are numerous other possible customizations – if readers are interested, perhaps I will describe some of them in future articles. If you have any special request, please post a comment below.

Using the full Java XML parsing (JAXP) functionality is admittedly quite intimidating for the uninitiated, but extremely powerful once you understand how all the pieces fit together. Over the years, several interesting utilities were submitted to the Matlab File Exchange that simplify this intimidating post-processing. See for example XML parsing tools, the extremely popular XML Toolbox and xml_io_tools, the recent XML data import and perhaps a dozen other utilities.

Each of Matlab’s main built-in XML-processing functions, xmlread, xmlwrite and xslt has an internal set of undocumented and unsupported functionalities, which builds on their internal Java implementation. As far as I could tell, these unsupported functionalities were supported at least as early as Matlab 7.2 (R2006a), and possibly even on earlier releases. For the benefit of the Java and/or JAXP -speakers out there (it will probably not help any others), I list Matlab’s internal description of these unsupported functionalities, annotated with API hyperlinks. These description (sans the links) can be seen by simply editing the m file, as in (the R2008a variant is described below):

edit xmlread

xmlread

function [parseResult,p] = xmlread(fileName,varargin)

• FILENAME can also be an InputSource, File, or InputStream object
• DOMNODE = XMLREAD(FILENAME,…,P,…) where P is a DocumentBuilder object
• DOMNODE = XMLREAD(FILENAME,…,’-validating’,…) will create a validating parser if one was not provided.
• DOMNODE = XMLREAD(FILENAME,…,ER,…) where ER is an EntityResolver will set the EntityResolver before parsing
• DOMNODE = XMLREAD(FILENAME,…,EH,…) where EH is an ErrorHandler will set the ErrorHandler before parsing
• [DOMNODE,P] = XMLREAD(FILENAME,…) will return a parser suitable for passing back to XMLREAD for future parses.

xmlwrite

function xmlwrite(FILENAME,DOMNODE);
function str = xmlwrite(DOMNODE);
function str = xmlwrite(SOURCE);

• FILENAME can also be a URN, java.io.OutputStream or java.io.Writer object
• SOURCE can also be a SAX InputSource, JAXP Source, InputStream, or Reader object

xslt

function [xResultURI,xProcessor] = xslt(SOURCE,STYLE,DEST,varargin)

• SOURCE can also be a XSLTInputSource
• STYLE can also be a StylesheetRoot or XSLTInputSource
• DEST can also be an XSLTResultTarget. Note that RESULT may be empty in this case since it may not be possible to determine a URL. If STYLE is absent or empty, the function uses the stylesheet named in the xml-stylesheet processing instruction in the SOURCE XML file. (This does not always work)
• There is also an entirely undocumented feature: passing a ‘-tostring’ input argument transforms the inputs into a displayed text segment, rather than into a displayed URI; the transformed text is returned in the xResultURI output argument.

Note: internal comments within the Matlab code seem to indicate that XSLT is SAXON-based, so interested users might use SAXON’s documentation for accessing additional XSLT-related features/capabilities (also see this related thread).

x=0:.01:10;
plot(x, sin(x), 'DisplayName','sin');
legend('-DynamicLegend');
 
hold all;   % add new plot lines on top of previous ones
plot(x, cos(x), 'DisplayName','cos');

We can see how the dynamic legend automatically keeps in sync with its associated axes contents when plot lines are added/removed, even down to the zoom-box lines… The legend automatically uses the plot lines ‘DisplayName’ property where available, or a standard ‘line#’ nametag where not available:

Dynamic legend

DynamicLegend works by attaching a listener to the axes child addition/deletion callback (actually, it works on the scribe object, which is a large topic for several future posts). It is sometimes necessary to selectively disable the dynamic behavior. For example, in my GUI I needed to plot several event lines which looked alike, and so I only wanted the first line to be added to the legend. To temporarily disable the DynamicLegend listener, do the following:

% Try to disable this axes's legend plot-addition listener
legendAxListener = [];
try
   legendListeners = get(gca,'ScribeLegendListeners');
   legendAxListener = legendListeners.childadded;
   set(legendAxListener,'Enable','off');
catch
   % never mind...
end
 
% Update the axes - the legend will not be updated
...
 
% Re-enable the dynamic legend listener
set(legendAxListener,'Enable','on');

Unfortunately, this otherwise-useful DynamicLegend feature throws errors when zooming-in on bar or stairs graphs. This can be replicated by:

figure;
bar(magic(4));  %or: stairs(magic(3),magic(3));
legend('-DynamicLegend');
zoom on;
% Now zoom-in using the mouse to get the errors on the Command Window

The fix: modify %MATLABROOT%\toolbox\matlab\scribe\@scribe\@legend\init.m line #528 as follows:

%old:
str = [str(1:insertindex-1);{newstr};str(insertindex:length(str))];
 
%new:
if size(str,2)>size(str,1)
    str=[str(1:insertindex-1),{newstr},str(insertindex:length(str))];
else
    str=[str(1:insertindex-1);{newstr};str(insertindex:length(str))];
end

The origin of the bug is that bar and stairs generate hggroup plot-children, which saves the legend strings column-wise rather than the expected row-wise. My fix solves this, but I do not presume this solves all possible problems in all scenarios (please report if you find anything else).

p.s. – object visibility in the legend can be controlled on an object-by-object basis using the semi-documented hasbehavior function.

Semi-documented

The DynamicLegend feature is semi-documented. This means that the feature is explained in a comment within the function (which can be seen via the edit(‘legend’) command), that is nonetheless not part of the official help or doc sections. It is an unsupported feature originally intended only for internal Matlab use (which of course doesn’t mean we can’t use it). This feature has existed many releases back (Matlab 7.1 for sure, perhaps earlier), so while it may be discontinued in some future Matlab release, it did have a very long life span… The down side is that it is not supported: I reported the bar/stairs issue back in mid-2007 and so far this has not been addressed (perhaps it will never be). Even my reported workaround in January this year went unanswered (no hard feelings…).

DynamicLegend is a good example of a useful semi-documented feature. Some other examples, which I may cover in future posts, include text(…,‘sc’), drawnow(‘discard’), several options in pan and datacursormode etc. etc.

There are also entire semi-documented functions: many of the uitools (e.g., uitree, uiundo), as well as hgfeval and others.

Have you discovered any useful semi-documented feature or function? If so, then please share your finding in the comments section below.