Unfortunately, I find that while the default interactions set is much more useful than the non-interactive default axes behavior in R2018a and earlier, it could still be improved in two important ways:

1. Performance – Matlab’s builtin Interaction objects are very inefficient. In cases of multiple overlapping axes (which is very common in multi-tab GUIs or cases of various types of axes), instead of processing events for just the top visible axes, they process all the enabled interactions for *all* axes (including non-visible ones!). This is particularly problematic with the default DataTipInteraction – it includes a Linger object whose apparent purpose is to detect when the mouse lingers for enough time on top of a chart object, and displays a data-tip in such cases. Its internal code is both inefficient and processed multiple times (for each of the axes), as can be seen via a profiling session.
2. Usability – In my experience, RegionZoomInteraction (which enables defining a region zoom-box via click-&-drag) is usually much more useful than PanInteraction for most plot types. ZoomInteraction, which is enabled by default only enables zooming-in and -out using the mouse-wheel, which is much less useful and more cumbersome to use than RegionZoomInteraction. The panning functionality can still be accessed interactively with the mouse by dragging the X and Y rulers (ticks) to each side.

For these reasons, I typically use the following function whenever I create new axes, to replace the default sluggish DataTipInteraction and PanInteraction with RegionZoomInteraction:

function axDefaultCreateFcn(hAxes, ~)
    try
        hAxes.Interactions = [zoomInteraction regionZoomInteraction rulerPanInteraction];
        hAxes.Toolbar = [];
    catch
        % ignore - old Matlab release
    end
end

The purpose of these two axes property changes shall become apparent below.

This function can either be called directly (axDefaultCreateFcn(hAxes), or as part of the containing figure’s creation script to ensure than any axes created in this figure has this fix applied:

set(hFig,'defaultAxesCreateFcn',@axDefaultCreateFcn);

Test setup

Figure with default axes toolbar and interactivity

hFig = figure('Pos',[10,10,400,300]);
hTabGroup = uitabgroup(hFig);
for iTab = 1 : 10
    hTab = uitab(hTabGroup, 'title',num2str(iTab));
    hPanel = uipanel(hTab);
    for iPanel = 1 : 10
        hPanel = uipanel(hPanel);
    end
    hAxes(iTab) = axes(hPanel); %see MLint note below
    plot(hAxes(iTab),1:5,'-ob');
end
drawnow

p.s. – there’s a incorrect MLint (Code Analyzer) warning in line 9 about the call to axes(hPanel) being inefficient in a loop. Apparently, MLint incorrectly parses this function call as a request to make the axes in-focus, rather than as a request to create the axes in the specified hPanel parent container. We can safely ignore this warning.

Now let’s create a run-time test script that simulates 2000 mouse movements using java.awt.Robot:

tic
monitorPos = get(0,'MonitorPositions');
y0 = monitorPos(1,4) - 200;
robot = java.awt.Robot;
for iEvent = 1 : 2000
    robot.mouseMove(150, y0+mod(iEvent,100));
    drawnow
end
toc

This takes ~45 seconds to run on my laptop: ~23ms per mouse movement on average, with noticeable “linger” when the mouse pointer is near the plotted data line. Note that this figure is extremely simplistic – In a real-life program, the mouse events processing lag the mouse movements, making the GUI far more sluggish than the same GUI on R2018a or earlier. In fact, in one of my more complex GUIs, the entire GUI and Matlab itself came to a standstill that required killing the Matlab process, just by moving the mouse for several seconds.

Notice that at any time, only a single axes is actually visible in our test setup. The other 9 axes are not visible although their Visible property is 'on'. Despite this, when the mouse moves within the figure, these other axes unnecessarily process the mouse events.

Changing the default interactions

Let’s modify the axes creation script as I mentioned above, by changing the default interactions (note the highlighted code addition):

hFig = figure('Pos',[10,10,400,300]);
hTabGroup = uitabgroup(hFig);
for iTab = 1 : 10
    hTab = uitab(hTabGroup, 'title',num2str(iTab));
    hPanel = uipanel(hTab);
    for iPanel = 1 : 10
        hPanel = uipanel(hPanel);
    end
    hAxes(iTab) = axes(hPanel);
    plot(hAxes(iTab),1:5,'-ob');
    hAxes(iTab).Interactions = [zoomInteraction regionZoomInteraction rulerPanInteraction];end
drawnow

The test script now takes only 12 seconds to run – 4x faster than the default and yet IMHO with better interactivity (using RegionZoomInteraction).

Effects of the axes toolbar

The axes-specific toolbar, another innovation of R2018b, does not just have interactivity aspects, which are by themselves much-contested. A much less discussed aspect of the axes toolbar is that it degrades the overall performance of axes. The reason is that the axes toolbar’s transparency, visibility, background color and contents continuously update whenever the mouse moves within the axes area.

Since we have set up the default interactivity to a more-usable set above, and since we can replace the axes toolbar with figure-level toolbar controls, we can simply delete the axes-level toolbars for even more-improved performance:

hFig = figure('Pos',[10,10,400,300]);
hTabGroup = uitabgroup(hFig);
for iTab = 1 : 10
    hTab = uitab(hTabGroup, 'title',num2str(iTab));
    hPanel = uipanel(hTab);
    for iPanel = 1 : 10
        hPanel = uipanel(hPanel);
    end
    hAxes(iTab) = axes(hPanel);
    plot(hAxes(iTab),1:5,'-ob');
    hAxes(iTab).Interactions = [zoomInteraction regionZoomInteraction rulerPanInteraction];
    hAxes(iTab).Toolbar = [];end
drawnow

This brings the test script’s run-time down to 6 seconds – 7x faster than the default run-time. At ~3ms per mouse event, the GUI is now as performant and snippy as in R2018a, even with the new interactive mouse actions of R2018b active.

Conclusions

MathWorks definitely did not intend for this slow-down aspect, but it is an unfortunate by-product of the choice to auto-enable DataTipInteraction and of its sub-optimal implementation. Perhaps this side-effect was never noticed by MathWorks because the testing scripts probably had only a few axes in a very simple figure – in such a case the performance lags are very small and might have slipped under the radar. But I assume that many real-life complex GUIs will display significant lags in R2018b and newer Matlab releases, compared to R2018a and earlier releases. I assume that such users will be surprised/dismayed to discover that in R2018b their GUI not only interacts differently but also runs slower, although the program code has not changed.

One of the common claims that I often hear against using undocumented Matlab features is that the program might break in some future Matlab release that would not support some of these features. But users certainly do not expect that their programs might break in new Matlab releases when they only use documented features, as in this case. IMHO, this case (and others over the years) demonstrates that using undocumented features is usually not much riskier than using the standard documented features with regards to future compatibility, making the risk/reward ratio more favorable. In fact, of the ~400 posts that I have published in the past decade (this blog is already 10 years old, time flies…), very few tips no longer work in the latest Matlab release. When such forward compatibility issues do arise, whether with fully-documented or undocumented features, we can often find workarounds as I have shown above.

If your Matlab program could use a performance boost, I would be happy to assist making your program faster and more responsive. Don’t hesitate to reach out to me for a consulting quote.

The original code snippet was this:

function test
    try
        if (false) or (true)            disp('Yaba');
        else
            disp('Daba');
        end
    catch
        disp('Doo!');
    end
end

With the following variants for the highlighted line #3:

        if (false) or (true)     % variant #1 (original)
        if (true)  or (false)    % variant #2
        if (true)  or (10< 9.9)  % variant #3
        if  true   or  10< 9.9   % variant #4
        if 10> 9.9 or  10< 9.9   % variant #5

Variant #1: if (false) or (true)

The first thing to note is that or is a function and not an operator, unlike some other programming languages. Since this function immediately follows a condition (true), it is not considered a condition by its own, and is not parsed as a part of the “if” expression.

In other words, as Roger Watt correctly stated, line #3 is actually composed of two separate expressions: if (false) and or(true). The code snippet can be represented in a more readable format as follows, where the executed lines are highlighted:

        if (false)             or (true)
            disp('Yaba');
        else
            disp('Daba');        end

Since the condition (false) is never true, the “if” branch of the condition is never executed; only the “else” branch is executed, displaying ‘Daba’ in the Matlab console. There is no parsing (syntactic) error so the code can run, and no run-time error so the “catch” block is never executed.

Also note that despite the misleading appearance of line #3 in the original code snippet, the condition only contains a single condition (false) and therefore neither short-circuit evaluation nor eager evaluation are relevant (they only come into play in expressions that contain 2+ conditions).

As Rik Wisselink speculated and Michelle Hirsch later confirmed, Matlab supports placing an expression immediately following an “if” statement, on the same line, without needing to separate the statements with a new line or even a comma (although this is suggested by the Editor’s Mlint/Code-Analyzer). As Michelle mentioned, this is mainly to support backward-compatibility with old Matlab code, and is a discouraged programming practice. Over the years Matlab has made a gradual shift from being a very weakly-typed and loose-format language to a more strongly-typed one having stricter syntax. So I would not be surprised if one day in the future Matlab would prevent such same-line conditional statements, and force a new line or comma separator between the condition statement and the conditional branch statement.

Note that the “if” conditional branch never executes, and in fact it is optimized away by the interpreter. Therefore, it does not matter that the “or” function call would have errored, since it is never evaluated.

Variant #2: if (true) or (false)

In this variant, the “if” condition is always true, causing the top conditional branch to execute. This starts with a call to or(false), which throws a run-time error because the or() function expects 2 input arguments and only one is supplied (as Chris Luengo was the first to note). Therefore, execution jumps to the “catch” block and ‘Doo!’ is displayed in the Matlab console.

In a more verbose manner, this is the code (executed lines highlighted):

function test
    try
        if (true)            or (false)            disp('Yaba');
        else
            disp('Daba');
        end
    catch
        disp('Doo!');    end
end

Variant #3: if (true) or (10< 9.9)

This is exactly the same as variant #2, since the condition 10< 9.9 is the same as false. The parentheses around the condition ensure that it is treated as a single logical expression (that evaluates to false) rather than being treated as 2 separate arguments. Since the or() function expects 2 input args, a run-time error will be thrown, resulting in a display of ‘Doo!’ in the Matlab console.

As Will correctly noted, this variant is simply a red herring whose aim was to lead up to the following variant:

Variant #4: if true or 10< 9.9

At first glance, this variant looks exactly the same as variant #3, because parentheses around conditions are not mandatory in Matlab. In fact, if a || b is equivalent to (and in many cases more readable/maintainable than) if (a) || (b). However, remember that “or” is not a logical operator but rather a function call (see variant #1 above). For this reason, the if true or 10< 9.9 statement is equivalent to the following:

        if true
            or 10< 9.9
            ...

Now, you might think that this will cause a run-time error just as before (variant #2), but take a closer look at the input to the or() function call: there are no parentheses and so the Matlab interpreter parses the rest of the line as space-separated command-line inputs to the or() function, which are parsed as strings. Therefore, the statement is in fact interpreted as follows:

        if true
            or('10<', '9.9')
            ...

This is a valid “or” statement that causes no run-time error, since the function receives 2 input arguments that happen to be 3-by-1 character arrays. 3 element-wise or are performed ('1'||'9' and so-on), based on the inputs’ ASCII codes. So, the code is basically the same as:

        if true
            or([49,48,60], [57,46,57])  % =ASCII values of '10<','9.9'
            disp('Yaba');

Which results in the following output in the Matlab console:

ans =
  1×3 logical array
   1   1   1
Yaba

As Will noted, this variant was cunningly crafted so that the 2 input args to “or” would each have exactly the same number of chars, otherwise a run-time error would occur (“Matrix dimensions must agree”, except for the edge case where one of the operands only has a single element). As Marshall noted, Matlab syntax highlighting (in the Matlab console or editor) can aid us understand the parsing, by highlighting the or() inputs in purple color, indicating strings.

Variant #5: if 10> 9.9 or 10< 9.9

This is another variant whose main aim is confusing the readers (sorry about that; well, not really…). This variant is exactly the same as variant #4, because (as noted above) Matlab conditions do not need to be enclosed by parentheses. But whereas 10> 9.9 is a single scalar condition (that evaluates to true), 10< 9.9 are in fact 2 separate 3-character string arguments to the “or” function. The end result is exactly the same as in variant #4.

I hope you enjoyed this little puzzle. Back to serious business in the next post!

USA visit

I will be travelling in the US (Boston, New York, Baltimore) in May/June 2019. Please let me know (altmany at gmail) if you would like to schedule a meeting or onsite visit for consulting/training, or perhaps just to explore the possibility of my professional assistance to your Matlab programming needs.

Without actually running the following code in Matlab, what do you expect its output to be? ‘Yaba’? ‘Daba’? perhaps ‘Doo!’? or maybe it won’t run at all because of a parsing error?

function test
    try
        if (false) or (true)
            disp('Yaba');
        else
            disp('Daba');
        end
    catch
        disp('Doo!');
    end
end

To muddy the waters a bit, do you think that short-circuit evaluation is at work here? or perhaps eager evaluation? or perhaps neither?
Would the results be different if we switched the order of the conditional operands, i.e. (true) or (false) instead of (false) or (true)? if so, how and why?
And does it matter if I used “false” or “10< 9.9” as the “or” conditional?
Are the parentheses around the conditions important? would the results be any different without these parentheses?

In other words, how and why would the results change for the following variants?

        if (false) or (true)     % variant #1
        if (true)  or (false)    % variant #2
        if (true)  or (10< 9.9)  % variant #3
        if  true   or  10< 9.9   % variant #4
        if 10> 9.9 or  10< 9.9   % variant #5

Please post your thoughts in a comment below (expected results and the reason, for the main code snippet above and its variants), and then run the code. You might be surprised at the results, but not less importantly at the reasons. This deceivingly innocuous code snippet leads to interesting insight on Matlab’s parser.

Full marks will go to the first person who posts the correct results and reasoning/interpretation of the variants above (hint: it’s not as trivial as it might look at first glance).

Addendum April 9, 2019: I have now posted my solution/analysis of this puzzle here.

USA visit

I will be travelling in the US (Boston, New York, Baltimore) in May/June 2019. Please let me know (altmany at gmail) if you would like to schedule a meeting or onsite visit for consulting/training, or perhaps just to explore the possibility of my professional assistance to your Matlab programming needs.

Scrollable Matlab GUI panel

Technical description

The basic idea is that in HG2 (Matlab release R2014b onward), uipanels are implemented using standard Java JPanel components. This enables all sorts of interesting customizations. For the purposes of today’s discussion, the important thing to note is that the underlying JPanel object can be re-parented to reside inside a Java JScrollPanel.

So, the idea is to get the Matlab panel’s underlying JPanel object reference, then embed it within a new JScrollPanel object that is placed at the exact same GUI coordinates as the original panel. The essential Matlab code snippet is this:

% Create the Matlab uipanel in the GUI
hPanel = uipanel(...); drawnow
 
% Get the panel's underlying JPanel object reference
jPanel = hPanel.JavaFrame.getGUIDEView.getParent;
 
% Embed the JPanel within a new JScrollPanel object
jScrollPanel = javaObjectEDT(javax.swing.JScrollPane(jPanel));
 
% Remove the JScrollPane border-line
jScrollPanel.setBorder([]);
 
% Place the JScrollPanel in same GUI location as the original panel
pixelpos = getpixelposition(hPanel);
hParent = hPanel.Parent;
[hjScrollPanel, hScrollPanel] = javacomponent(jScrollPanel, pixelpos, hParent);
hScrollPanel.Units = 'norm';
 
% Ensure that the scroll-panel and contained panel have linked visibility
hLink = linkprop([hPanel,hScrollPanel],'Visible');
setappdata(hPanel,'ScrollPanelVisibilityLink',hLink);

Note that this code will only work with panels created in legacy figures, not web-based uifigures (as I mentioned above, a similar solution for uifigures will be presented here next week).

Also note that the new scroll-panel is created with javaObjectEDT, in order to avoid EDT synchronization problems

We also want to link the visibility of the scroll-panel and its contained Matlab panel (hPanel), so that when the panel is set to be non-visible (hPanel.Visible='off'), the entire scroll-panel (scrollbars included) will become invisible, and vice-versa. We can do this by linking the Visible property of the Matlab panel and the scroll-panel container (hScrollPanel) using the linkprop function at the bottom of the script above. Note that we must persist the resulting hLink otherwise it becomes defunct – this is done by using setappdata to store the link in the panel (this way, when the panel is deleted, so does the link).

Resizing the container

The scroll-panel is created with a specific pixelpos location and size, and then its container is made to have normalized units. This ensures that when the container (hParent) grows, the scroll-panel grows as well, and no scrollbars appear (since they are not needed). But when the container shrinks in the X and/or Y direction, corresponding scrollbars appear as-needed. It sounds complicated, but it’s actually very intuitive, as the animated image above shows.

When the container resizes, the displayed viewport image may “jump” sideways. To fix this we can attach a simple repaint callback function to the scroll-panel’s SizeChangedFcn property:

% Attach a repaint callback function
hScrollPanel.SizeChangedFcn = @repaintScrollPane;
 
% Define the callback function:
function repaintScrollPane(hScrollPanel, varargin)
    drawnow
    jScrollPanel = hScrollPanel.JavaPeer;
    offsetX = 0; %or: jScrollPanel.getHorizontalScrollBar.getValue;
    offsetY = 0; %or: jScrollPanel.getVerticalScrollBar.getValue;
    jOffsetPoint = java.awt.Point(offsetX, offsetY);
    jViewport = jScrollPanel.getViewport;
    jViewport.setViewPosition(jOffsetPoint);
    jScrollPanel.repaint;
end

Scrollbars automatically appear as-needed during resize

Viewport position/offset

It would be convenient to have an easy-to-use ViewOffset property in the hScrollPanel object, in order to be able to programmatically query and set the current viewport position (i.e., scrollbars offset). We can add this property via the addprop function:

% Add a new Viewoffset property to hSCrollPanel object
hProp = addprop(hScrollPanel, 'ViewOffset');
hProp.GetMethod = @getViewOffset;  %viewOffset = getViewOffset(hScrollPanel)
hProp.SetMethod = @setViewOffset;  %setViewOffset(hScrollPanel, viewOffset)
 
% Getter method for the dynamic ViewOffset property
function viewOffset = getViewOffset(hScrollPanel, varargin)
    jScrollPanel = hScrollPanel.JavaPeer;
    jPoint = jScrollPanel.getViewport.getViewPosition;
    viewOffset = [jPoint.getX, jPoint.getY];
end
 
% Setter method for the dynamic ViewOffset property
function setViewOffset(hScrollPanel, viewOffset)
    jPoint = java.awt.Point(viewOffset(1), viewOffset(2));
    jScrollPanel = hScrollPanel.JavaPeer;
    jScrollPanel.getViewport.setViewPosition(jPoint);
    jScrollPanel.repaint;
end

This enables us to both query and update the scroll-panel’s view position – [0,0] means top-left corner (i.e., no scroll); [12,34] mean scrolling 12 to the right and 34 down:

>> offset = hScrollPanel.ViewOffset   % or: get(hScrollPanel,'ViewOffset')
offset = 
     0     0
 
>> offset = hScrollPanel.ViewOffset   % or: get(hScrollPanel,'ViewOffset')
offset = 
    12    34
 
% Scroll 30 pixels right, 50 pixels down
>> hScrollPanel.ViewOffset = [30,50];   % or: set(hScrollPanel,'ViewOffset',[30,50])

attachScrollPanelTo utility

I have prepared a utility called attachScrollPanelTo (downloadable from the Matlab File Exchange), which encapsulates all of the above, plus a few other features: inputs validation, Viewport property in the output scroll-pane object, automatic encasing in a new panel for input object that are not already a panel, etc. Feel free to download the utility, use it in your program, and modify the source-code to fit your needs. Here are some usage examples:

attachScrollPanelTo();  % display the demo
 
attachScrollPanelTo(hPanel) % place the specified hPanel in a scroll-panel
 
hScroll = attachScrollPanelTo(hPanel);
hScroll.ViewOffset = [30,50];  % set viewport offset (30px right, 50px down)
set(hScroll, 'ViewOffset',[30,50]);  % equivalent alternative

If you’d like me to add flare to your Matlab GUI, don’t hesitate to contact me on my Consulting page.

Let’s start with a simple Matlab plot with a legend:

hold all; 
hLine1 = plot(1:5); 
hLine2 = plot(2:6); 
hLegend = legend([hLine1,hLine2], 'Location','SouthEast');
hLegend.Title.String = 'MyLegend';

Standard Matlab legend

A pivotal object of the legend group are the LegendEntry items, one per legend row:

>> hLegendEntry = hLegend.EntryContainer.NodeChildren(1);
>> get(hLegendEntry)
              Children: [3×1 Graphics]
                 Color: [0 0 0]
                 Dirty: 0
             FontAngle: 'normal'
              FontName: 'Helvetica'
              FontSize: 8
            FontWeight: 'normal'
      HandleVisibility: 'on'
               HitTest: 'on'
                  Icon: [1×1 LegendIcon]
                 Index: 0
           Interpreter: 'tex'
                 Label: [1×1 Text]
            LayoutInfo: [1×1 matlab.graphics.illustration.legend.ItemLayoutInfo]
                Legend: [1×1 Legend]
              Listener: [1×1 event.listener]
                Object: [1×1 Line]
               Overlay: [1×1 TriangleStrip]
          OverlayAlpha: 0.65
                Parent: [1×1 Group]
           PeerVisible: 'on'
         PickableParts: 'visible'
              Selected: 'off'
    SelectionHighlight: 'on'
               Visible: 'on'
       VisibleListener: [1×1 event.proplistener]

Each LegendEntry contains a back-reference to the original graphics object. In my example above, hLegend.EntryContainer.NodeChildren(2).Object == hLine1, and hLegend.EntryContainer.NodeChildren(2).Object == hLine1. Note how the default legend entries order is the reverse of the order of creation of the original graphics objects. Naturally, we can modify this order by creating the legend py passing it an array of handles that is ordered differently (see the documentation of the legend function).

To get all the original graphic objects together, in a single array, we could use one of two mechanisms (note the different order of the returned objects):

% Alternative #1
>> [hLegend.EntryContainer.NodeChildren.Object]'
ans = 
  2×1 Line array:
 
  Line    (data2)
  Line    (data1)
 
% Alternative #2
>> hLegend.PlotChildren
ans = 
  2×1 Line array:
 
  Line    (data1)
  Line    (data2)

For some reason, accessing the displayed graphic line in LegendEntry‘s Icon is not simple. For example, the LineStrip object that corresponds to hLine2 can be gotten via:

hLegendEntry = hLegend.EntryContainer.NodeChildren(1);
hLegendIconLine = hLegendEntry.Icon.Transform.Children.Children;  % a LineStrip object in our example

I assume that this was done to enable non-standard icons for patches and other complex objects (in which case the displayed icon would not necessarily be a LineStrip object). In the case of a line with markers, for example, hLegendIconLine would be an array of 2 objects: a LineStrip object and a separate Marker object. Still, I think that a direct reference in a hLegend.EntryContainer.NodeChildren(1).Icon property would have helped in 99% of all cases, so that we wouldn’t need to pass through the Transform object.

Anyway, once we have this object reference(s), we can modify its/their properties. In the case of a LineStrip this includes LineStyle, LineWidth, ColorData (4×1 uint8), and VertexData (which controls position/length):

>> get(hLegendIconLine(end))  % LineStrip
          AlignVertexCenters: 'on'
             AmbientStrength: 0.3
                ColorBinding: 'object'
                   ColorData: [4×1 uint8]
                   ColorType: 'truecolor'
             DiffuseStrength: 0.6
            HandleVisibility: 'on'
                     HitTest: 'off'
                       Layer: 'middle'
                     LineCap: 'none'
                    LineJoin: 'round'
                   LineStyle: 'solid'
                   LineWidth: 0.5
               NormalBinding: 'none'
                  NormalData: []
                      Parent: [1×1 Group]
               PickableParts: 'visible'
    SpecularColorReflectance: 1
            SpecularExponent: 10
            SpecularStrength: 0.9
                   StripData: []
                     Texture: [0×0 GraphicsPlaceholder]
                  VertexData: [3×2 single]
               VertexIndices: []
                     Visible: 'on'
       WideLineRenderingHint: 'software'

and in the presense of markers:

>> get(hLegendIconLine(1))  % Marker
    EdgeColorBinding: 'object'
       EdgeColorData: [4×1 uint8]
       EdgeColorType: 'truecolor'
    FaceColorBinding: 'object'
       FaceColorData: []
       FaceColorType: 'truecolor'
    HandleVisibility: 'on'
             HitTest: 'off'
               Layer: 'middle'
           LineWidth: 0.5
              Parent: [1×1 Group]
       PickableParts: 'visible'
                Size: 6
         SizeBinding: 'object'
               Style: 'circle'
          VertexData: [3×1 single]
       VertexIndices: []
             Visible: 'on'

An additional undocumented legend property that is of interest is ItemTokenSize. This is a 2-element numeric array specifying the minimal size of the legend entries’ icon and label. By default hLegend.ItemTokenSize == [30,18], but we can either expand or shrink the icons/labels by setting different values. For example:

hLegend.ItemTokenSize == [10,1];  % shrink legend icons and labels

Note that regardless of the amount that we specify, the actual amount that will be used will be such that all legend labels appear.
Fun: try playing with negative values for the icon and the label and see what happens

Have you come across any other interesting undocumented aspect of Matlab legends? If so, then please share it in a comment below.

Slider alternatives in Matlab GUI

The controllib.widget.Slider component is a class in Matlab’s internal controllib package (last week I discussed a different utility function in this package, controllib.internal.util.hString2Char).

controllib.widget.Slider accepts 3 input arguments: containing figure handle, position in pixels, and data values. For example:

>> hSlider = controllib.widget.Slider(gcf, [10,10,100,50], 5:25)
hSlider = 
  Slider with properties:
 
        Data: [6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25]
       Index: 11
       Value: 15
    FontSize: 8
    Position: [10 10 100 50]

This creates an invisible axes at the specified figure location and displays graphic axes objects that provide the appearance of the slider. When you move the slider’s knob, or click its centerline or arrows (“Steppers”), the slider’s value changes accordingly.

You can attach a callback function to the slider as follows:

myCallback = @(h,e) disp(h.Value);  % as an example
addlistener(hSlider, 'ValueChanged', myCallback);

Note that controllib.widget.Slider is based on pure-Matlab code and fully-supported functionality. The Matlab source code is available (%matlabroot%/toolbox/shared/controllib/graphics/+controllib/+widget/Slider.m) and quite readable. So while it does not actually work with the new web-based uifigures, is should be relatively easy to adapt the code so that this component could be displayed in such uifigures.

Below is a script to recreate the screenshot above. Note the two alternative mechanisms for setting properties (Java setter-method notation, and HG set notation):

hFig = figure('Color','w');
 
% 1. controllib.widget.Slider
hSlider1 = controllib.widget.Slider(hFig, [10,10,100,50], 1:20);
hSlider1.Value = 12;
 
% 2. uicontrol
hSlider2 = uicontrol('style','slider', 'units','pixels', 'pos',[10,80,100,20], 'Min',0', 'Max',20, 'Value',12);
 
% 3. JScrollBar
jSlider3 = javaObjectEDT(javax.swing.JScrollBar);
jSlider3.setOrientation(jSlider3.HORIZONTAL);  % Java setter-method notation
set(jSlider3, 'VisibleAmount',1, 'Minimum',0, 'Maximum',20, 'Value',12);  % HG set notation
[hSlider3, hContainer3] = javacomponent(jSlider3, [10,130,100,20], hFig);
 
% 4. JSlider #1
jSlider4 = javaObjectEDT(javax.swing.JSlider(0,20,12))
jSlider4.setBackground(java.awt.Color.white);  % Java setter-method notation
set(jSlider4, 'MinorTickSpacing',1, 'MajorTickSpacing',4, 'SnapToTicks',true, 'PaintLabels',true);  % HG set notation
[hSlider4, hContainer4] = javacomponent(jSlider4, [10,180,100,30], hFig);
 
% 5. JSlider #2
jSlider5 = javaObjectEDT(javax.swing.JSlider(0,20,12))
jSlider5.setBackground(java.awt.Color.white);  % Java setter-method notation
jSlider5.setPaintTicks(true);
set(jSlider5, 'MinorTickSpacing',1, 'MajorTickSpacing',4, 'SnapToTicks',true, 'PaintLabels',true);  % HG set notation
[hSlider5, hContainer5] = javacomponent(jSlider5, [10,230,100,40], hFig);

For additional details regarding the other slider alternatives, please refer to my earlier post on this subject.

Have you ever used another interesting utility or class in Matlab’s builtin packages? If so, please tell us about it in a comment below.

hImage = imagesc(imageData); colormap(gray); colorbar;

Raw image with default Matlab CLim

Rescaling the axes color-limits

As you can see, this image is pretty useless for human-eye analysis. The reason is that while all of the interesting features in the central portion of the image have a z-value of ~-6, the few pixels in the margins that have a z-value of 350+ screw up the color limits and ruin the perceptual resolution (image contrast). We could of course start to guess (or histogram the z-values) to get the interesting color-limit range, and then manually set hAxes.CLim to get a much more usable image:

hAxes = hImage.Parent; hAxes.CLim = [-7.5,-6];

Raw image with a custom CLim

Auto-scaling the axes color-limits

Since the z-values range and distribution changes between different images, it would be better to automatically scale the axes color-limits based on an analysis of the image. A very simple technique for doing this is to take the 5%,95% or 10%,90% percentiles of the data, clamping all outlier data pixels to the extreme colors. If you have the Stats Toolbox you can use the prctile function for this, but if not (or even if you do), here’s a very fast alternative that automatically scales the axes color limits based on the specified threshold (a fraction between 0-0.49):

% Rescale axes CLim based on displayed image portion's CData
function rescaleAxesClim(hImage, threshold)
    % Get the displayed image portion's CData
    CData = hImage.CData;
    hAxes = hImage.Parent;
    XLim = fix(hAxes.XLim);
    YLim = fix(hAxes.YLim);
    rows = min(max(min(YLim):max(YLim),1),size(CData,1)); % visible portion
    cols = min(max(min(XLim):max(XLim),1),size(CData,2)); % visible portion
    CData = CData(unique(rows),unique(cols));
    CData = CData(:);  % it's easier to work with a 1d array
 
    % Find the CLims from this displayed portion's CData
    CData = sort(CData(~isnan(CData)));  % or use the Stat Toolbox's prctile()
    thresholdVals = [threshold, 1-threshold];
    thresholdIdxs = fix(numel(CData) .* thresholdVals);
    CLim = CData(thresholdIdxs);
 
    % Update the axes
    hAxes.CLim = CLim;
end

Note that a threshold of 0 uses the full color range, resulting in no CLim rescaling at all. At the other extreme, a threshold approaching 0.5 reduces the color-range to a single value, basically reducing the image to an unusable B/W (rather than grayscale) image. Different images might require different thresholds for optimal contrast. I believe that a good starting point for the threshold is a value of 0.10, which corresponds to the 10-90% range of CData values.

Dynamic auto-scaling of axes color-limits

This is very nice for the initial image display, but if we zoom-in, or pan a sub-image around, or update the image in some way, we would need to repeat calling this rescaleAxesClim() function every time the displayed image portion changes, otherwise we might still get unusable images. For example, if we zoom into the image above, we will see that the color-limits that were useful for the full image are much less useful on the local sub-image scale. The first (left) image uses the static custom color limits [-7.5,-6] above (i.e., simply zooming-in on that image, without modifying CLim again); the second (right) image is the result of repeating the call to rescaleAxesClim(), which improves the image contrast:

Zoomed-in image with a custom static CLim

Zoomed-in image with a re-applied custom CLim

We could in theory attach the rescaleAxesClim() function as a callback to the zoom and pan functions (that provide such callback hooks). However, we would still need to remember to manually call this function whenever we modify the image or its containing axes programmatically.

A much simpler way is to attach our rescaleAxesClim() function as a callback to the image’s undocumented MarkedClean event:

% Instrument image: add a listener callback to rescale upon any image update
addlistener(hImage, 'MarkedClean', @(h,e)rescaleAxesClim(hImage,threshold));

In order to avoid callback recursion (potentially caused by modifying the axes CLim within the callback), we need to add a bit of code to the callback that prevents recursion/reentrancy (details). Here’s one simple way to do this:

% Rescale axes CLim based on displayed image portion's CData
function rescaleAxesClim(hImage, threshold)
    % Check for callback reentrancy
    inCallback = getappdata(hImage, 'inCallback');
    if ~isempty(inCallback), return, end
    try
        setappdata(hImage, 'inCallback',1);  % prevent reentrancy
 
        % Get the displayed image portion's CData
        ...  (copied from above)
 
        % Update the axes
        hAx.CLim = CLim;
        drawnow; pause(0.001);  % finish all graphic updates before proceeding
    catch
    end
    setappdata(hImage, 'inCallback',[]);  % reenable this callback
end

The result of this dynamic automatic color-scaling can be seen below:

Zoomed-in image with dynamic CLim

autoScaleImageCLim utility

I have created a small utility called autoScaleImageCLim, which includes all the above, and automatically sets the specified input image(s) to use auto color scaling. Feel free to download this utility from the Matlab File Exchange. Here are a few usage examples:

autoScaleImageCLim()           % auto-scale the current axes' image
autoScaleImageCLim(hImage,5)   % auto-scale image using 5%-95% CData limits
autoScaleImageCLim(hImage,.07) % auto-scale image using 7%-93% CData limits

(note that the hImage input parameter can be an array of image handles)

Hopefully one day the so-useful MarkedClean event will become a documented and fully-supported event for all HG objects in Matlab, so that we won’t need to worry that it might not be supported by some future Matlab release…

We start by binning the raw data into pre-selected bins. This can easily be done using the builtin histc (deprecated) or histcounts functions. We can then use the bar function to plot the results:

[binCounts, binEdges] = histcounts(data);
hBars = bar(hAxes, binEdges(1:end-1), binCounts);

Basic histogram bar plot

Let’s improve the appearance: In my specific case, the data was financial return (percentage) values, so let’s modify the x-label format accordingly and display a title. To make the labels and title more legible, we decrease the axes FontSize to 8 and remove the axes box:

hAxes = hBar.Parent;
xtickformat(hAxes, '%g%%');
title(hAxes, 'Distribution of total returns (monthly %)');
set(hAxes, 'FontSize',8, 'Box','off')

Improved histogram bar plot

So far nothing undocumented. Note that the xtickformat/ytickformat functions were only introduced in R2016b – for earlier Matlab releases see this post (which does rely on undocumented aspects).

Now, let’s use a couple of undocumented properties: to remove the excess white-space margin around the axes we’ll set the axes’ LooseInset property, and to remove the annoying white space between the tick labels and the X-axis we’ll set the XRuler‘s TickLabelGapOffset property to -2 (default: +2):

set(hAxes, 'LooseInset',[0,0,0,0]);    % default = [.13,.11,.095,.075]
hAxes.XRuler.TickLabelGapOffset = -2;  % default = +2

Even better histogram bar plot

Note that I used the undocumented axes XRuler property instead of the axes’ documented XAxis property, because XAxis is only available since R2015b, whereas XRuler (which points to the exact same object as XAxis) exists ever since R2014b, and so is better from a backward-compatibility standpoint. In either case, the ruler’s TickLabelGapOffset property is undocumented. Note that the ruler also contains another associated and undocumented TickLabelGapMultiplier property (default: 0.2), which I have not modified in this case.

Now let’s take a look at the bin labels: The problem with the bar plot above is that it’s not intuitively clear whether the bin for “5%”, for example, includes data between 4.5-5.5 or between 5.0-6.0 (which is the correct answer). It would be nicer if the labels were matched to the actual bin edges. There are 3 basic ways to fix this:

1. We could modify the bar plot axes tick values and labels, in essence “cheating” by moving the tick labels half a bin leftward of their tick values (don’t forget to add the extra tick label on the right):

   hAxes.XTick(end+1) = hAxes.XTick(end) + 1;  % extra tick label on the right
   labels = hAxes.XTickLabels;       % preserve tick labels for later use below
   hAxes.XTick = hAxes.XTick - 0.5;  % move tick labels 1/2 bin leftward
   hAxes.XTickLabel = labels;        % restore pre-saved tick labels
   hAxes.XLim = hAxes.XLim - 0.5;    % ...and align the XLim

   

   Improved labels

2. We could use the bar function’s optional 'histc' flag, in order to display the bars in histogram mode. The problem in histogram mode is that while the labels are now placed correctly, the bars touch each other – I personally find distinct bars that are separated by a small gap easier to understand.

   hBars = bar(..., 'histc');
   % [snip] - same customizations to hAxes as done above

   

   Basic histogram plot

   With the original bar chart we could use the built-in BarWidth to set the bar/gap width (default: 0.8 meaning a 10% gap on either side of the bar). Unfortunately, calling bar with 'hist' or 'histc' (i.e. histogram mode) results in a Patch (not Bar) object, and patches do not have a BarWidth property. However, we can modify the resulting patch vertices in order to achieve the same effect:

   % Modify the patch vertices (5 vertices per bar, row-based)
   hBars.Vertices(:,1) = hBars.Vertices(:,1) + 0.1;
   hBars.Vertices(4:5:end,1) = hBars.Vertices(4:5:end,1) - 0.2;
   hBars.Vertices(5:5:end,1) = hBars.Vertices(5:5:end,1) - 0.2;
    
   % Align the bars & labels at the center of the axes
   hAxes.XLim = hAxes.XLim + 0.5;

   This now looks the same as option #1 above, except that the top-level handle is a Patch (not Bar) object. For various additional customizations, either Patch or Bar might be preferable, so you have a choice.

   

   Improved histogram plot

3. Lastly, we could have used the builtin histogram function instead of bar. This function also displays a plot with touching bars, as above, using Quadrilateral objects (a close relative of Patch). The solution here is very similar to option #2 above, but we need to dig a bit harder to modify the patch faces, since their vertices is not exposed as a public property of the Histogram object. To modify the vertices, we first get the private Face property (explanation), and then modify its vertices, keeping in mind that in this specific case the bars have 4 vertices per bar and a different vertices matrix orientation:

   hBars = histogram(data, 'FaceAlpha',1.0, 'EdgeColor','none');
   % [snip] - same customizations to hAxes as done above
    
   % Get access to *ALL* the object's properties
   oldWarn = warning('off','MATLAB:structOnObject');
   warning off MATLAB:hg:EraseModeIgnored
   hBarsStruct = struct(hBars);
   warning(oldWarn);
    
   % Modify the patch vertices (4 vertices per bar, column-based)
   drawnow;  % this is important, won't work without this!
   hFace = hBarsStruct.Face;  % a Quadrilateral object (matlab.graphics.primitive.world.Quadrilateral)
   hFace.VertexData(1,:) = hFace.VertexData(1,:) + 0.1;
   hFace.VertexData(1,3:4:end) = hFace.VertexData(1,3:4:end) - 0.2;
   hFace.VertexData(1,4:4:end) = hFace.VertexData(1,4:4:end) - 0.2;

In conclusion, there are many different ways to improve the appearance of charts in Matlab. Even if at first glance it may seem that some visualization function does not have the requested customization property or feature, a little digging will often find either a relevant undocumented property, or an internal object whose properties could be modified. If you need assistance with customizing your charts for improved functionality and appearance, then consider contacting me for a consulting session.

tracer4m utility uses conditional breakpoints that evaluate but never become live

Matlab breakpoints are documented and supported functionality, and yet their documented use is typically focused at interactive programming in the Matlab editor, or as interactive commands that are entered in the Matlab console using the set of db* functions: dbstop, dbclear, dbstatus, dbstack etc. However, nothing prevents us from using these db* functions directly within our code.

For example, the dbstack function can help us diagnose the calling tree for the current function, in order to do action A if one of the calling ancestors was FunctionX, or to do action B otherwise (for example, to avoid nested recursions).

Similarly, we could add a programmatic call to dbstop in order to stop at a certain code location downstream (for debugging), if a certain condition happens upstream.

Per extended this idea very cleverly in tracer4m: conditional breakpoints evaluate a string in run-time: if the result is true (non-zero) then the code run is stopped at that location, but if it’s false (or zero) then the code run continues normally. To instrument calls to specific functions, Per created a function tracer() that logs the function call (using dbstack) and always returns the value false. He then dynamically created a string that contains a call to this new function and used the dbstop function to create a conditional breakpoint based on this function, something similar to this:

dbstop('in', filename, 'at', location, 'if', 'tracer()');

We can use this same technique for other purposes. For example, if we want to do some action (not necessarily log – perhaps do something else) when a certain code point is reached. The benefit here is that we don’t need to modify the code at all – we’re adding ad-hoc code pieces using the conditional breakpoint mechanism without affecting the source code. This is particularly useful when we do not have access to the source code (such as when it’s compiled or write-protected). All you need to do is to ensure that the instrumentation function always returns false so that the breakpoint does not become live and for code execution to continue normally.

The tracer4m utility is quite sophisticated in the sense that it uses mlint and smart regexp to parse the code and know which functions/methods occur on which line numbers and have which type (more details). In this sense, Per used undocumented functionality. I’m certain that Jiro was not aware of the dependency on undocumented features when he posted about the utility, so please don’t take this to mean that Jiro or MathWorks officially support this or any other undocumented functionality. Undocumented aspects are often needed to achieve top functionality, and I’m happy that the POTW blog highlights utilities based on their importance and merit, even if they do happen to use some undocumented aspect.

tracer4m‘s code also contains references to the undocumented profiler option -history, but this is not in fact used by the code itself, only in comments. I use this feature in my profile_history utility, which displays the function call/timing history in an interactive GUI window. This utility complements tracer4m by providing a lot more information, but this can result in a huge amount of information for large and/or long-running programs. In addition, tracer4m has the benefit of only logging those functions/methods that the user finds useful, rather than all the function call, which enables easier debugging when the relevant code area is known. In short, I wish I had known about tracer4m when I created profile_history. Now that I know about it, maybe I’ll incorporate some of its ideas into profile_history in order to make it more useful. Perhaps another moral of this is that we should actively monitor the POTW blog, because true gems are quite often highlighted there.

Function call timeline profiling

For anyone who missed the announcement in my previous post, I’m hosting a series of live webinars on advanced Matlab topics in the upcoming 2 weeks – I’ll be happy if you would join.

Using TabComplete for user-defined functions

Unfortunately, this mechanism apparently broke in R2016a and was replaced with a new mechanism, as explained below.

The new mechanism relies on a file called functionSignatures.json which exists in every single folder that contains Matlab files that have functions whose input parameters ought to be tab-completable.

The new mechanism offers far greater versatility and flexability in defining the input types and inter-relationsships compared to the old TC.*-based mechanism. Another important benefit is that we can now add custom user-defined functionSignatures.json files to our user folders, next to our m-files, without having to modify any Matlab system file.

Note that you may need to restart Matlab the first time that you create a functionSignatures.json file. But once it’s created, you can modify it within a Matlab session and the changes take effect immediately.

Note: Credit for first posting about this new mechanism goes to Nicholas Mati. I’ve known about this new mechanism for over a year, but I never found the time to write about it until now, so Nicholas gets credit for breaking the scoop. The discussion below uses and expands Nicholas’ original post.

Syntax

The functionSignatures.json file has the general form:

{
	"FunctionName1":
	{
		"key1":"val1",
		"key2":"val2",
		"keyn":"valn"
	},
	"FunctionName2":
	{
		"key1":"val1",
		"key2":"val2",
		"keyn":"valn"
	}
}

A number of keys are supported including “platform“, “setsAns“, “inputs“, and “outputs“, although inputs and outputs are by far the most common (and presumably only inputs are relevant for tab-completion). These keys take an array of (or a single) object value(s). The objects typically take one of the following forms:

{"name":"variable_name", "kind":"kind_option", "type":"string_or_array_of_type"}
{"mutuallyExclusiveGroup":
	[
		...
	]
}
{"name":"varargin", "kind":"optional", "multiplicity":"append"}

The value for “kind” can be “required”, “optional”, “positional”, “flag”, “namevalue” or “platform” (and perhaps a few other lesser-used kinds):

• “required” means that the specified input is mandatory
• “optional” means that it can be added or omitted
• “positional” means that it’s an optional input but if it is specified then it must appear at the specified position relative to the previous (earlier) inputs
• “flag” means that it’s an optional input flag, from a predefined list of one or more single-token strings. For example, in regexp(s1,s2,'once') the last input arg ('once') is such a flag.
• “namevalue” means that it follows Matlab’s standard practice of using P-V pairs (parameter name followed by its value). For example, func('propName',propValue)
• “platform” indicates that this input is only available on the specified platform(s)

These “kind”s are all explained below.

The value for “type” can be a string such as “char” or “numeric” or “filepath”, or a more complicated JSON array (see below).

In addition to “name”, “kind” and “type”, we can also define a “default” value (e.g. "default":"false") and a “display” string. While these are [currently] not used by Desktop tab-completion, they might be used by other components such as the JIT compiler or the Editor, if not today then perhaps in a future release.

Note that while pure JSON format does not accept comments, Matlab’s functionSignatures.json does accept C++-style comments, as discovered by Heiko in a comment below. To add a comment, simply add // comment text at the end of any line, or /* comment text */ anywhere within a line.

Usage examples

Multiple examples of functionSignatures.json files can be found in subfolders of %matlabroot%/toolbox/matlab. For example, here’s the tab-completion definition for the visdiff function, which displays a visual comparison between two files, and resides in %matlabroot%/toolbox/shared/comparisons/functionSignatures.json:

{
"visdiff":
{
    "inputs":
    [
        {"name":"filename1", "kind":"required",   "type":"filepath"},
        {"name":"filename2", "kind":"required",   "type":"filepath"},
        {"name":"type",      "kind":"positional", "type":"choices={'text', 'binary'}"}
    ]
}
}

As can be seen in this example, the first and second inputs are expected to be a filename, whereas the third input is one of the two predefined strings ‘text’ or ‘binary’. This third input has “kind”:”positional”, meaning that it is optional, but if it is provided then it must be in the 3rd position and cannot appear sooner. Moreover, if the user specifies any input argument to the “right” of a positional input, then the positional argument becomes required, not optional.

Whereas a “positional” parameter has a specific position in the args list (#3 in the case of visdiff above), an “optional” parameter may appear anywhere in the list of inputs.

Here’s a more complex example, for the built-in regexprep function (in %matlabroot%/toolbox/matlab/strfun/functionSignatures.json). This example shows how to limit the input to certain data types and how to specify optional input flags with pre-defined choices:

"regexprep":
{
	"inputs":
	[
		{"name":"str",               "kind":"required",  "type":[["char"], ["cell"], ["string"]]},
		{"name":"expression",        "kind":"required",  "type":[["char"], ["cell"], ["string"]]},
		{"name":"replace",           "kind":"required",  "type":[["char"], ["cell"], ["string"]]},
		{"name":"optMatch",          "kind":"flag",      "display":"", "type":[["char", "choices={'all','once'}"], ["numeric", "scalar"]],   "default":"'all'"},
		{"name":"optWarnings",       "kind":"flag",      "display":"", "type":["char", "choices={'nowarnings','warnings'}"],                 "default":"'nowarnings'"},
		{"name":"optCase",           "kind":"flag",      "display":"", "type":["char", "choices={'matchcase','ignorecase','preservecase'}"], "default":"'matchcase'"},
		{"name":"optEmptyMatch",     "kind":"flag",      "display":"", "type":["char", "choices={'noemptymatch','emptymatch'}"],             "default":"'noemptymatch'"},
		{"name":"optDotAll",         "kind":"flag",      "display":"", "type":["char", "choices={'dotall','dotexceptnewline'}"],             "default":"'dotall'"},
		{"name":"optStringAnchors",  "kind":"flag",      "display":"", "type":["char", "choices={'stringanchors','lineanchors'}"],           "default":"'stringanchors'"},
		{"name":"optSpacing",        "kind":"flag",      "display":"", "type":["char", "choices={'literalspacing','freespacing'}"],          "default":"'literalspacing'"}
	],
	"outputs":
	[
		{"name":"newStr", "type":[["char"], ["cell"], ["string"]]}
	]
},

Here’s an even more complex example, this time for the codegen function (in %matlabroot%/toolbox/coder/matlabcoder/functionSignatures.json, part of the Matlab Coder toolbox). This example shows how to limit the filenames to certain extensions and how to specify name-value input pairs:

"codegen":
{
	"inputs":
	[
		{"name":"compile_only",  "kind":"flag",       "type":"choices={'-c'}"},
		{"name":"config_flag",   "kind":"flag",       "type":"choices={'-config:mex','-config:lib','-config:dll','-config:exe','-config:hdl'}"},
		{"name":"debug",         "kind":"flag",       "type":"choices={'-g'}"},
		{"name":"report",        "kind":"flag",       "type":"choices={'-report'}"},
		{"name":"launchreport",  "kind":"flag",       "type":"choices={'-launchreport'}"},
		{"name":"file",          "kind":"flag",       "type":"filepath=*.m,*.mlx,*.c,*.cpp,*.h,*.o,*.obj,*.a,*.so,*.lib,*.tmf", "multiplicity":"append"},
		{"name":"-d",            "kind":"namevalue",  "type":"folderpath"},
		{"name":"-I",            "kind":"namevalue",  "type":"folderpath"},
		{"name":"-globals",      "kind":"namevalue"},
		{"name":"-o",            "kind":"namevalue",  "type":[["char"], ["filepath"]]},
		{"name":"-O",            "kind":"namevalue",  "type":"choices={'enable:inline','disable:inline','enable:blas','disable:blas','enable:openmp','disable:openmp'}"},
		{"name":"-args",         "kind":"namevalue",  "type":[["identifier=variable"], ["char"]]},
		{"name":"-config",       "kind":"namevalue",  "type":[["identifier=variable"], ["char"]]},
		{"name":"verbose",       "kind":"flag",       "type":"choices={'-v'}"},
		{"name":"singleC",       "kind":"flag",       "type":"choices={'-singleC'}"},
		{"name":"-test",         "kind":"namevalue",  "type":"identifier=function"}
	]
},

Argument types

As noted above, we use "type":... to specify the expected data type of each parameter. This can be a simple string such as “char”, “cellstr”, “numeric”, “table”, “categorical”, “filepath”, “folderpath”, “matlabpath”, class name, or a more complicated JSON array. For example:

• "type":["numeric","scalar"]
• "type":["numeric","numel=3",">=4"]
• "type":[["char"], ["cellstr"], ["numeric"], ["logical","vector"]]
• "type":[["char", "choices={'-ascii'}"]]
• "type":[["filepath"], ["matlabpath=*.m,*.mlx"], ["char"]]
• "type":"identifier=variable,function,localfunction,package,classdef"
• "type":"matlab.graphics.axis.Axes"
• "type":"choices={'yes','no','maybe'}"

We can even specify on-the-fly Matlab computation that returns a cell-array of values, for example a list of available fonts via "type":"choices=listfonts". A more complex example is the definition of the rmfield function, where the possible input choices for the second input arg (highlighted) depend on the struct that is provided in the first input arg (by running the fieldnames function on it):

"rmfield":
{
	"inputs":
	[
		{"name":"s",     "kind":"required", "type":"struct"},
		{"name":"field", "kind":"required", "type":"choices=fieldnames(s)"}	],
	"outputs":
	[
		{"name":"s", "type":"struct"}
	]
},

Alternative inputs

Multiple alternative inputs can be specified in the functionSignatures.json file. The easiest way to do so is to simply create multiple different definitions for the same function, one beneath the other. Matlab’s tab-completion parser is smart enough to combine those definitions and proceed with the most appropriate one based on the user-entered inputs.

For example, in the same Coder file above we find 6 alternative definitions. If (for example) we start typing coder('-ecoder', and click <Tab>, Matlab would automatically auto-complete the second input to “false”, and then another <Tab> click would set the third input to the required ‘-new’ parameter (see highlighted lines below):

...
"coder":
{
	"inputs":
	[
		{"name":"projectname", "kind":"required", "type":"filepath=*.prj"}
	]
},
"coder":
{
	"inputs":
	[
		{"name":"-open", "kind":"namevalue", "type":"filepath=*.prj"}
	]
},
"coder":
{
	"inputs":
	[
		{"name":"-build", "kind":"namevalue", "type":"filepath=*.prj"}
	]
},
"coder":
{
	"inputs":
	[
		{"name":"-new", "kind":"namevalue", "type":[["filepath=*.prj"], ["char"]]}
	]
},
"coder":
{
	"inputs":
	[
		{"name":"ecoderFlag",  "kind":"required", "type":"choices={'-ecoder'}"},		{"name":"ecoderValue", "kind":"required", "type":[["logical"], ["choices={'false'}"]]},		{"name":"newFlag",     "kind":"required", "type":"choices={'-new'}"},		{"name":"newvalue",    "kind":"required", "type":[["filepath=*.prj"], ["char"]]}	]
},
"coder":
{
	"inputs":
	[
		{"name":"tocodeFlag",  "kind":"required", "type":"choices={'-tocode'}"},
		{"name":"tocodevalue", "kind":"required", "type":"filepath=*.prj"},
		{"mutuallyExclusiveGroup":
			[
				[],
				[
					{"name":"scriptFlag", "kind":"required", "type":"choices={'-script'}"},
					{"name":"scriptname", "kind":"required", "type":[["filepath=*.m"], ["char"]]}
				]
			]
		}
	]
}

This example also shows, in the last definition for the coder function, another mechanism for specifying alternative inputs, using “mutuallyExclusiveGroup” (aka “MEGs”). A MEG is defined using an array of options, enclosed in square brackets ([]). Each of the MEG options is exclusive to each of the others, meaning that we can only work with one of them and not the others. This is equivalent to duplicating the definition as we saw above, and saves us some copy-paste (in some cases a lot of copy-pastes, especially with multiple and/or nested MEGs). However, MEGs have a major drawback of reduced readability. I believe that in most cases we only have a single MEG and few input args, and in such cases it makes more sense to use repeated function defs rather than a MEG. The Matlab signature files contain numerous usage examples for either of these two mechanisms.

Platform dependencies

If a specific function (or a specific signature variant) depends on the running platform, this can be specified via the “platform” directive. For example, the winopen function only works on Windows, but not on Linux/Mac. Its corresponding signature definition is:

"winopen":
{
	"platform":"win32,win64",	"inputs":
	[
		{"name":"filename", "kind":"required", "type":"filepath"},
		{"name":"varargin", "kind":"optional", "multiplicity":"append"}
	]
}

Platform dependence could also be specified at the parameter level, not just the entire function level. For example, in the xlsread function (defined in %matlabroot%/toolbox/matlab/iofun/functionSignatures.json), we see that the usage variant xlsread(filename,-1) is only available on Windows (note that the numeric value is defined as "<=-1", not necessarily -1), and so is the “functionHandle” input (which is called processFcn in the documentation – for some reason that escapes me the names of many input args do not match in the documentation and functionSignature):

"xlsread":
{
	"inputs":
	[
		{"name":"filename", "kind":"required", "type":"filepath=*.xls,*.xlsx,*.xlsb,*.csv"},
		{"mutuallyExclusiveGroup":
			[
				{"name":"openExcel", "kind":"required", "display":"", "type":["numeric", "<=-1"], "platform":"win64,win32"},				{"name":"xlRange",   "kind":"required", "type":["char", "@(x) isempty(x) || ~isempty(strfind(x, ':'))"], "default":"''"},
				[
					{"name":"sheet",          "kind":"positional", "type":[["char", "choices=matlab.internal.language.introspective.tabcompletion.xlsread_vsheet(filename)"], ["numeric", ">=1"]], "default":"1"},
					{"name":"xlRange",        "kind":"positional", "type":"char", "default":"''"},
					{"name":"basic",          "kind":"positional", "display":"", "type":["char", "choices={'basic',''}"]},
					{"name":"functionHandle", "kind":"positional", "type":"function_handle", "platform":"win64,win32"}				]
			]
		}
	],
        ...

Parsing errors

The new mechanism is not very user-friendly when you get something wrong. In the best case, it issues a cryptic error message (see below), and in the worst case it simply ignores the changes and the user has no idea why the new custom tab-completion is not working as intended.

The most likely causes of such problems are:

• The most common problem is that you placed the functionSignatures.json file in a different folder than the Matlab function. For example, if the myFunction() function is defined in myFunction.m, then the tab-completion of this function MUST be located in a functionSignatures.json file that resides in the same folder, not anywhere else on the Matlab path. In other words, the Matlab path is NOT relevant for tab-completion.
• Your functionSignatures.json file does not follow the [extremely strict] syntax rules above, to the letter. For example, forgetting the top or final curly braces, forgetting a comma or adding an extra one, or not closing all brackets/braces properly.
• You mistyped one or more of the input parameters, types or options.

In case of a parsing error, you’d see a red error message on the Matlab console the next time that you try to use tab-completion:

Error parsing JSON data; Boost reports "(189): expected ',' or ']'".

Unfortunately the error message only tells us the problematic line location within the functionSignatures.json file, but not the file’s location, so if we haven’t recently edited this file we’d need to find it in the relevant folder. For example:

edit(fullfile(fileparts(which('myFunction')), 'functionSignatures.json')

Moreover, when a JSON syntax error (such as the one above) occurs, the entire file is not parsed, not just the definition that caused the error.

Another limitation of tab-completion is that it does not work while the main Matlab thread is working (e.g., during a uiwait or waitfor). This may be somewhat misleading since most editor/debugging actions do work.

Arguably, this new tab-completion mechanism could be made more programmer-friendly. Perhaps this will improve in a future Matlab release.

Addendum: MathWorks has now made this functionality supported and documented. Read about it here.

For a related mechanism, see my article on tab-completion for class properties and methods from 2014, which is apparently still relevant and functional.

Alignment

For example, a question that I am often asked (latest example) is whether it is possible to left/center/right align the label within a Matlab button, listbox or table. While Matlab does not (yet) have properties that control alignment in uicontrols, we can indeed use HTML for this. There’s a catch though: if we simply tried to use <div align="left">…, it will not work. No error will be generated but we will not see any visible left-alignment. The reason is that internally, the text is contained within a snugly-fitting box. Aligning anything within a tight-fitting box obviously has no effect.

To solve the problem, we need to tell Matlab (or rather, the HTML interpreter used by the underlying Java control) to widen this internal box. One way to do this is to specify the width of the div tag, which can be enormous in order to span the entire available apace (<div width="999px" align="left">…). Another method is to simulate a simple HTML table that contains a single cell that holds the text, and then tell HTML the table cell’s width:

hButton.String   = '<html><tr><td width=9999 align=left>Left-aligned';  % left-align within a button
hTable.Data{2,1} = '<html><tr><td width=9999 align=right>And right';   % right-align within a specific uitable cell

 

Centered (default) and right-aligned button labels

Non-default alignment of uitable cells

I discussed the specific aspect of uicontrol content alignment in another post last year.

Background color

The same problem (and solution) applies to background colors: if we don’t enlarge the snugly-fitting internal bounding-box, any HTML bgcolor that we specify would only be shown under the text (i.e., within the internal box’s confines). In order to display bgcolor across the entire control/cell width, we need to enlarge the internal box’s width (the align and bgcolor tags can of course be used together):

hButton.String   = '<html><tr><td width=9999 bgcolor=#ffff00>Yellow';  % bgcolor within a button
hTable.Data{2,1} = '<html><tr><td width=9999 bgcolor=#ffff00>Yellow';  % bgcolor within a specific uitable cell

CSS

We can also use simple CSS, which provides more formatting customizability than plain HTML:

hTable.Data{2,1} = '<html><tr><td width=9999 style="background-color:yellow">Yellow';

HTML/CSS formatting is a poor-man’s hack. It is very crude compared to the numerous customization options available via Java. However, it does provide a reasonable solution for many use-cases, without requiring any Java. I discussed the two approaches for uitable cell formatting in this post.

[Non-]support in uifigures

Important note: HTML formatting is NOT [yet] supported by the new web-based uifigures. While uifigures can indeed be hacked with HTML/CSS content (details), this is not an easy task. Since it should be trivially easy for MathWorks to enable HTML content in the new web-based uifigures, I implore anyone who uses HTML in their Matlab GUI to let MathWorks know about it so that they could prioritize this R&D effort into an upcoming Matlab release. You can send an email to Eric.Sargent at mathworks.com, who handles such aspects in MathWorks’ R&D efforts to transition from Java-based GUIs to web-based ones. In my previous post I spotlit MathWorks user-feedback surveys about users’ use of Java GUI aspects, aimed in order to migrate as many of the use-cases as possible onto the new web-based framework. HTML/CSS support is a natural by-product of the fact that Matlab’s non-web-based GUI is based on Java Swing components (that inherently support HTML/CSS). But unfortunately the MathWorks surveys are specific to the javacomponent function and the figure’s JavaFrame property. In other words, many users might be using undocumented Java aspects by simply using HTML content in their GUI, without ever realizing it or using javacomponent. So I think that in this case a simple email to Eric.Sargent at mathworks.com to let him know how you’re using HTML would be more useful. Maybe one day MathWorks will be kind enough to post a similar survey specific to HTML support, or maybe one day they’s just add the missing HTML support, if only to be done with my endless nagging.

p.s. – I am well aware that we can align and bgcolor buttons in AppDesigner. But we can’t do this with individual table/listbox cells, and in general we can’t use HTML within uifigures without extensive hacks. I merely used the simple examples of button and uitable cell formatting in today’s post to illustrate the issue. So please don’t get hung up on the specifics, but rather on the broader issue of HTML support in uifigures.

And in the meantime, for as long as non-web-based GUI is still supported in Matlab, keep on enjoying the benefits that HTML/CSS provides.

Automated bug-fix emails

In an unrelated matter, I wish to express my Kudos to the nameless MathWorkers behind the scenes who, bit by bit, improve Matlab and the user experience: Over the years I’ve posted a few times my frustrations with the opaqueness of MathWorks’ bug-reporting mechanism. One of my complaints was that users who file bugs are not notified when a fix or workaround becomes available. That at least seems to have been fixed now. I just received a seemingly-automated email notifying me that one of the bugs that I reported a few years ago has been fixed. This is certainly a good step in the right direction, so thank you!

Happy Passover/Easter to all!

Data serialization quirks

Dimitri Shvorob may not appear at first glance to be a prolific contributor on Matlab Central, but from the little he has posted over the years I regard him to be a Matlab power-user. So when Dimitri reports something, I take it seriously. Such was the case several months ago, when he contacted me regarding very odd behavior that he saw in his code: the for loop worked well, but the parfor version returned different (incorrect) results. Eventually, Dimitry traced the problem to something originally reported by Dan Austin on his Fluffy Nuke It blog.

The core issue is that if we have a class object that is used within a for loop, Matlab can access the object directly in memory. But with a parfor loop, the object needs to be serialized in order to be sent over to the parallel workers, and deserialized within each worker. If this serialization/deserialization process involves internal class methods, the workers might see a different version of the class object than the one seen in the serial for loop. This could happen, for example, if the serialization/deserialization method croaks on an error, or depends on some dynamic (or random) conditions to create data.

In other words, when we use data objects in a parfor loop, the data object is not necessarily sent “as-is”: additional processing may be involved under the hood that modify the data in a way that may be invisible to the user (or the loop code), resulting in different processing results of the parallel (parfor) vs. serial (for) loops.

For additional aspects of Matlab serialization/deserialization, see my article from 2 years ago (and its interesting feedback comments).

Data precision quirks

The following section was contributed by guest blogger Lior Perlmuter-Shoshany, head algorithmician at a private equity fund.

In my work, I had to work with matrixes in the order of 10⁹ cells. To reduce the memory footprint (and hopefully also improve performance), I decided to work with data of type single instead of Matlab’s default double. Furthermore, in order to speed up the calculation I use parfor rather than for in the main calculation. In the end of the run I am running a mini for-loop to see the best results.

What I discovered to my surprise is that the results from the parfor and for loop variants is not the same!

The following simplified code snippet illustrate the problem by calculating a simple standard-deviation (std) over the same data, in both single– and double-precision. Note that the loops are ran with only a single iteration, to illustrate the fact that the problem is with the parallelization mechanism (probably the serialization/deserialization parts once again), not with the distribution of iterations among the workers.

clear
rng('shuffle','twister');
 
% Prepare the data in both double and single precision
arr_double = rand(1,100000000);
arr_single = single(arr_double);
 
% No loop - direct computation
std_single0 = std(arr_single);
std_double0 = std(arr_double);
 
% Loop #1 - serial for loop
std_single = 0;
std_double = 0;
for i=1
    std_single(i) = std(arr_single);
    std_double(i) = std(arr_double);
end
 
% Loop #2 - parallel parfor loop
par_std_single = 0;
par_std_double = 0;
parfor i=1
    par_std_single(i) = std(arr_single);
    par_std_double(i) = std(arr_double);
end
 
% Compare results of for loop vs. non-looped computation
isForSingleOk = isequal(std_single, std_single0)
isForDoubleOk = isequal(std_double, std_double0)
 
% Compare results of single-precision data (for vs. parfor)
isParforSingleOk = isequal(std_single, par_std_single)
parforSingleAccuracy = std_single / par_std_single
 
% Compare results of double-precision data (for vs. parfor)
isParforDoubleOk = isequal(std_double, par_std_double)
parforDoubleAccuracy = std_double / par_std_double

Output example :

isForSingleOk = 
    1                   % <= true (of course!)
isForDoubleOk =
    1                   % <= true (of course!)
 
isParforSingleOk =
    0                   % <= false (odd!)
parforSingleAccuracy =
    0.73895227413361    % <= single-precision results are radically different in parfor vs. for
 
isParforDoubleOk =
    0                   % <= false (odd!)
parforDoubleAccuracy =
    1.00000000000021    % <= double-precision results are almost [but not exactly] the same in parfor vs. for

From my testing, the larger the data array, the bigger the difference is between the results of single-precision data when running in for vs. parfor.

In other words, my experience has been that if you have a huge data matrix, it’s better to parallelize it in double-precision if you wish to get [nearly] accurate results. But even so, I find it deeply disconcerting that the results are not exactly identical (at least on R2015a-R2016b on which I tested) even for the native double-precision .

Hmmm… bug?

Upcoming travels – Zürich & Geneva

I will shortly be traveling to clients in Zürich and Geneva, Switzerland. If you are in the area and wish to meet me to discuss how I could bring value to your work with some advanced Matlab consulting or training, then please email me (altmany at gmail):

• Zürich: January 15-17
• Geneva: January 18-21

Happy new year everybody!

Matlab allows flexible adjustment of visibility of warning messages. Some, or even all, messages can be disabled from showing on the screen by warning command.

The little known fact is that status of some warnings may be used to change the execution path in algorithms. For example, if warning 'Matlab:nearlySingularMatrix' is disabled, then the linear system solver (mldivide operator) might skip estimation of reciprocal condition number which is used exactly for the purpose of detection of nearly singular matrices. If the trick is used, it allows 20%-50% boost in solver performance, since rcond estimation is a time consuming process.

Therefore it is important to be able to retrieve status of warnings in Matlab. Especially in MEX libraries targeted for improved performance. Unfortunately Matlab provides no simple way to check status of warning message from MEX module.

Today’s article outlines two workarounds for the issue:

1. Using mexCallMATLABWithTrap (documented)
2. Using utGetWarningStatus (undocumented)

Using mexCallMATLABWithTrap (documented)

The first idea is to use documented mexCallMATLABWithTrap function to execute warning(‘query’,…) command using Matlab’s interpreter and then parse the returned result:

bool mxIsWarningEnabled(const char* warningId)
{
    bool enabled = true;
 
    if (NULL != warningId)
    {
        mxArray *mxCommandResponse = NULL, *mxException = NULL;
        mxArray *args[2];
 
        /* warning('query', warningId); */
        args[0] = mxCreateString("query");
        args[1] = mxCreateString(warningId);
        mxException = mexCallMATLABWithTrap(1,&mxCommandResponse,2,args,"warning");
        if (NULL == mxException && NULL != mxCommandResponse)
        {
            if (mxIsStruct(mxCommandResponse))
            {
                const mxArray* state_field = mxGetField(mxCommandResponse, 0, "state");
                if (mxIsChar(state_field))
                {
                    char state_value[8] = {0};
                    enabled = (0 == mxGetString(state_field, state_value, 8)) &&
                              (0 == strcmp(state_value,"on"));
                }
            }
            mxDestroyArray(mxCommandResponse);
        }
        else
        {
            /* 'warning' returned with error */
            mxDestroyArray(mxException);
        }
        mxDestroyArray(args[0]);
        mxDestroyArray(args[1]);
    }
    return enabled;
}

This approach is slow, but works fine in most standard situations. See the bottom of this post for a usage example.

However, this approach has an important drawback – we should be careful with recursive calls to the Matlab interpreter (Matlab -> MEX -> Matlab) and with handling Matlab errors in MEX. It is safe only if we use identical standard libraries and compiler to build both MEX and Matlab.

In other cases, for example when MEX is targeted to work with different versions of Matlab, or was built with a different standard library and compiler, etc. – cross boundary handling of errors (which are just C++ exceptions) might lead to unpredictable results, most likely segfaults.

Using utGetWarningStatus (undocumented)

To avoid all the overhead of calling Matlab interpreter and unsafe error handling, we can use some undocumented internal Matlab functions:

/* Link with libut library to pick-up undocumented functions: */
extern "C" void* utGetWarningManagerContext(void);
extern "C" bool  utIsValidMessageIdentifier(const char *warningId);
extern "C" bool  utGetWarningStatus(void* context, const char *warningId);
 
/* 
   Returns true if warning with warningId enabled 
   Matlab versions supported/tested: R2008b - R2016b
*/
bool mxIsWarningEnabled(const char *warningId)
{
    bool enabled = true;
 
    if (NULL != warningId && utIsValidMessageIdentifier(warningId))
    {
        void* context = utGetWarningManagerContext();
        enabled = (NULL != context) && utGetWarningStatus(context, warningId);
    }
    return enabled;
}

Now the code is clean, fast and safe – we bypass the interpreter and work directly with Matlab kernel. All the undocumented functions involved are present in Matlab for at least 10 years and do simple logical checks under the hood.

The standard function mexWarnMsgIdAndTxt uses similar code to check if it should display the warning or just suppress it, and that code remains unchanged since R2008b. This is a good indication of code stability and makes us believe that it will not be changed in future versions of Matlab.

For both workarounds, usage is simple:

if (mxIsWarningEnabled("Matlab:nearlySingularMatrix"))
{
   /* compute rcond */
}
else
{
   /* do something else */
}

None of the information I’ll present today is really new – it was all there already if you just knew what to search for online. But hopefully today’s post will concentrate all these loose ends in a single place, so it may have some value:

• Using a secure mail server
• Emailing multiple recipients
• Sending text messages
• User configuration panel

Using a secure mail server

All modern mail servers use end-to-end TLS/SSL encryption. The sendmail function needs extra configuration to handle such connections, since it is configured for a non-encrypted connection by default. Here’s the code that does this for gmail, using SMTP server smtp.gmail.com and default port #465 (for other SMTP servers, see here):

setpref('Internet', 'E_mail', from_address);  % sender "from" address, typically same as username, e.g. 'xyz@gmail.com'
setpref('Internet', 'SMTP_Username', username);
setpref('Internet', 'SMTP_Password', password);
setpref('Internet', 'SMTP_Server',   'smtp.gmail.com');
 
props = java.lang.System.getProperties;
props.setProperty('mail.smtp.auth',                'true');  % Note: 'true' as a string, not a logical value!
props.setProperty('mail.smtp.starttls.enable',     'true');  % Note: 'true' as a string, not a logical value!
props.setProperty('mail.smtp.socketFactory.port',  '465');   % Note: '465'  as a string, not a numeric value!
props.setProperty('mail.smtp.socketFactory.class', 'javax.net.ssl.SSLSocketFactory');
 
sendmail(recipient, title, body, attachments);  % e.g., sendmail('recipient@gmail.com', 'Hello world', 'What a nice day!', 'C:\images\sun.jpg')

All this is not enough to enable Matlab to connect to gmail’s SMTP servers. In addition, we need to set the Google account to allow access from “less secure apps” (details, direct link). Without this, Google will not allow Matlab to relay emails. Other mail servers may require similar server-side account configurations to enable Matlab’s access.

Note: This code snippet uses a bit of Java as you can see. Under the hood, all networking code in Matlab relies on Java, and sendmail is no exception. For some reason that I don’t fully understand, MathWorks chose to label the feature of using sendmail with secure mail servers as a feature that relies on “undocumented commands” and is therefore not listed in sendmail‘s documentation. Considering the fact that all modern mail servers are secure, this seems to make sendmail rather useless without the undocumented extension. I assume that TMW are well aware of this, which is the reason they posted a partial documentation in the form of an official tech-support answer. I hope that one day MathWorks will incorporate it into sendmail as optional input args, so that using sendmail with secure servers would become fully documented and officially supported.

Emailing multiple recipients

To specify multiple email recipients, it is not enough to set sendmail‘s recipient input arg to a string with , or ; delimiters. Instead, we need to provide a cell array of individual recipient strings. For example:

sendmail({'recipient1@gmail.com','recipient2@gmail.com'}, 'Hello world', 'What a nice day!')

Note: this feature is actually fully documented in sendmail‘s doc-page, but for some reason I see that some users are not aware of it (to which it might be said: RTFM!).

Sending phone text (SMS) messages

With modern smartphones, text (SMS) messages have become rather outdated, as most users get push notifications of incoming emails. Still, for some users text messages may still be a useful. To send such messages, all we need is to determine our mobile carrier’s email gateway for SMS messages, and send a simple text message to that email address. For example, to send a text message to T-Mobile number 123-456-7890 in the US, simply email the message to 1234567890@tmomail.net (details).

Ke Feng posted a nice Matlab File Exchange utility that wraps this messaging for a wide variety of US carriers.

User configuration panel

Many GUI programs contain configuration panels/tabs/windows. Enabling the user to set up their own email provider is a typical use-case for such a configuration. Naturally, you’d want your config panel not to display plain-text password, nor non-integer port numbers. You’d also want the user to be able to test the email connection.

Here’s a sample implementation for such a panel that I implemented for a recent project – I plan to discuss the implementation details of the password and port (spinner) controls in my next post, so stay tuned:

User configuration of emails in Matlab GUI (click to zoom-in)

 

Matlab listbox (left) & popup menu (right) with icon images

My initial thought was that this should surely be possible, since Colormap is a documented figure property, that should therefore be listed inside the inspector window, and should therefore have an associated builtin Java control for the dropdown (just like other inspector controls, which are part of the com.mathworks.mlwidgets package, or possibly as a standalone control in the com.mathworks.mwswing package). To my surprise it turns out that for some unknown reason MathWorks neglected to add the Colormap property (and associated Java controls) to the inspector. This property is fully documented and all, just like Color and other standard figure properties, but unlike them Colormap can only be modified programmatically, not via the inspector window. Matlab does provide the related colormapeditor function and associated dialog window, but I would have expected a simple drop-down of the standard builtin colormaps to be available in the inspector. Anyway, this turned out to be a dead-end.

It turns out that we can relatively easily implement the requested listbox/combo-box using a bit of HTML magic, as I explained last week. The basic idea is for each of the listbox/combobox items to be an HTML string that contains both an <img> tag for the icon and the item label text. For example, such a string might contain something like this (parula is Matlab’s default colormap in HG2, starting in R2014b):

<html><img src="http://www.mathworks.com/help/matlab/ref/colormap_parula.png">parula

parula colormap image

Of course, it would be a bit inefficient for each of the icons to be fetched from the internet. Luckily, the full set of Matlab documentation is typically installed on the local computer as part of the standard Matlab installation, beneath the docroot folder (e.g., C:\Program Files\Matlab\R2016b\help). In our specific case, the parula colormap image is located in:

imageFilename = [docroot, '/matlab/ref/colormap_parula.png']

Note that for a local image to be accepted by HTML, it needs to follow certain conventions. In our case, the HTML string for displaying the above image is:

<html><img src="file:///C:/Program%20Files/Matlab/R2016b/help/matlab/ref/colormap_parula.png">parula

Warning: it’s easy when dealing with HTML images in Matlab to get the format confused, resulting in a red-x icon. I discussed this issue some 4 years ago, which is still relevant.

How can we get the list of available builtin colormaps? The standard Matlab way of doing this would be something like this:

>> possibleColormaps = set(gcf,'Colormap')
possibleColormaps = 
     {}

but as we can see, for some unknown reason (probably another MathWorks omission), Matlab does not list the names of its available builtin colormaps.

Fortunately, all the builtin colormaps have image filenames that follow the same convention, which make it easy to get this list by simply listing the names of the relevant files, from which we can easily create the necessary HTML strings:

>> iconFiles = dir([docroot, '/matlab/ref/colormap_*.png']);
 
>> colormapNames = regexprep({iconFiles.name}, '.*_(.*).png', '$1')
colormapNames =  
  Columns 1 through 9
    'autumn'    'bone'    'colorcube'    'cool'    'copper'    'flag'    'gray'    'hot'    'hsv'
  Columns 10 through 18
    'jet'    'lines'    'parula'    'pink'    'prism'    'spring'    'summer'    'white'    'winter'
 
>> htmlStrings = strcat('<html><img width=200 height=10 src="file:///C:/Program%20Files/Matlab/R2016a/help/matlab/ref/colormap_', colormapNames', '.png">', colormapNames')
str = 
    '<html><img width=200 height=10 src="file:///C:/Program%20Files/Matlab/R2016a/help/matlab/ref/colormap_autumn.png">autumn'
    '<html><img width=200 height=10 src="file:///C:/Program%20Files/Matlab/R2016a/help/matlab/ref/colormap_bone.png">bone'
    '<html><img width=200 height=10 src="file:///C:/Program%20Files/Matlab/R2016a/help/matlab/ref/colormap_colorcube.png">colorcube'
    ...
 
>> hListbox = uicontrol(gcf, 'Style','listbox', 'Units','pixel', 'Pos',[10,10,270,200], 'String',htmlStrings);
>> hPopup   = uicontrol(gcf, 'Style','popup',   'Units','pixel', 'Pos',[10,500,270,20], 'String',htmlStrings);

…which results in the screenshots at the top of this post.

Note how I scaled the images to 10px high (so that the labels would be shown and not cropped vertically) and 200px wide (so that it becomes narrower than the default 434px). There’s really no need in this case for the full 434×27 image size – such flat images scale very nicely, even when their aspect ratio is not preserved. You can adjust the height and width values for a best fit with you GUI.

Unfortunately, it seems that HTML strings are not supported in the new web-based uifigure controls. This is not really Matlab’s fault because the way to customize labels in HTML controls is via CSS: directly embedding HTML code in labels does not work (it’s a Java-Swing feature, not a browser feature). I really hope that either HTML or CSS processing will be enabled for web-based uicontrol in a future Matlab release, because until that time uifigure uicontrols will remain seriously deficient compared to standard figure uicontrols. Until then, if we must use uifigures and wish to customize our labels or listbox items, we can directly access the underlying web controls, as Iliya explained here.

A blog reader recently complained that I’m abusing Swing and basically making Matlab work in unnatural ways, “something it was never meant to be“. I feel that using HTML as I’ve shown last week and in this post would fall under the same category in his eyes. To him and to others who complain I say that I have absolutely no remorse about doing this. When I purchase anything I have the full rights (within the scope of the license) to adapt it in whatever way fits my needs. As a software developer and manager for over 25 years, I’ve developed in dozens of programming languages and environments, and I still enjoy [ab]using Matlab. Matlab is a great environment to get things done quickly and if this sometimes requires a bit of HTML or Java hacks that make some people cringe, then that’s their problem, not mine – I’m content with being able to do in Matlab [nearly] everything I want, quickly, and move on to the next project. As long as it gets the job done, that’s fine by me. If this makes me more of an engineer than a computer scientist, then so be it.

On the flip side, I say to those who claim that Matlab is lacking in this or that aspect, that in most likelihood the limitation is only in their minds, not in Matlab – we can do amazing stuff with Matlab if we just open our minds, and possibly use some undocumented hacks. I’m not saying that Matlab has no limitations, I’m just saying that in most cases they can be overcome if we took the time and trouble to look for a solution. Matlab is a great tool and yet many people are not aware of its potential. Blaming Matlab for its failings is just an easy excuse in many cases. Of course, MathWorks could help my crusade on this subject by enabling useful features such as easy GUI component customizations…

On this sad day, I wish you all Shanah Tova!

Matlab's new AppDesigner (a somewhat outdated screenshot)

However, while AppDesigner has become officially supported, the underlying technology used for the new uifigures remained undocumented. This is not surprising: MathWorks did a good job of retaining backward compatibility with the existing figure handle, and so a new uifigure returns a handle that programmatically appears similar to figure handles, reducing the migration cost when MathWorks decides (presumably around 2018-2020) that web-based (rather than Java-based) figures should become the default figure type. By keeping the underlying figure technology undocumented and retaining the documented top-level behavior (properties and methods of the figure handle), Matlab users who only use the documented interface should expect a relatively smooth transition at that time.

So does this mean that users who start using AppDesigner today (and especially in a few years when web figures become the default) can no longer enjoy the benefits of figure-based customization offered to the existing Java-based figure users (which I listed in last month’s post)? Absolutely not! All we need is to get a hook into the uifigure‘s underlying object and then we can start having fun.

The uifigure Controller

One way to do this is to use the uifigure handle’s hidden (private) Controller property (a matlab.ui.internal.controller.FigureController MCOS object whose source-code appears in %matlabroot%/toolbox/matlab/uitools/uicomponents/components/+matlab/+ui/+internal/+controller/).

Controller is not only a hidden but also a private property of the figure handle, so we cannot simply use the get function to get its value. This doesn’t stop us of course: We can get the controller object using either my getundoc utility or the builtin struct function (which returns private/protected properties as an undocumented feature):

>> hFig = uifigure('Name','Yair', ...);
 
>> figProps = struct(hFig);  % or getundoc(hFig)
Warning: Calling STRUCT on an object prevents the object from hiding its implementation details and should thus be
avoided. Use DISP or DISPLAY to see the visible public details of an object. See 'help struct' for more information.
(Type "warning off MATLAB:structOnObject" to suppress this warning.)
 
Warning: figure JavaFrame property will be obsoleted in a future release. For more information see
the JavaFrame resource on the MathWorks web site.
(Type "warning off MATLAB:HandleGraphics:ObsoletedProperty:JavaFrame" to suppress this warning.)
 
figProps = 
                      JavaFrame: []
                    JavaFrame_I: []
                       Position: [87 40 584 465]
                   PositionMode: 'auto'
                            ...
                     Controller: [1x1 matlab.ui.internal.controller.FigureController]
                 ControllerMode: 'auto'
                            ...
 
>> figProps.Controller
ans = 
  FigureController with properties:
 
       Canvas: []
    ProxyView: [1x1 struct]
 
>> figProps.Controller.ProxyView
ans = 
            PeerNode: [1x1 com.mathworks.peermodel.impl.PeerNodeImpl]
    PeerModelManager: [1x1 com.mathworks.peermodel.impl.PeerModelManagerImpl]
 
>> struct(figProps.Controller)
Warning: Calling STRUCT on an object prevents the object from hiding its implementation details and should thus be
avoided. Use DISP or DISPLAY to see the visible public details of an object. See 'help struct' for more information.
(Type "warning off MATLAB:structOnObject" to suppress this warning.)
 
ans = 
               PositionListener: [1x1 event.listener]
    ContainerPositionCorrection: [1 1 0 0]
                      Container: [1x1 matlab.ui.internal.controller.FigureContainer]
                         Canvas: []
                  IsClientReady: 1
              PeerEventListener: [1x1 handle.listener]
                      ProxyView: [1x1 struct]
                          Model: [1x1 Figure]
               ParentController: [0x0 handle]
      PropertyManagementService: [1x1 matlab.ui.internal.componentframework.services.core.propertymanagement.PropertyManagementService]
          IdentificationService: [1x1 matlab.ui.internal.componentframework.services.core.identification.WebIdentificationService]
           EventHandlingService: [1x1 matlab.ui.internal.componentframework.services.core.eventhandling.WebEventHandlingService]

I will discuss all the goodies here in a future post (if you are curious then feel free to start drilling in there yourself, I promise it won’t bite you…). However, today I wish to concentrate on more immediate benefits from a different venue:

The uifigure webwindow

uifigures are basically webpages rather than desktop windows (JFrames). They use an entirely different UI mechanism, based on HTML webpages served from a localhost webserver that runs CEF (Chromium Embedded Framework version 3.2272 on Chromium 41 in R2016a). This runs the so-called CEF client (apparently an adaptation of the CefClient sample application that comes with CEF; the relevant Matlab source-code is in %matlabroot%/toolbox/matlab/cefclient/). It uses the DOJO Javascript toolkit for UI controls visualization and interaction, rather than Java Swing as in the existing JFrame figures. I still don’t know if there is a way to combine the seemingly disparate sets of GUIs (namely adding Java-based controls to web-based figures or vice-versa).

Anyway, the important thing to note for my purposes today is that when a new uifigure is created, the above-mentioned Controller object is created, which in turn creates a new matlab.internal.webwindow. The webwindow class (%matlabroot%/toolbox/matlab/cefclient/+matlab/+internal/webwindow.m) is well-documented and easy to follow (although the non camel-cased class name escaped someone’s attention), and allows access to several important figure-level customizations.

The figure’s webwindow reference can be accessed via the Controller‘s Container‘s CEF property:

>> hFig = uifigure('Name','Yair', ...);
>> warning off MATLAB:structOnObject      % suppress warning (yes, we know it's naughty...)
>> figProps = struct(hFig);
 
>> controller = figProps.Controller;      % Controller is a private hidden property of Figure
>> controllerProps = struct(controller);
 
>> container = controllerProps.Container  % Container is a private hidden property of FigureController
container = 
  FigureContainer with properties:
 
    FigurePeerNode: [1x1 com.mathworks.peermodel.impl.PeerNodeImpl]
         Resizable: 1
          Position: [86 39 584 465]
               Tag: ''
             Title: 'Yair'
              Icon: 'C:\Program Files\Matlab\R2016a\toolbox\matlab\uitools\uicomponents\resources\images…'
           Visible: 1
               URL: 'http://localhost:31417/toolbox/matlab/uitools/uifigureappjs/componentContainer.html…'
              HTML: 'toolbox/matlab/uitools/uifigureappjs/componentContainer.html'
     ConnectorPort: 31417
         DebugPort: 0
     IsWindowValid: 1
 
>> win = container.CEF   % CEF is a regular (public) hidden property of FigureContainer
win = 
  webwindow with properties:
 
                             URL: 'http://localhost:31417/toolbox/matlab/uitools/uifigureappjs/component…'
                           Title: 'Yair'
                            Icon: 'C:\Program Files\Matlab\R2016a\toolbox\matlab\uitools\uicomponents\re…'
                        Position: [86 39 584 465]
     CustomWindowClosingCallback: @(o,e)this.Model.hgclose()
    CustomWindowResizingCallback: @(event,data)resizeRequest(this,event,data)
                  WindowResizing: []
                   WindowResized: []
                     FocusGained: []
                       FocusLost: []
                DownloadCallback: []
        PageLoadFinishedCallback: []
           MATLABClosingCallback: []
      MATLABWindowExitedCallback: []
             PopUpWindowCallback: []
             RemoteDebuggingPort: 0
                      CEFVersion: '3.2272.2072'
                 ChromiumVersion: '41.0.2272.76'
                   isWindowValid: 1
               isDownloadingFile: 0
                         isModal: 0
                  isWindowActive: 1
                   isAlwaysOnTop: 0
                     isAllActive: 1
                     isResizable: 1
                         MaxSize: []
                         MinSize: []
 
>> win.URL
ans =
http://localhost:31417/toolbox/matlab/uitools/uifigureappjs/componentContainer.html?channel=/uicontainer/393ed66a-5e34-41f3-8ac0-0b0f3b0738cd&snc=5C2353

An alternative way to get the webwindow is via the list of all webwindows stored by a central webwindowmanager:

webWindows = matlab.internal.webwindowmanager.instance.findAllWebwindows();  % manager method returning an array of all open webwindows
webWindows = matlab.internal.webwindowmanager.instance.windowList;           % equivalent alternative via manager's windowList property

Note that the controller, container and webwindow class objects, like most Matlab MCOS objects, have internal (hidden) properties/methods that you can explore. For example:

>> getundoc(win)
ans = 
                   Channel: [1x1 asyncio.Channel]
       CustomEventListener: [1x1 event.listener]
           InitialPosition: [100 100 600 400]
    JavaScriptReturnStatus: []
     JavaScriptReturnValue: []
     NewWindowBeingCreated: 0
          NewWindowCreated: 1
           UpdatedPosition: [86 39 584 465]
              WindowHandle: 2559756
                    newURL: 'http://localhost:31417/toolbox/matlab/uitools/uifigureappjs/componentContai…'

Using webwindow for figure-level customizations

We can use the methods of this webwindow object as follows:

win.setAlwaysOnTop(true);   % always on top of other figure windows (a.k.a. AOT)
 
win.hide();
win.show();
win.bringToFront();
 
win.minimize();
win.maximize();
win.restore();
 
win.setMaxSize([400,600]);  % enables resizing up to this size but not larger (default=[])
win.setMinSize([200,300]);  % enables resizing down to this size but not smaller (default=[])
win.setResizable(false);
 
win.setWindowAsModal(true);
 
win.setActivateCurrentWindow(false);  % disable interaction with this entire window
win.setActivateAllWindows(false);     % disable interaction with *ALL* uifigure (but not Java-based) windows
 
result = win.executeJS(jsStr, timeout);  % run JavaScript

In addition to these methods, we can set callback functions to various callbacks exposed by the webwindow as regular properties (too bad that some of their names [like the class name itself] don’t follow Matlab’s standard naming convention, in this case by appending “Fcn” or “Callback”):

win.FocusGained = @someCallbackFunc;
win.FocusLost = @anotherCallbackFunc;

In summary, while the possible customizations to Java-based figure windows are more extensive, the webwindow methods appear to cover most of the important ones. Since these functionalities (maximize/minimize, AOT, disable etc.) are now common to both the Java and web-based figures, I really hope that MathWorks will create fully-documented figure properties/methods for them. Now that there is no longer any question whether these features will be supported by the future technology, and since there is no question as to their usefulness, there is really no reason not to officially support them in both figure types. If you feel the same as I do, please let MathWorks know about this – if enough people request this, MathWorks will be more likely to add these features to one of the upcoming Matlab releases.

Warning: the internal implementation is subject to change across releases, so be careful to make your code cross-release compatible whenever you rely on one of Matlab’s internal objects.

Note that I labeled this post as “part 1” – I expect to post additional articles on uifigure customizations in upcoming years.

>> hListbox = uicontrol('Style','list', 'String',{'item #1','item #2','item #3','item #4','item #5','item #6'});
>> set(hListbox,'Value',[]);
Warning: Single-selection 'listbox' control requires a scalar Value.
Control will not be rendered until all of its parameter values are valid
(Type "warning off MATLAB:hg:uicontrol:ValueMustBeScalar" to suppress this warning.)

The reader’s question was whether there is a way to bypass this limitation so that no listbox item will be selected. The answer to this question was provided by MathWorker Steve(n) Lord. Steve is a very long-time benefactor of the Matlab community with endless, tireless, and patient advise to queries small and large (way beyond the point that would have frustrated mere mortals). Steve pointed out that by default, Matlab listboxes only enable a single selection – not more and not less. However, when the listbox’s Max value is set to be >1, the listbox enables multiple-items selection, meaning that Value accepts and reports an array of item indices, and there is nothing that prevents this array from being empty (meaning no items selected):

>> hListbox = uicontrol('Style','list', 'Max',2, 'String',{'item #1','item #2','item #3','item #4','item #5','item #6'});
>> set(hListbox,'Value',[]);  % this is ok - listbox appears with no items selected

Note: actually, the listbox checks the value of Max–Min, but by default Min=0 and there is really no reason to modify this default value, just Max.

While this makes sense if you think about it, the existing documentation makes no mention of this fact:

The Max property value helps determine whether the user can select multiple items in the list box simultaneously. If Max – Min > 1, then the user can select multiple items simultaneously. Otherwise, the user cannot select multiple items simultaneously. If you set the Max and Min properties to allow multiple selections, then the Value property value can be a vector of indices.

Some readers might think that this feature is not really undocumented, since it does not directly conflict with the documentation text, but then so are many other undocumented aspects and features on this blog, which are not mentioned anywhere in the official documentation. I contend that if this feature is officially supported, then it deserves an explicit sentence in the official documentation.

However, the original CSSM reader wanted to preserve Matlab’s single-selection model while enabling deselection of an item. Basically, the reader wanted a selection model that enables 0 or 1 selections, but not 2 or more. This requires some tweaking using the listbox’s selection callback:

set(hListbox,'Callback',@myCallbackFunc);
 
...
function test(hListbox, eventData)
   value = get(hListbox, 'Value');
   if numel(value) > 1
       set(hListbox, 'Value', value(1));
   end
end

…or a callback-function version that is a bit better because it takes the previous selection into account and tries to set the new selection to the latest-selected item (this works in most cases, but not with shift-clicks as explained below):

function myCallbackFunc(hListbox, eventData)
   lastValue = getappdata(hListbox, 'lastValue');
   value = get(hListbox, 'Value');
   if ~isequal(value, lastValue)
      value2 = setdiff(value, lastValue);
      if isempty(value2)
         setappdata(hListbox, 'lastValue', value);
      else
         value = value2(1);  % see quirk below
         setappdata(hListbox, 'lastValue', value);
         set(hListbox, 'Value', value);
      end
   end
end

This does the job of enabling only a single selection at the same time as allowing the user to interactively deselect that item (by ctrl-clicking it).

There’s just a few quirks: If the user selects a block of items (using shift-click), then only the second-from-top item in the block is selected, rather than the expected last-selected item. This is due to line #9 in the callback code which selects the first value. Matlab does not provide us with information about which item was clicked, so this cannot be helped using pure Matlab. Another quirk that cannot easily be solved using pure Matlab is the flicker that occurs when the selection changes and is then corrected by the callback.

We can solve both of these problems using the listbox’s underlying Java component, which we can retrieve using my findjobj utility:

% No need for the standard Matlab callback now
set(hListbox,'Callback',[]);
 
% Get the underlying Java component peer
jScrollPane = findjobj(h);
jListbox = jScrollPane.getViewport.getView;
jListbox = handle(jListbox,'CallbackProperties');  % enable callbacks
 
% Attach our callback to the listbox's Java peer
jListbox.ValueChangedCallback = {@myCallbackFunc, hListbox};
 
...
function myCallbackFunc(jListbox, eventData, hListbox)
   if numel(jListbox.getSelectedIndices) > 1
      set(hListbox, 'Value', jListbox.getLeadSelectionIndex+1);  % +1 because Java indices start at 0
   end
end

We can use a similar mechanism to control other aspects of selection, for example to enable only up to 3 selections but no more etc.

We can use this underlying Java component peer for a few other useful selection-related hacks: First, we can use the peer’s RightSelectionEnabled property or setRightSelectionEnabled() method to enable the user to select by right-clicking listbox items (this is disabled by default):

jListbox.setRightSelectionEnabled(true);  % false by default
set(jListbox,'RightSelectionEnabled',true);  % equivalent alternative

A similarly useful property is DragSelectionEnabled (or the corresponding setDragSelectionEnabled() method), which is true by default, and controls whether the selection is extended to other items when the mouse drags an item up or down the listbox.

Finally, we can control whether in multi-selection mode we enable the user to only select a single contiguous block of items, or not (which is Matlab’s default behavior). This is set via the SelectionMode property (or associated setSelectionMode() method), as follows:

jListbox.setSelectionMode(javax.swing.ListSelectionModel.SINGLE_INTERVAL_SELECTION);
jListbox.setSelectionMode(1);  % equivalent alternative (less maintainable/readable, but simpler)

SINGLE_SELECTION =0	SINGLE_INTERVAL_SELECTION =1	MULTIPLE_INTERVAL_SELECTION =2
(Matlab default for Max=1)	(only possible with Java)	(Matlab default for Max>1)

Additional listbox customizations can be found in related posts on this blog (see links below), or in section 6.6 of my Matlab-Java Programming Secrets book (which is still selling nicely almost five years after its publication, to the pleasant surprise of my publisher…).

Matlab Expo 2016 keynote presentation

The overall effect can be dramatic: The performance (speed) difference between a sub-optimal and optimized parfor‘ed code can be up to a full order of magnitude, depending on the specific situation. Naturally, to use any of today’s tips, you need to have MathWorks’ Parallel Computing Toolbox (PCT).

Before diving into the technical details, let me say that MathWorks has extensive documentation on PCT. In today’s post I will try not to reiterate the official tips, but rather those that I have not found mentioned elsewhere, and/or are not well-known (my apologies in advance if I missed an official mention of one or more of the following). Furthermore, I limit myself only to parfor in this post: much can be said about spmd, GPU and other parallel constructs, but not today.

parpool(numCores)

The first tip is to not [always] use the default number of workers created by parpool (or matlabpool in R2013a or earlier). By default, Matlab creates as many workers as logical CPU cores. On Intel CPUs, the OS reports two logical cores per each physical core due to hyper-threading, for a total of 4 workers on a dual-core machine. However, in many situations, hyperthreading does not improve the performance of a program and may even degrade it (I deliberately wish to avoid the heated debate over this: you can find endless discussions about it online and decide for yourself). Coupled with the non-negligible overhead of starting, coordinating and communicating with twice as many Matlab instances (workers are headless [=GUI-less] Matlab processes after all), we reach a conclusion that it may actually be better in many cases to use only as many workers as physical (not logical) cores.

I know the documentation and configuration panel seem to imply that parpool uses the number of physical cores by default, but in my tests I have seen otherwise (namely, logical cores). Maybe this is system-dependent, and maybe there is a switch somewhere that controls this, I don’t know. I just know that in many cases I found it beneficial to reduce the number of workers to the actual number of physical cores:

p = parpool;     % use as many workers as logical CPUs (4 on my poor laptop...)
p = parpool(2);  % use only 2 parallel workers

Of course, this can vary greatly across programs and platforms, so you should test carefully on your specific setup. I suspect that for the majority of Matlab programs it would turn out that using the number of physical cores is better.

It would of course be better to dynamically retrieve the number of physical cores, rather than hard-coding a constant value (number of workers) into our program. We can get this value in Matlab using the undocumented feature(‘numcores’) function:

numCores = feature('numcores');
p = parpool(numCores);

Running feature(‘numcores’) without assigning its output displays some general debugging information:

>> feature('numcores')
MATLAB detected: 2 physical cores.
MATLAB detected: 4 logical cores.
MATLAB was assigned: 4 logical cores by the OS.
MATLAB is using: 2 logical cores.
MATLAB is not using all logical cores because hyper-threading is enabled.
ans =
     2

Naturally, this specific tip is equally valid for both parfor loops and spmd blocks, since both of them use the pool of workers started by parpool.

Running separate code in parfor loops

The conventional wisdom is that parfor loops (and loops in general) can only run a single code segment over all its iterations. Of course, we can always use conditional constructs (such as if or switch) based on the data. But what if we wanted some workers to run a different code path than the other workers? In spmd blocks we could use a conditional based on the labindex value, but unfortunately labindex is always set to the same value 1 within parfor loops. So how can we let worker A run a different code path than worker B?

An obvious answer is to create a parfor loop having as many elements as there are separate code paths, and use a switch-case mechanism to run the separate paths, as follows:

% Naive implementation example - do NOT use!
parfor idx = 1 : 3
   switch idx
      case 1,  result{1} = fun1(data1, data2);
      case 2,  result{2} = fun2(data3, data4, data5);
      case 3,  result{3} = fun3(data6);
   end
end

There are several problems with this naive implementation. First, it unnecessarily broadcasts all the input data to all workers (more about this issue below). Secondly, it appears clunky and too verbose. A very nice extension of this mechanism, posted by StackOverflow heavyweight Jonas, uses indexed arrays of function handles and input args, thereby solving both problems:

funcList = {@fun1, @fun2, @fun3};
dataList = {data1, data2, data3};  %# or pass file names 
parfor idx = 1 : length(funcList)
    result{idx} = funcList{idx}(dataList{idx});
end

Reduce the amount of broadcast data

It is often easy, too-easy, to convert for loops into parfor loops. In many cases, all we need to do is to add the “par” prefix to the for keyword and we’re done (assuming we have no incompatibly-used variables that should be converted into sliced variables etc.). This transformation was intentionally made simple by MathWorks (which is great!). On the other hand, it also hides a lot under the hood. One of the things that is often overlooked in such simple loop transformations is that a large part of the data used within the loop needs to be copied (broadcast) to each of the workers separately. This means that each of the data items needs to be serialized (i.e., copied in memory), packaged, communicated to and accepted by each of the workers. This can mean a lot of memory, networking bandwidth and time-consuming. It can even mean thrashing to hard-disk in case the number of workers times the amount of transferred data exceeds the available RAM. For example, if we have 10GB available RAM and try to communicate 3GB to 4 workers, we will not have enough RAM and the OS will start swapping to hard-disk. This will kill performance and Matlab will appear “hung” and will need to be hard-killed.

You might think that it would be very difficult to reach the RAM limit, but in fact it can be far too easy when you consider the multiplication by the number of workers, and the fact that each worker uses 1+GB of memory just for its MATLAB process, even before the data, and all this in addition to the parent (client) Matlab process. That’s a lot of GBs flying around…

Moreover, it’s enough for one small part of a Matlab struct or array to be used within the parfor loop for the entire Matlab construct to be broadcast to all workers. For example, a very common use-case is to store program data, both raw and processed, within a simple Matlab struct. Let’s say that we have data.raw and data.processed and within the loop we only need data.processed – the entire data variable (which might include many GBs due to the raw data) is broadcast, although the loop’s code only needs data.processed. In such cases, it makes sense to separate the broadcast data into standalone variables, and only use them within the loop:

data.raw = ...
data.processed = ...
 
% Inefficient variant:
parfor idx = 1 : N
   % do something with data.processed
end
 
% This is better:
processedData = data.processed;
parfor idx = 1 : N
   % do something with processedData
end

Moreover, if you can convert a broadcast variable into a sliced one, this would be even better: in this case each worker will only be communicated its small share (“slice”) of the entire data, rather than a full copy of the entire data.

All this would of course be much simpler if Matlab’s computational engine was multi-threaded, since then PCT could be implemented using lightweight threads rather than heavyweight processes. The memory and communication overheads would then be drastically reduced and performance would improve significantly. Unfortunately, Matlab’s computational engine is [still] single-threaded, preventing this. Hopefully Matlab’s new engine (which debuted in R2015b) will enable true multithreading at some future release. PCT will still need to retain an option of using headless worker processes to run on multiple machines (i.e., distributed/grid/cloud computing), but single-machine parallelization should employ multithreading instead.

Additional speedup tips can be found in my book “Accelerating MATLAB Performance“.

Do you have some other important parfor tips that you found useful? If so, please post them in a comment below.

% Undocumented syntax - works well since at least R2010a (possibly earlier)
classdef Packet
    properties
        PacketType@char
        HeaderLength@uint16
        PayloadLength@uint16 scalar = uint16(0);  % initial value
        PacketData@uint8 vector
    end
end

In the recent release of Matlab R2016a, a similar feature have finally become fully supported and documented. The corresponding snippet above would look something like this:

% Documented syntax - only works in R2016a or newer
classdef Packet
    properties
        PacketType char
        HeaderLength uint16
        PayloadLength uint16 = uint16(0);  % initial value
        PacketData uint8
    end
end

Unfortunately, I dislike the new documented functionality, so I didn’t feel like promoting it in this blog when it came out. But since a blog reader mentioned it a few days ago, I wanted to come out publicly with my opinion and a detailed explanation.

If you look closely at the code snippets above, you will notice two important differences:

1. The “@” symbol was replaced with a space
2. The dimensionality and complexity cannot be specified

The new syntax has some drawbacks compared to the previous (undocumented) one:

1. Backward compatibility – We can run the older (undocumented) syntax on any Matlab release since who-knows-when (at least as far back as R2010a [tested], and possibly older releases [untested]), including the very latest R2016a. On the other hand, the new (documented) syntax will only work on R2016a and will crash the program if you try to run it in older releases. This is not even something that you can catch with a try-catch block – the class will simply not load on any older Matlab release. If you need your code to run on older releases in addition to 16a, you have no choice other than to use the older syntax.
2. Dimensionality – the new syntax, unlike the undocumented syntax, does not enable users to limit the data dimensionality (scalar/vector/array). This is a very important feature for program robustness and maintainability. Complexity is another type limitation that is missing, although it is less important than the dimensionality. And just in case you were wondering – the new syntax does not accept the additional scalar, vector, matrix and complex attributes like the older syntax; using them with the new syntax evokes an error.
3. Cross compatibility – it is very confusing to users coming to Matlab from other programming languages, all of which (without any important exception) place the type name to the LEFT of the identifier name, not to its RIGHT. People coding in both Matlab and Java/C/C++ would easily get confused and frustrated.
4. Consistency – despite what I hoped, the new syntax still does not apply to function input args: we cannot (AFAIK) limit the input/output args of methods/functions in the same way that we can limit properties. If there’s a way to do this, I’d be delighted to learn (this comment may indicate that it is work in progress). It is true that this feature is not a drawback of the new syntax compared to the older one, since the old syntax didn’t have it either (AFAIK). But I would have expected a documented feature to be consistent across the Matlab language (or at least across the MCOS subset), and unfortunately the new feature fails this test.

In fact, aside from the fact that the new syntax is documented, I can see no advantages that it offers over the older syntax, only disadvantages. Or am I missing something? Please do tell if you see any important advantages that I’ve missed.

Luckily for us, the old syntax remains operational, side-by-side with the new one. This enables us to keep running our existing code without worrying [too much] that it might break in R2016a. Maybe the new syntax will grow on me (or improve) in upcoming years, but for the time being I see no benefit in switching away from the @ syntax.

For the past few years, I hoped that the property typing feature will become documented and that it will be a continuation of the undocumented syntax rather than what eventually aired. I’m afraid it’s too late to revert it now that it has… Realistically speaking, the best we can hope for now is for the older syntax to remain operational, and not be withdrawn in some future Matlab release. Making the undocumented syntax documented as-is would be great, but I’m afraid it is unrealistic given the new circumstances.

I’m sorry if I take the wind off MathWorks’ sails a bit here, but MathWorks knows that it can count on me to speak my mind without bullshit. Sometimes for the good, sometimes not. All in good spirit and the common interest of improving Matlab over time. No offence intended – it’s just my personal opinion after all.

In my opinion this is one of those rare cases where the developers obviously intended to make something better but eventually came out with something worse. They should have stuck to what was. After all, the first and foremost rule of engineering is, and always was:

The basic idea is simple: whenever the user types in the search box, a Matlab callback function checks the data for the search term. If one or more matches are found then the searchbox’s background remains white, otherwise it is colored yellow to highlight the term. When the user presses <Enter>, the search action is triggered to highlight the term in the data, and any subsequent press of <Enter> will highlight the next match (cycling back at the top as needed). Very simple and intuitive:

Interactive search-box in Matlab figure toolbar

In my specific case, the search action (highlighting the search term in the data) involved doing a lot of work: updating multiple charts and synchronizing row selection in several connected uitables. For this reason, I chose not to do this action interactively (upon each keypress in the search box) but rather only upon clicking <Enter>. In your implementation, if the search action is simpler and faster, you could do it interactively for an even more intuitive effect.

Technical components

The pieces of today’s post were already discussed separately on this website, but never shown together as I will do today:

• The search box component (com.mathworks.widgets.SearchTextField) was discussed in last year’s article on auto-completion widgets
• I showed how to add custom controls to the figure toolbar in a 2009 post (time flies!)
• I discussed controlling toolbar components’ size in another old post
• I discussed my findjobj utility, used for accessing the underlying Java components of Matlab uicontrols in another article
• I discussed Matlab’s use of EDT in a dedicated article on the subject back in 2010 (and I’ll have more to say about this subject next week)
• Finally, I discussed how to trap Java control events in Matlab in separate articles here and here

Adding a search-box to the figure toolbar

As a first step, let’s create the search-box component and add it to our figure’s toolbar:

% First, create the search-box component on the EDT, complete with invokable Matlab callbacks:
jSearch = com.mathworks.widgets.SearchTextField('Symbol');  % 'Symbol' is my default search prompt
jSearchPanel = javaObjectEDT(jSearch.getComponent);  % this is a com.mathworks.mwswing.MJPanel object
jSearchPanel = handle(jSearchPanel, 'CallbackProperties');  % enable Matlab callbacks
 
% Now, set a fixed size for this component so that it does not resize when the figure resizes:
jSize = java.awt.Dimension(100,25);  % 100px wide, 25px tall
jSearchPanel.setMaximumSize(jSize)
jSearchPanel.setMinimumSize(jSize)
jSearchPanel.setPreferredSize(jSize)
jSearchPanel.setSize(jSize)
 
% Now, attach the Matlab callback function to search box events (key-clicks, Enter, and icon clicks):
jSearchBox = handle(javaObjectEDT(jSearchPanel.getComponent(0)), 'CallbackProperties');
set(jSearchBox, 'ActionPerformedCallback', {@searchSymbol,hFig,jSearchBox})
set(jSearchBox, 'KeyPressedCallback',      {@searchSymbol,hFig,jSearchBox})
 
jClearButton = handle(javaObjectEDT(jSearchPanel.getComponent(1)), 'CallbackProperties');
set(jClearButton, 'ActionPerformedCallback', {@searchSymbol,hFig,jSearchBox})
 
% Now, get the handle for the figure's toolbar:
hToolbar = findall(hFig,'tag','FigureToolBar');
jToolbar = get(get(hToolbar,'JavaContainer'),'ComponentPeer');  % or: hToolbar.JavaContainer.getComponentPeer
 
% Now, justify the search-box to the right of the toolbar using an invisible filler control
% (first add the filler control to the toolbar, then the search-box control):
jFiller = javax.swing.Box.createHorizontalGlue;  % this is a javax.swing.Box$Filler object
jToolbar.add(jFiller,      jToolbar.getComponentCount);
jToolbar.add(jSearchPanel, jToolbar.getComponentCount);
 
% Finally, refresh the toolbar so that the new control is displayed:
jToolbar.revalidate
jToolbar.repaint

Search action callback functionality

Now that the control is displayed in the toolbar, let’s define what our Matlab callback function searchSymbol() does. Remember that this callback function is invoked whenever any of the possible events occur: keypress, <Enter>, or clicking the search-box’s icon (typically the “x” icon, to clear the search term).

We first reset the search-box appearance (foreground/background colors), then we check the search term (if non-empty). Based on the selected tab, we search the corresponding data table’s symbol column(s) for the search term. If no match is found, we highlight the search term by setting the search-box’s text to be red over yellow. Otherwise, we change the table’s selected row to the next match’s row index (i.e., the row following the table’s currently-selected row, cycling back at the top of the table if no match is found lower in the table).

Reading and updating the table’s selected row requires using my findjobj utility – for performance considerations the jTable handle should be cached (perhaps in the hTable’s UserData or ApplicationData):

% Callback function to search for a symbol
function searchSymbol(hObject, eventData, hFig, jSearchBox)
    try
        % Clear search-box formatting
        jSearchBox.setBackground(java.awt.Color.white)
        jSearchBox.setForeground(java.awt.Color.black)
        jSearchBox.setSelectedTextColor(java.awt.Color.black)
        jSearchBox.repaint
 
        % Search for the specified symbol in the data table
        symbol = char(jSearchBox.getText);
        if ~isempty(symbol)
            handles = guidata(hFig);
            hTab = handles.hTabGroup.SelectedTab;
            colOffset = 0;
            forceCol0 = false;
            switch hTab.Title
                case 'Scanning'
                    hTable = handles.tbScanResults;
                    symbols = cell(hTable.Data(:,1));
                case 'Correlation'
                    hTable = handles.tbCorrResults;
                    symbols = cell(hTable.Data(:,1:2));
                case 'Backtesting'
                    hTab = handles.hBacktestTabGroup.SelectedTab;
                    hTable = findobj(hTab, 'Type','uitable', 'Tag','results');
                    pairs = cell(hTable.Data(:,1));
                    symbols = cellfun(@(c)strsplit(c,'/'), pairs, 'uniform',false);
                    symbols = reshape([symbols{:}],2,[])';
                    forceCol0 = true;
                case 'Trading'
                    hTable = handles.tbTrading;
                    symbols = cell(hTable.Data(:,2:3));
                    colOffset = 1;
                otherwise  % ignore
                    return
            end
            if isempty(symbols)
                return
            end
            [rows,cols] = ind2sub(size(symbols), find(strcmpi(symbol,symbols)));
            if isempty(rows)
                % Not found - highlight the search term
                jSearchBox.setBackground(java.awt.Color.yellow)
                jSearchBox.setForeground(java.awt.Color.red)
                jSearchBox.setSelectedTextColor(java.awt.Color.red)
                jSearchBox.repaint
            elseif isa(eventData, 'java.awt.event.KeyEvent') && isequal(eventData.getKeyCode,10)
                % Found with <Enter> event - highlight the relevant data row
                jTable = findjobj(hTable);
                try jTable = jTable.getViewport.getView; catch, end  % in case findjobj returns the containing scrollpane rather than the jTable
                [rows, sortedIdx] = sort(rows);
                cols = cols(sortedIdx);
                currentRow = jTable.getSelectedRow + 1;
                idx = find(rows>currentRow,1);
                if isempty(idx),  idx = 1;  end
                if forceCol0
                    jTable.changeSelection(rows(idx)-1, 0, false, false)
                else
                    jTable.changeSelection(rows(idx)-1, cols(idx)-1+colOffset, false, false)
                end
                jTable.repaint
                jTable.getTableHeader.repaint
                jTable.getParent.getParent.repaint
                drawnow
            end
        end
    catch
        % never mind - ignore
    end
end

That’s all there is to it. In my specific case, changing the table’s selected row cased an immediate trigger that updated the associated charts, synchronized the other data tables and did several other background tasks.

What about the new web-based uifigure?

The discussion above refers only to traditional Matlab figures (both HG1 and HG2), not to the new web-based (AppDesigner) uifigures that were officially introduced in R2016a (I wrote about it last year).

AppDesigner uifigures are basically webpages rather than desktop windows (JFrames). They use an entirely different UI mechanism, based on HTML webpages served from a localhost webserver, using the DOJO Javascript toolkit for visualization and interaction, rather than Java Swing as in the existing JFrame figures. The existing figures still work without change, and are expected to continue working alongside the new uifigures for the foreseeable future. I’ll discuss the new uifigures in separate future posts (in the meantime you can read a bit about them in my post from last year).

I suspect that the new uifigures will replace the old figures at some point in the future, to enable a fully web-based (online) Matlab. Will this happen in 2017 or 2027 ? – your guess is as good as mine, but my personal guesstimate is around 2018-2020.