Pure Matlab – Undocumented Matlab https://undocumentedmatlab.com Charting Matlab's unsupported hidden underbelly Thu, 19 Jul 2018 20:59:48 +0000 en-US hourly 1 https://wordpress.org/?v=4.4.1 Plot legend customizationhttps://undocumentedmatlab.com/blog/plot-legend-customization https://undocumentedmatlab.com/blog/plot-legend-customization#comments Thu, 12 Jul 2018 14:11:40 +0000 https://undocumentedmatlab.com/?p=7744

Related posts:
1. Plot LineSmoothing property LineSmoothing is a hidden and undocumented plot line property that creates anti-aliased (smooth unpixelized) lines in Matlab plots...
2. Multi-column (grid) legend This article explains how to use undocumented axes listeners for implementing multi-column plot legends...
3. Controlling plot data-tips Data-tips are an extremely useful plotting tool that can easily be controlled programmatically....
4. getundoc – get undocumented object properties getundoc is a very simple utility that displays the hidden (undocumented) properties of a specified handle object....

]]>
Three years ago I explained how we can use a couple of undocumented hidden properties of the legend in order to add a legend title (the legend object had no Title property back then – this was only added in a later Matlab release, perhaps as a result of my post). Today I will expand on that article by explaining the plot legend’s internal graphics hierarchy, how we can access each of these components, and then how this information could be used to customize the separate legend components. Note that the discussion today is only relevant for HG2 legends (i.e. R2014b or newer).

hold all; hLine1 = plot(1:5); hLine2 = plot(2:6); hLegend = legend([hLine1,hLine2], 'Location','SouthEast'); hLegend.Title.String = 'MyLegend';

Standard Matlab legend

This legend is composed of the following visible internal components, which can be customized separately:

Id in screenshotAccessed viaObject typeDescriptionImportant properties
1hLegend.TitleTextTitle of the legendVisible, String, Color, FontSize, FontWeight.
2hLegend.TitleSeparatorLineStripSeparator line between title and legend entries. Only appears when title is set.Visible, LineStyle, LineWidth, ColorData (4×1 uint8)
3hLegend.BoxEdgeLineLoopBox (border) line around the entire legend (including title)Visible, LineStyle, LineWidth, ColorData (4×1 uint8)
4hLegend.EntryContainer.NodeChildren(2)LegendEntryEntry row in the legend, corresponding to hLine1Icon, Label, Object (line object in main axes)
5hLegend.EntryContainer.NodeChildren(1)LegendEntryEntry row in the legend, corresponding to hLine2Icon, Label, Object (line object in main axes)
6hLegend.EntryContainer.NodeChildren(1).LabelTextLabel of legend entryVisible, String, Color, FontSize, FontWeight
7hLegend.EntryContainer.NodeChildren(1).IconLegendIconIcon/marker of legend entryVisible, Transform.Children.Children (LineStrip object)

A pivotal object of the legend group are the LegendEntry items, one per legend row:

>> hLegendEntry = hLegend.EntryContainer.NodeChildren(1); >> get(hLegendEntry) Children: [3×1 Graphics] Color: [0 0 0] Dirty: 0 FontAngle: 'normal' FontName: 'Helvetica' FontSize: 8 FontWeight: 'normal' HandleVisibility: 'on' HitTest: 'on' Icon: [1×1 LegendIcon] Index: 0 Interpreter: 'tex' Label: [1×1 Text] LayoutInfo: [1×1 matlab.graphics.illustration.legend.ItemLayoutInfo] Legend: [1×1 Legend] Listener: [1×1 event.listener] Object: [1×1 Line] Overlay: [1×1 TriangleStrip] OverlayAlpha: 0.65 Parent: [1×1 Group] PeerVisible: 'on' PickableParts: 'visible' Selected: 'off' SelectionHighlight: 'on' Visible: 'on' VisibleListener: [1×1 event.proplistener]

Each LegendEntry contains a back-reference to the original graphics object. In my example above, hLegend.EntryContainer.NodeChildren(2).Object == hLine1, and hLegend.EntryContainer.NodeChildren(2).Object == hLine1. Note how the default legend entries order is the reverse of the order of creation of the original graphics objects. Naturally, we can modify this order by creating the legend py passing it an array of handles that is ordered differently (see the documentation of the legend function).

To get all the original graphic objects together, in a single array, we could use one of two mechanisms (note the different order of the returned objects):

% Alternative #1 >> [hLegend.EntryContainer.NodeChildren.Object]' ans = 2×1 Line array:   Line (data2) Line (data1)   % Alternative #2 >> hLegend.PlotChildren ans = 2×1 Line array:   Line (data1) Line (data2)

For some reason, accessing the displayed graphic line in LegendEntry‘s Icon is not simple. For example, the LineStrip object that corresponds to hLine2 can be gotten via:

hLegendEntry = hLegend.EntryContainer.NodeChildren(1); hLegendIconLine = hLegendEntry.Icon.Transform.Children.Children; % a LineStrip object in our example

I assume that this was done to enable non-standard icons for patches and other complex objects (in which case the displayed icon would not necessarily be a LineStrip object). In the case of a line with markers, for example, hLegendIconLine would be an array of 2 objects: a LineStrip object and a separate Marker object. Still, I think that a direct reference in a hLegend.EntryContainer.NodeChildren(1).Icon property would have helped in 99% of all cases, so that we wouldn’t need to pass through the Transform object.

Anyway, once we have this object reference(s), we can modify its/their properties. In the case of a LineStrip this includes LineStyle, LineWidth, ColorData (4×1 uint8), and VertexData (which controls position/length):

>> get(hLegendIconLine(end)) % LineStrip AlignVertexCenters: 'on' AmbientStrength: 0.3 ColorBinding: 'object' ColorData: [4×1 uint8] ColorType: 'truecolor' DiffuseStrength: 0.6 HandleVisibility: 'on' HitTest: 'off' Layer: 'middle' LineCap: 'none' LineJoin: 'round' LineStyle: 'solid' LineWidth: 0.5 NormalBinding: 'none' NormalData: [] Parent: [1×1 Group] PickableParts: 'visible' SpecularColorReflectance: 1 SpecularExponent: 10 SpecularStrength: 0.9 StripData: [] Texture: [0×0 GraphicsPlaceholder] VertexData: [3×2 single] VertexIndices: [] Visible: 'on' WideLineRenderingHint: 'software'

and in the presense of markers:

>> get(hLegendIconLine(1)) % Marker EdgeColorBinding: 'object' EdgeColorData: [4×1 uint8] EdgeColorType: 'truecolor' FaceColorBinding: 'object' FaceColorData: [] FaceColorType: 'truecolor' HandleVisibility: 'on' HitTest: 'off' Layer: 'middle' LineWidth: 0.5 Parent: [1×1 Group] PickableParts: 'visible' Size: 6 SizeBinding: 'object' Style: 'circle' VertexData: [3×1 single] VertexIndices: [] Visible: 'on'

An additional undocumented legend property that is of interest is ItemTokenSize. This is a 2-element numeric array specifying the minimal size of the legend entries’ icon and label. By default hLegend.ItemTokenSize == [30,18], but we can either expand or shrink the icons/labels by setting different values. For example:

hLegend.ItemTokenSize == [10,1]; % shrink legend icons and labels

Note that regardless of the amount that we specify, the actual amount that will be used will be such that all legend labels appear.
Fun: try playing with negative values for the icon and the label and see what happens

Have you come across any other interesting undocumented aspect of Matlab legends? If so, then please share it in a comment below.

]]>
https://undocumentedmatlab.com/blog/plot-legend-customization/feed 3
Sliders in Matlab GUI – part 2https://undocumentedmatlab.com/blog/sliders-in-matlab-gui-part-2 https://undocumentedmatlab.com/blog/sliders-in-matlab-gui-part-2#comments Thu, 05 Jul 2018 11:40:56 +0000 https://undocumentedmatlab.com/?p=7728

Related posts:
1. Context-Sensitive Help Matlab has a hidden/unsupported built-in mechanism for easy implementation of context-sensitive help...
2. Customizing uitree This article describes how to customize Matlab GUI tree controls created using the undocumented uitree function...
3. Date selection components The JIDE package, pre-bundled in Matlab, contains several GUI controls for selecting dates - this article explains how they can be used...
4. uitree This article describes the undocumented Matlab uitree function, which displays data in a GUI tree component...

]]>
Exactly 3 years ago I posted about various alternatives for embedding sliders in Matlab GUI. Today I will follow up on that post with a description of yet another undocumented builtin alternative – controllib.widget.Slider. A summary of the various alternatives can be seen in the following screenshot:

Slider alternatives in Matlab GUI

The controllib.widget.Slider component is a class in Matlab’s internal controllib package (last week I discussed a different utility function in this package, controllib.internal.util.hString2Char).

controllib.widget.Slider accepts 3 input arguments: containing figure handle, position in pixels, and data values. For example:

>> hSlider = controllib.widget.Slider(gcf, [10,10,100,50], 5:25) hSlider = Slider with properties:   Data: [6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25] Index: 11 Value: 15 FontSize: 8 Position: [10 10 100 50]

This creates an invisible axes at the specified figure location and displays graphic axes objects that provide the appearance of the slider. When you move the slider’s knob, or click its centerline or arrows (“Steppers”), the slider’s value changes accordingly.

You can attach a callback function to the slider as follows:

myCallback = @(h,e) disp(h.Value); % as an example addlistener(hSlider, 'ValueChanged', myCallback);

Note that controllib.widget.Slider is based on pure-Matlab code and fully-supported functionality. The Matlab source code is available (%matlabroot%/toolbox/shared/controllib/graphics/+controllib/+widget/Slider.m) and quite readable. So while it does not actually work with the new web-based uifigures, is should be relatively easy to adapt the code so that this component could be displayed in such uifigures.

Below is a script to recreate the screenshot above. Note the two alternative mechanisms for setting properties (Java setter-method notation, and HG set notation):

hFig = figure('Color','w');   % 1. controllib.widget.Slider hSlider1 = controllib.widget.Slider(hFig, [10,10,100,50], 1:20); hSlider1.Value = 12;   % 2. uicontrol hSlider2 = uicontrol('style','slider', 'units','pixels', 'pos',[10,80,100,20], 'Min',0', 'Max',20, 'Value',12);   % 3. JScrollBar jSlider3 = javaObjectEDT(javax.swing.JScrollBar); jSlider3.setOrientation(jSlider3.HORIZONTAL); % Java setter-method notation set(jSlider3, 'VisibleAmount',1, 'Minimum',0, 'Maximum',20, 'Value',12); % HG set notation [hSlider3, hContainer3] = javacomponent(jSlider3, [10,130,100,20], hFig);   % 4. JSlider #1 jSlider4 = javaObjectEDT(javax.swing.JSlider(0,20,12)) jSlider4.setBackground(java.awt.Color.white); % Java setter-method notation set(jSlider4, 'MinorTickSpacing',1, 'MajorTickSpacing',4, 'SnapToTicks',true, 'PaintLabels',true); % HG set notation [hSlider4, hContainer4] = javacomponent(jSlider4, [10,180,100,30], hFig);   % 5. JSlider #2 jSlider5 = javaObjectEDT(javax.swing.JSlider(0,20,12)) jSlider5.setBackground(java.awt.Color.white); % Java setter-method notation jSlider5.setPaintTicks(true); set(jSlider5, 'MinorTickSpacing',1, 'MajorTickSpacing',4, 'SnapToTicks',true, 'PaintLabels',true); % HG set notation [hSlider5, hContainer5] = javacomponent(jSlider5, [10,230,100,40], hFig);

For additional details regarding the other slider alternatives, please refer to my earlier post on this subject.

Have you ever used another interesting utility or class in Matlab’s builtin packages? If so, please tell us about it in a comment below.

]]>
https://undocumentedmatlab.com/blog/sliders-in-matlab-gui-part-2/feed 4
String/char compatibilityhttps://undocumentedmatlab.com/blog/string-char-compatibility https://undocumentedmatlab.com/blog/string-char-compatibility#comments Thu, 28 Jun 2018 12:57:59 +0000 https://undocumentedmatlab.com/?p=7710

Related posts:
1. Undocumented mouse pointer functions Matlab contains several well-documented functions and properties for the mouse pointer. However, some very-useful functions have remained undocumented and unsupported. This post details their usage....
2. Matlab layout managers: uicontainer and relatives Matlab contains a few undocumented GUI layout managers, which greatly facilitate handling GUI components in dynamically-changing figures....
3. Tab panels – uitab and relatives This article describes several undocumented Matlab functions that support tab-panels...
4. uitree This article describes the undocumented Matlab uitree function, which displays data in a GUI tree component...

]]>
In numerous functions that I wrote over the years, some input arguments were expected to be strings in the old sense, i.e. char arrays for example, 'on' or 'off'. Matlab release R2016b introduced the concept of string objects, which can be created using the string function or [starting in R2017a] double quotes ("on").

The problem is that I have numerous functions that supported the old char-based strings but not the new string objects. If someone tries to enter a string object ("on") as input to a function that expects a char-array ('on'), in many cases Matlab will error. This by itself is very unfortunate – I would have liked everything to be fully backward-compatible. But unfortunately this is not the case: MathWorks did invest effort in making the new strings backward-compatible to some degree (for example, graphic object property names/values and many internal functions that now accept either form as input). However, backward compatibility of strings is not 100% perfect.

In such cases, the only solution is to make the function accept both forms (char-arrays and string objects), for example, by type-casting all such inputs as char-arrays using the builtin char function. If we do this at the top of our function, then the rest of the function can remain unchanged. For example:

function test(stage) if isa(stage,'string') stage = char(stage); end % from this point onward, we don't need to worry about string inputs - any such strings will become plain-ol' char-arrays   switch stage case 'stage 1', ... case 'stage 2', ... ... end end

That was simple enough. But what if our function expects complex inputs (cell-arrays, structs etc.) that may contain strings in only some of their cells/fields?

Luckily, Matlab contains an internal utility function that can help us: controllib.internal.util.hString2Char. This function, whose Matlab source-code is available (%matlabroot%/toolbox/shared/controllib/general/+controllib/+internal/+util/hString2Char.m) recursively scans the input value and converts any string object into the corresponding char-array, leaving all other data-types unchanged. For example:

>> controllib.internal.util.hString2Char({123, 'char-array', "a string"}) ans = 1×3 cell array {[123]} {'char-array'} {'a string'}   >> controllib.internal.util.hString2Char(struct('a',"another string", 'b',pi)) ans = struct with fields: a: 'another string' b: 3.14159265358979

In order to keep our code working not just on recent releases (that support strings and controllib.internal.util.hString2Char) but also on older Matlab releases (where they did not exist), we simply wrap the call to hString2Char within a trycatch block. The adaptation of our function might then look as follows:

function test(varargin) try varargin = controllib.internal.util.hString2Char(varargin); catch, end % from this point onward, we don't need to worry about string inputs - any such strings will become plain-ol' char-arrays ... end

Note that controllib.internal.util.hString2Char is a semi-documented function: it contains a readable internal help section (accessible via help controllib.internal.util.hString2Char), but not a doc-page. Nor is this function mentioned anywhere in Matlab’s official documentation. I think that this is a pity, because it’s such a useful little helper function.

]]>
https://undocumentedmatlab.com/blog/string-char-compatibility/feed 15
Speeding-up builtin Matlab functions – part 2https://undocumentedmatlab.com/blog/speeding-up-builtin-matlab-functions-part-2 https://undocumentedmatlab.com/blog/speeding-up-builtin-matlab-functions-part-2#comments Sun, 06 May 2018 16:26:19 +0000 https://undocumentedmatlab.com/?p=7533

Related posts:
1. Speeding-up builtin Matlab functions – part 1 Built-in Matlab functions can often be profiled and optimized for improved run-time performance. This article shows a typical example. ...
2. Solving a MATLAB bug by subclassing Matlab's Image Processing Toolbox's impoint function contains an annoying bug that can be fixed using some undocumented properties....
3. Docs of old Matlab releases MathWorks recently posted archived documentation for many previous Matlab releases...
4. Array resizing performance Several alternatives are explored for dynamic array growth performance in Matlab loops. ...

]]>
Last week I showed how we can speed-up built-in Matlab functions, by creating local copies of the relevant m-files and then optimizing them for improved speed using a variety of techniques. Today I will show another example of such speed-up, this time of the Financial Toolbox’s maxdrawdown function, which is widely used to estimate the relative risk of a trading strategy or asset. One might think that such a basic indicator would be optimized for speed, but experience shows otherwise. In fact, this function turned out to be the main run-time performance hotspot for one of my clients. The vast majority of his code was optimized for speed, and he naturally assumed that the built-in Matlab functions were optimized as well, but this was not the case. Fortunately, I was able to create an equivalent version that was 30-40 times faster (!), and my client remains a loyal Matlab fan.

In today’s post I will show how I achieved this speed-up, using different methods than the ones I showed last week. A combination of these techniques can be used in a wide range of other Matlab functions. Additional speed-up techniques can be found in other performance-related posts on this website, as well in my book Accelerating MATLAB Performance.

### Profiling

As I explained last week, the first step in speeding up any function is to profile its current run-time behavior using Matlab’s builtin Profiler tool, which can either be started from the Matlab Editor toolstrip (“Run and Time”) or via the profile function.

The profile report for the client’s function showed that it had two separate performance hotspots:

1. Code that checks the drawdown format (optional 2nd input argument) against a set of allowed formats
2. Main code section that iteratively loops over the data-series values to compute the maximal drawdown

In order top optimize the code, I copied %matlabroot%/toolbox/finance/finance/maxdrawdown.m to a local folder on the Matlab path, renaming the file (and the function) maxdrawdn, in order to avoid conflict with the built-in version.

### Optimizing input args pre-processing

The main problem with the pre-processing of the optional format input argument is the string comparisons, which are being done even when the default format is used (which is by far the most common case). String comparison are often more expensive than numerical computations. Each comparison by itself is very short, but when maxdrawdown is run in a loop (as it often is), the run-time adds up.

Here’s a snippet of the original code:

if nargin < 2 || isempty(Format) Format = 'return'; end if ~ischar(Format) || size(Format,1) ~= 1 error(message('finance:maxdrawdown:InvalidFormatType')); end choice = find(strncmpi(Format,{'return','arithmetic','geometric'},length(Format))); if isempty(choice) error(message('finance:maxdrawdown:InvalidFormatValue')); end

An equivalent code, which avoids any string processing in the common default case, is faster, simpler and more readable:

if nargin < 2 || isempty(Format) choice = 1; elseif ~ischar(Format) || size(Format,1) ~= 1 error(message('finance:maxdrawdown:InvalidFormatType')); else choice = find(strncmpi(Format,{'return','arithmetic','geometric'},length(Format))); if isempty(choice) error(message('finance:maxdrawdown:InvalidFormatValue')); end end

The general rule is that whenever you have a common case, you should check it first, avoiding unnecessary processing downstream. Moreover, for improved run-time performance (although not necessarily maintainability), it is generally preferable to work with numbers rather than strings (choice rather than Format, in our case).

### Vectorizing the main loop

The main processing loop uses a very simple yet inefficient iterative loop. I assume that the code was originally developed this way in order to assist debugging and to ensure correctness, and that once it was ready nobody took the time to also optimize it for speed. It looks something like this:

MaxDD = zeros(1,N); MaxDDIndex = ones(2,N); ... if choice == 1 % 'return' format MaxData = Data(1,:); MaxIndex = ones(1,N); for i = 1:T MaxData = max(MaxData, Data(i,:)); q = MaxData == Data(i,:); MaxIndex(1,q) = i; DD = (MaxData - Data(i,:)) ./ MaxData; if any(DD > MaxDD) p = DD > MaxDD; MaxDD(p) = DD(p); MaxDDIndex(1,p) = MaxIndex(p); MaxDDIndex(2,p) = i; end end else % 'arithmetic' or 'geometric' formats ...

This loop can relatively easily be vectorized, making the code much faster, and arguably also simpler, more readable, and more maintainable:

if choice == 3 Data = log(Data); end MaxDDIndex = ones(2,N); MaxData = cummax(Data); MaxIndexes = find(MaxData==Data); DD = MaxData - Data; if choice == 1 % 'return' format DD = DD ./ MaxData; end MaxDD = cummax(DD); MaxIndex2 = find(MaxDD==DD,1,'last'); MaxIndex1 = MaxIndexes(find(MaxIndexes<=MaxIndex2,1,'last')); MaxDDIndex(1,:) = MaxIndex1; MaxDDIndex(2,:) = MaxIndex2; MaxDD = MaxDD(end,:);

Let’s make a short run-time and accuracy check – we can see that we achieved a 31-fold speedup (YMMV), and received the exact same results:

>> data = rand(1,1e7);   >> tic, [MaxDD1, MaxDDIndex1] = maxdrawdown(data); toc % builtin Matlab function Elapsed time is 7.847140 seconds.   >> tic, [MaxDD2, MaxDDIndex2] = maxdrawdn(data); toc % our optimized version Elapsed time is 0.253130 seconds.   >> speedup = round(7.847140 / 0.253130) speedup = 31   >> isequal(MaxDD1,MaxDD2) && isequal(MaxDDIndex1,MaxDDIndex2) % check accuracy ans = logical 1

Disclaimer: The code above seems to work (quite well in fact) for a 1D data vector. You’d need to modify it a bit to handle 2D data – the returned maximal drawdown are still computed correctly but not the returned indices, due to their computation using the find function. This modification is left as an exercise for the reader…

Very similar code could be used for the corresponding maxdrawup function. Although this function is used much less often than maxdrawdown, it is in fact widely used and very similar to maxdrawdown, so it is surprising that it is missing in the Financial Toolbox. Here is the corresponding code snippet:

% Code snippet for maxdrawup (similar to maxdrawdn) MaxDUIndex = ones(2,N); MinData = cummin(Data); MinIndexes = find(MinData==Data); DU = Data - MinData; if choice == 1 % 'return' format DU = DU ./ MinData; end MaxDU = cummax(DU); MaxIndex = find(MaxDD==DD,1,'last'); MinIndex = MinIndexes(find(MinIndexes<=MaxIndex,1,'last')); MaxDUIndex(1,:) = MinIndex; MaxDUIndex(2,:) = MaxIndex; MaxDU = MaxDU(end,:);

Similar vectorization could be applied to the emaxdrawdown function. This too is left as an exercise for the reader…

### Conclusions

Matlab is composed of thousands of internal functions. Each and every one of these functions was meticulously developed and tested by engineers, who are after all only human. Whereas supreme emphasis is always placed with Matlab functions on their accuracy, run-time performance sometimes takes a back-seat. Make no mistake about this: code accuracy is almost always more important than speed (an exception are cases where some accuracy may be sacrificed for improved run-time performance). So I’m not complaining about the current state of affairs.

But when we run into a specific run-time problem in our Matlab program, we should not despair if we see that built-in functions cause slowdown. We can try to avoid calling those functions (for example, by reducing the number of invocations, or limiting the target accuracy, etc.), or optimize these functions in our own local copy, as I’ve shown last week and today. There are multiple techniques that we could employ to improve the run time. Just use the profiler and keep an open mind about alternative speed-up mechanisms, and you’d be half-way there.

Let me know if you’d like me to assist with your Matlab project, either developing it from scratch or improving your existing code, or just training you in how to improve your Matlab code’s run-time/robustness/usability/appearance. I will be visiting Boston and New York in early June and would be happy to meet you in person to discuss your specific needs.

]]>
https://undocumentedmatlab.com/blog/speeding-up-builtin-matlab-functions-part-2/feed 8
Speeding-up builtin Matlab functions – part 1https://undocumentedmatlab.com/blog/speeding-up-builtin-matlab-functions-part-1 https://undocumentedmatlab.com/blog/speeding-up-builtin-matlab-functions-part-1#comments Sun, 29 Apr 2018 09:46:29 +0000 https://undocumentedmatlab.com/?p=7509

Related posts:
1. Speeding-up builtin Matlab functions – part 2 Built-in Matlab functions can often be profiled and optimized for improved run-time performance. This article shows a typical example. ...
2. Solving a MATLAB bug by subclassing Matlab's Image Processing Toolbox's impoint function contains an annoying bug that can be fixed using some undocumented properties....
3. Matrix processing performance Matrix operations performance is affected by internal subscriptions in a counter-intuitive way....
4. Improving save performance There are many different ways of improving Matlab's standard save function performance. ...

]]>
A client recently asked me to assist with his numeric analysis function – it took 45 minutes to run, which was unacceptable (5000 runs of ~0.55 secs each). The code had to run in 10 minutes or less to be useful. It turns out that 99% of the run-time was taken up by Matlab’s built-in fitdist function (part of the Statistics Toolbox), which my client was certain is already optimized for maximal performance. He therefore assumed that to get the necessary speedup he must either switch to another programming language (C/Java/Python), and/or upgrade his computer hardware at considerable expense, since parallelization was not feasible in his specific case.

Luckily, I was there to assist and was able to quickly speed-up the code down to 7 minutes, well below the required run-time. In today’s post I will show how I did this, which is relevant for a wide variety of other similar performance issues with Matlab. Many additional speed-up techniques can be found in other performance-related posts on this website, as well in my book Accelerating MATLAB Performance.

### Profiling

The first step in speeding up any function is to profile its current run-time behavior using Matlab’s builtin Profiler tool, which can either be started from the Matlab Editor toolstrip (“Run and Time”) or via the profile function.

The profile report for the client’s function showed that 99% of the time was spent in the Statistics Toolbox’s fitdist function. Drilling into this function in the profiling report, we see onion-like functions that processed input parameters, ensured data validation etc. The core processing is done inside a class that is unique to each required distribution (e.g., prob.StableDistribution, prob.BetaDistribution etc.) that is invoked within fitdist using an feval call, based on the distribution name that was specified by the user.

In our specific case, the external onion layers of sanity checks were unnecessary and could be avoided. In general, I advise not to discard such checks, because you never know whether future uses might have a problem with outlier data or input parameters. Moreover, in the specific case of fitdist they take only a very minor portion of the run-time (this may be different in other cases, such as the ismember function that I described years ago, where the sanity checks have a significant run-time impact compared to the core processing in the internal ismembc function).

However, since we wanted to significantly improve the total run-time and this was spent within the distribution class (prob.StableDistribution in the case of my client), we continue to drill-down into this class to determine what can be done.

It turns out that prob.StableDistribution basically does 3 things in its main fit() method:

1. pre-process the input data (prob.ToolboxFittableParametricDistribution.processFitArgs() and .removeCensoring() methods) – this turned out to be unnecessary in my client’s data, but has little run-time impact.
2. call stablefit() in order to get fitting parameters – this took about half the run-time
3. call stablelike() in order to get likelihood data – this took about half the run-time as well
4. call prob.StableDistribution.makeFitted() to create a probability-distribution object returned to the caller – this also took little run-time that was not worth bothering about.

### The speed-up improvement process

With user-created code we could simply modify our code in-place. However, a more careful process is necessary when modifying built-in Matlab functions (either in the core Matlab distribution or in one of the toolboxes).

The basic idea here is to create a side-function that would replicate the core processing of fitdist. This is preferable to modifying Matlab’s installation files because we could then reuse the new function in multiple computers, without having to mess around in the Matlab installation (which may not even be possible if we do not have admin privileges). Also, if we ever upgrade our Matlab we won’t need to remember to update the installed files (and obviously retest).

I called the new function fitdist2 and inside it I initially placed only the very core functionality of prob.StableDistribution.fit():

% fitdist2 - fast replacement for fitdist(data,'stable') % equivalent to pd = prob.StableDistribution.fit(data); function pd = fitdist2(data) % Bypass pre-processing (not very important) [cens,freq,opt] = deal([]); %[data,cens,freq,opt] = prob.ToolboxFittableParametricDistribution.processFitArgs(data); %data = prob.ToolboxFittableParametricDistribution.removeCensoring(data,cens,freq,'stable');   % Main processing in stablefit(), stablelike() params = stablefit(data,0.05,opt); [nll,cov] = stablelike(params,data);   % Combine results into the returned probability distribution object pd = prob.StableDistribution.makeFitted(params,nll,cov,data,cens,freq); end

If we try to run this as-is, we’d see errors because stablefit() and stablelike() are both sub-functions within %matlabroot%/toolbox/stats/stats/+prob/StableDistribution.m. So we copy these sub-functions (and their dependent subfunctions infoMtxCal(), intMle(), tabpdf(), neglog_pdf(), stable_nloglf(), varTrans) to the bottom of our fitdist2.m file, about 400 lines in total.

We also remove places that call checkargs(…) since that seems to be unnecessary – if you want to keep it then add the checkargs() function as well.

Now we re-run our code, after each speedup iteration verifying that the returned pd object returned by our fitdist2 is equivalent to the original object returned by fitdist.

### Speeding-up stablefit()

A new profiling run shows that the vast majority of the time in stablefit() is spent in two lines:

1. s = load('private/StablePdfTable.mat');
2. [parmhat,~,err,output] = fminsearch(@(params)stable_nloglf(x,params),phi0,options);

The first of these lines is reloading a static data file. The very same static data file is later reloaded in stablelike(). Both of these data-loads is done in every single invocation of fitdist, so if we have 5000 data fits, we load the same static data file 10,000 times! This is certainly not indicative of good programming practice. It would be much faster to reload the static data once into memory, and then use this cached data for the next 9,999 invocation. Since the original authors of StableDistribution.m seem to like single-character global variables (another bad programming practice, for multiple reasons), we’ll follow their example (added lines are highlighted):

persistent s % this should have a more meaningful name (but at least is not global...)!if isempty(s) fit_path = fileparts(which('fitdist')); s = load([fit_path '/private/StablePdfTable.mat']); a = s.a; b = s.b; xgd = s.xgd; p = s.p; end

In order to speed-up the second line (that calls fminsearch), we can reduce the tolerances used by this function, by updating the options structure passed to it, so that we use tolerances of 1e-3 rather than the default 1e-6 (in our specific case this resulted in acceptable errors of ~0.1%). Specifically, we modify the code from this:

function [parmhat,parmci] = stablefit(x,alpha,options) ... if nargin < 3 || isempty(options) options = statset('stablefit'); else options = statset(statset('stablefit'),options); end   % Maximize the log-likelihood with respect to the transformed parameters [parmhat,~,err,output] = fminsearch(@(params)stable_nloglf(x,params),phi0,options); ... end

to this (note that the line that actually calls fminsearch remains unchanged):

function [parmhat,parmci] = stablefit(x,alpha,unused_options)... persistent optionsif isempty(options) options = statset('stablefit'); options.TolX = 1e-3; options.TolFun = 1e-3; options.TolBnd = 1e-3;end   % Maximize the log-likelihood with respect to the transformed parameters [parmhat,~,err,output] = fminsearch(@(params)stable_nloglf(x,params),phi0,options); ... end

The fminsearch internally calls tabpdf() repeatedly. Drilling down in the profiling report we see that it recomputes a griddedInterpolant object that is essentially the same for all iterations (and therefore a prime candidate for caching), and also that it uses the costly cubic interpolation rather than a more straight-forward linear interpolation:

function y = tabpdf(x,alpha,beta) ... persistent G % this should have a more meaningful name (but at least is not global...)!if isempty(G) G = griddedInterpolant({b, a, xgd}, p, 'linear','none'); % 'linear' instead of 'cubic'end%G = griddedInterpolant({b, a, xgd}, p, 'cubic','none'); % original y = G({beta,alpha,x}); ...

These cases illustrate two important speedup technique categories: caching data in order to reduce the number of times that a certain code hotspot is being run, and modifying the parameters/inputs in order to reduce the individual run-time of the hotspot code. Variations of these techniques form the essence of effective speedup and can often be achieved by just reflecting on the problem and asking yourself two questions:

1. can I reduce the number of times that this code is being run? and
2. can I reduce the run-time of this specific code section?

Additional important speed-up categories include parallelization, vectorization and algorithmic modification. These are sometimes more difficult programmatically and riskier in terms of functional equivalence, but may be required in case the two main techniques above are not feasible. Of course, we can always combine these techniques, we don’t need to choose just one or the other.

### Speeding-up stablelike()

We now turn our attentions to stablelike(). As for the loaded static file, we could simply use the cached s to load the data in order to avoid repeated reloading of the data from disk. But it turns out that this data is actually not used at all inside the function (!) so we just comment-out the old code:

%s = load('private/StablePdfTable.mat'); %a = s.a; %b = s.b; %xgd = s.xgd; %p = s.p;

Think about this – the builtin Matlab code loads a data file from disk and then does absolutely nothing with it – what a waste!

Another important change is to reduce the run-time of the integral function, which is called thousands of times within a double loop. We do this by reducing the tolerances specified in the integral call from 1e-6:

F(i,j) = integral(@(x)infoMtxCal(x,params,step,i,j),-Inf,Inf,'AbsTol',1e-6,'RelTol',1e-4); % original F(i,j) = integral(@(x)infoMtxCal(x,params,step,i,j),-Inf,Inf,'AbsTol',1e-3,'RelTol',1e-3); % new

You can see that once again these two cases follow the two techniques that I mentioned above: we reduced the number of times that we load the data file (to 0 in our case), and we improved the run-time of the individual integral calculation by reducing its tolerances.

### Conclusions

The final result of applying the techniques above was a 6-fold speedup, reducing the total run-time from 45 minutes down to 7 minutes. I could probably have improved the run-time even further, but since we reached our target run-time I stopped there. The point after all was to make the code usable, not to reach a world speed record.

In my next article I will present another example of dramatically improving the run-time speed of a built-in Matlab function, this time a very commonly-used function in the Financial Toolbox that I was able to speed-up by a factor of 40.

Matlab releases improve continuously, so hopefully my techniques above (or alternatives) would find their way into the builtin Matlab functions, making them faster than today, out-of-the-box.

Until this happens, we should not lose hope when faced with a slow Matlab function, even if it is a built-in/internal one, as I hope to have clearly illustrated today, and will also show in my next article. Improving the performance is often easy. In fact, it took me much longer to write this article than it was to improve my client’s code…

Let me know if you’d like me to assist with your Matlab project, either developing it from scratch or improving your existing code, or just training you in how to improve your Matlab code’s run-time/robustness/usability/appearance. I will be visiting Boston and New York in early June and would be happy to meet you in person to discuss your specific needs.

]]>
https://undocumentedmatlab.com/blog/speeding-up-builtin-matlab-functions-part-1/feed 6

Related posts:
1. Matlab and the Event Dispatch Thread (EDT) The Java Swing Event Dispatch Thread (EDT) is very important for Matlab GUI timings. This article explains the potential pitfalls and their avoidance using undocumented Matlab functionality....
2. Tab panels – uitab and relatives This article describes several undocumented Matlab functions that support tab-panels...
3. uitree This article describes the undocumented Matlab uitree function, which displays data in a GUI tree component...
4. Borderless button used for plot properties A borderless button can be used to add unobtrusive functionality to plot axes...

]]>
Matlab objects have numerous built-in properties (some of them publicly-accessible/documented and others not, but that’s a different story). For various purposes, it is sometimes useful to attach custom user-defined properties to such objects. While there was never a fully-documented way to do this, most users simply attached such properties as fields in the UserData property or the object’s [hidden] ApplicationData property (accessible via the documented setappdata/getappdata functions).

An undocumented way to attach actual new user-defined properties to objects such as GUI handles or Java references has historically (in HG1, up to R2014a) been to use the undocumented schema.prop function, as I explained here. As I wrote in that post, in HG2 (R2014b onward), we can use the fully-documented addprop function to add new custom properties (and methods) to such objects. What is still NOT documented, as far as I could tell, is that all of Matlab’s builtin handle graphics objects indirectly inherit the dynamicprops class, which allows this. The bottom line is that we can dynamically add custom properties in run-time to any HG object, without affecting any other object. In other words, the new properties will only be added to the handles that we specifically request, and not to any others.

All this is important, because for some unexplained reason that escapes my understanding, MathWorks chose to seal its classes, thus preventing users to extend them with sub-classes that contain the new properties. So much frustration could have been solved if MathWorks would simply remove the Sealed class meta-property from its classes. Then again, I’d have less to blog about in that case…

Anyway, why am I rehashing old news that I have already reported a few years ago?

Well, first, because my experience has been that this little tidbit is [still] fairly unknown by Matlab developers. Secondly, I happened to run into a perfect usage example a short while ago that called for this solution: a StackExchange user asked whether it is possible to tell a GUI figure’s age, in other words the elapsed time since the figure was created. The simple answer would be to use setappdata with the creation date whenever we create a figure. However, a “cleaner” approach seems to be to create new read-only properties for the figure’s CreationTime and Age:

First, create a small Matlab function as follows, that attaches the CreationTime property to a figure:

function setCreationTime(hFig,varargin) hProp = addprop(hFig,'CreationTime'); hFig.CreationTime = now; hProp.SetAccess = 'private'; % make property read-only after setting its initial value   hProp = addprop(hFig,'Age'); hProp.GetMethod = @(h,e) etime(datevec(hFig.CreationTime), clock); % compute on-the-fly hProp.SetAccess = 'private'; % make property read-only end

Now assign this function as the default CreateFcn callback function for all new figures from now on:

set(0,'DefaultFigureCreateFcn',@setCreationTime)

That’s it – you’re done! Whenever a new figure will be created from now on, it will have two custom read-only properties: CreationTime and Age.

For example:

>> newFig = figure; >> newFig.CreationTime ans = 737096.613706748   >> ageInDays = now - newFig.CreationTime ageInDays = 0.0162507836846635 >> ageDuration = duration(ageInDays*24,0,0) ageDuration = duration 00:23:24 >> ageString = datestr(ageInDays, 'HH:MM:SS.FFF') ageString = '00:23:24.068'   >> ageInSecs = newFig.Age ageInSecs = 1404.06771035492

Note that an alternative way to set the computed property Age would have been to set its value to be an anonymous function, but this would have necessitated invoking it with parenthesis (as in: ageInSecs = newFig.Age()). By setting the property’s GetMethod meta-property we avoid this need.

Keen readers will have noticed that the mechanism that I outlined above for the Age property/method can also be used to add custom user methods. For example, we can create a new custom property named refresh that would be read-only and have a GetMethod which is the function handle of the function that refreshes the object in some way.

Do you have any special uses for custom user-defined properties/methods in your program? or perhaps you have a use-case that might show MathWorks why sub-classing the built-in classes might improve your work? if so, then please place a comment about it below. If enough users show MathWorks why this is important, then maybe it will be fixed in some future release.

]]>
Customizing axes tick labelshttps://undocumentedmatlab.com/blog/customizing-axes-tick-labels https://undocumentedmatlab.com/blog/customizing-axes-tick-labels#comments Wed, 24 Jan 2018 13:38:26 +0000 https://undocumentedmatlab.com/?p=7304

Related posts:
1. HG2 update HG2 appears to be nearing release. It is now a stable mature system. ...
2. Performance: accessing handle properties Handle object property access (get/set) performance can be significantly improved using dot-notation. ...
3. Customizing axes rulers HG2 axes can be customized in numerous useful ways. This article explains how to customize the rulers. ...
4. Plot legend title Titles to plot legends are easy to achieve in HG1 (R2014a or earlier), but much more difficult in HG2 (R2014b or newer). ...

]]>
In last week’s post, I discussed various ways to customize bar/histogram plots, including customization of the tick labels. While some of the customizations that I discussed indeed rely on undocumented properties/features, many Matlab users are not aware that tick labels can be individually customized, and that this is a fully documented/supported functionality. This relies on the fact that the default axes TickLabelInterpreter property value is 'tex', which supports a wide range of font customizations, individually for each label. This includes any combination of symbols, superscript, subscript, bold, italic, slanted, face-name, font-size and color – even intermixed within a single label. Since tex is the default interpreter, we don’t need any special preparation – simply set the relevant X/Y/ZTickLabel string to include the relevant tex markup.

To illustrate this, have a look at the following excellent answer by user Ubi on Stack Overflow:

Axes with Tex-customized tick labels

plot(1:10, rand(1,10)) ax = gca;   % Simply color an XTickLabel ax.XTickLabel{3} = ['\color{red}' ax.XTickLabel{3}];   % Use TeX symbols ax.XTickLabel{4} = '\color{blue} \uparrow';   % Use multiple colors in one XTickLabel ax.XTickLabel{5} = '\color[rgb]{0,1,0}green\color{orange}?';   % Color YTickLabels with colormap nColors = numel(ax.YTickLabel); cm = jet(nColors); for i = 1:nColors ax.YTickLabel{i} = sprintf('\\color[rgb]{%f,%f,%f}%s', cm(i,:), ax.YTickLabel{i}); end

In addition to 'tex', we can also set the axes object’s TickLabelInterpreter to 'latex' for a Latex interpreter, or 'none' if we want to use no string interpretation at all.

As I showed in last week’s post, we can control the gap between the tick labels and the axle line, using the Ruler object’s undocumented TickLabelGapOffset, TickLabelGapMultiplier properties.

Also, as I explained in other posts (here and here), we can also control the display of the secondary axle label (typically exponent or units) using the Ruler’s similarly-undocumented SecondaryLabel property. Note that the related Ruler’s Exponent property is documented/supported, but simply sets a basic exponent label (e.g., '\times10^{6}' when Exponent==6) – to set a custom label string (e.g., '\it\color{gray}Millions'), or to modify its other properties (position, alignment etc.), we should use SecondaryLabel.

]]>
https://undocumentedmatlab.com/blog/customizing-axes-tick-labels/feed 5
Customizing histogram plotshttps://undocumentedmatlab.com/blog/customizing-histogram-plots https://undocumentedmatlab.com/blog/customizing-histogram-plots#respond Wed, 17 Jan 2018 20:41:15 +0000 https://undocumentedmatlab.com/?p=7292

Related posts:
1. getundoc – get undocumented object properties getundoc is a very simple utility that displays the hidden (undocumented) properties of a specified handle object....
2. Customizing axes part 3 – Backdrop Matlab HG2 axes can be customized in many different ways. This article explains some of the undocumented aspects. ...
3. Customizing axes part 4 – additional properties Matlab HG2 axes can be customized in many different ways. This article explains some of the undocumented aspects. ...
4. Plot line transparency and color gradient Static and interpolated (gradient) colors and transparency can be set for plot lines in HG2. ...

]]>
Earlier today, I was given the task of displaying a histogram plot of a list of values. In today’s post, I will walk through a few customizations that can be done to bar plots and histograms in order to achieve the desired results.

We start by binning the raw data into pre-selected bins. This can easily be done using the builtin histc (deprecated) or histcounts functions. We can then use the bar function to plot the results:

[binCounts, binEdges] = histcounts(data); hBars = bar(hAxes, binEdges(1:end-1), binCounts);

Basic histogram bar plot

Let’s improve the appearance: In my specific case, the data was financial return (percentage) values, so let’s modify the x-label format accordingly and display a title. To make the labels and title more legible, we decrease the axes FontSize to 8 and remove the axes box:

hAxes = hBar.Parent; xtickformat(hAxes, '%g%%'); title(hAxes, 'Distribution of total returns (monthly %)'); set(hAxes, 'FontSize',8, 'Box','off')

Improved histogram bar plot

So far nothing undocumented. Note that the xtickformat/ytickformat functions were only introduced in R2016b – for earlier Matlab releases see this post (which does rely on undocumented aspects).

Now, let’s use a couple of undocumented properties: to remove the excess white-space margin around the axes we’ll set the axes’ LooseInset property, and to remove the annoying white space between the tick labels and the X-axis we’ll set the XRuler‘s TickLabelGapOffset property to -2 (default: +2):

set(hAxes, 'LooseInset',[0,0,0,0]); % default = [.13,.11,.095,.075] hAxes.XRuler.TickLabelGapOffset = -2; % default = +2

Even better histogram bar plot

Note that I used the undocumented axes XRuler property instead of the axes’ documented XAxis property, because XAxis is only available since R2015b, whereas XRuler (which points to the exact same object as XAxis) exists ever since R2014b, and so is better from a backward-compatibility standpoint. In either case, the ruler’s TickLabelGapOffset property is undocumented. Note that the ruler also contains another associated and undocumented TickLabelGapMultiplier property (default: 0.2), which I have not modified in this case.

Now let’s take a look at the bin labels: The problem with the bar plot above is that it’s not intuitively clear whether the bin for “5%”, for example, includes data between 4.5-5.5 or between 5.0-6.0 (which is the correct answer). It would be nicer if the labels were matched to the actual bin edges. There are 3 basic ways to fix this:

1. We could modify the bar plot axes tick values and labels, in essence “cheating” by moving the tick labels half a bin leftward of their tick values (don’t forget to add the extra tick label on the right):
hAxes.XTick(end+1) = hAxes.XTick(end) + 1; % extra tick label on the right labels = hAxes.XTickLabels; % preserve tick labels for later use below hAxes.XTick = hAxes.XTick - 0.5; % move tick labels 1/2 bin leftward hAxes.XTickLabel = labels; % restore pre-saved tick labels hAxes.XLim = hAxes.XLim - 0.5; % ...and align the XLim

Improved labels

2. We could use the bar function’s optional 'histc' flag, in order to display the bars in histogram mode. The problem in histogram mode is that while the labels are now placed correctly, the bars touch each other – I personally find distinct bars that are separated by a small gap easier to understand.
hBars = bar(..., 'histc'); % [snip] - same customizations to hAxes as done above

Basic histogram plot

With the original bar chart we could use the built-in BarWidth to set the bar/gap width (default: 0.8 meaning a 10% gap on either side of the bar). Unfortunately, calling bar with 'hist' or 'histc' (i.e. histogram mode) results in a Patch (not Bar) object, and patches do not have a BarWidth property. However, we can modify the resulting patch vertices in order to achieve the same effect:

% Modify the patch vertices (5 vertices per bar, row-based) hBars.Vertices(:,1) = hBars.Vertices(:,1) + 0.1; hBars.Vertices(4:5:end,1) = hBars.Vertices(4:5:end,1) - 0.2; hBars.Vertices(5:5:end,1) = hBars.Vertices(5:5:end,1) - 0.2;   % Align the bars & labels at the center of the axes hAxes.XLim = hAxes.XLim + 0.5;

This now looks the same as option #1 above, except that the top-level handle is a Patch (not Bar) object. For various additional customizations, either Patch or Bar might be preferable, so you have a choice.

Improved histogram plot

3. Lastly, we could have used the builtin histogram function instead of bar. This function also displays a plot with touching bars, as above, using Quadrilateral objects (a close relative of Patch). The solution here is very similar to option #2 above, but we need to dig a bit harder to modify the patch faces, since their vertices is not exposed as a public property of the Histogram object. To modify the vertices, we first get the private Face property (explanation), and then modify its vertices, keeping in mind that in this specific case the bars have 4 vertices per bar and a different vertices matrix orientation:
hBars = histogram(data, 'FaceAlpha',1.0, 'EdgeColor','none'); % [snip] - same customizations to hAxes as done above   % Get access to *ALL* the object's properties oldWarn = warning('off','MATLAB:structOnObject'); warning off MATLAB:hg:EraseModeIgnored hBarsStruct = struct(hBars); warning(oldWarn);   % Modify the patch vertices (4 vertices per bar, column-based) drawnow; % this is important, won't work without this! hFace = hBarsStruct.Face; % a Quadrilateral object (matlab.graphics.primitive.world.Quadrilateral) hFace.VertexData(1,:) = hFace.VertexData(1,:) + 0.1; hFace.VertexData(1,3:4:end) = hFace.VertexData(1,3:4:end) - 0.2; hFace.VertexData(1,4:4:end) = hFace.VertexData(1,4:4:end) - 0.2;

In conclusion, there are many different ways to improve the appearance of charts in Matlab. Even if at first glance it may seem that some visualization function does not have the requested customization property or feature, a little digging will often find either a relevant undocumented property, or an internal object whose properties could be modified. If you need assistance with customizing your charts for improved functionality and appearance, then consider contacting me for a consulting session.

]]>
https://undocumentedmatlab.com/blog/customizing-histogram-plots/feed 0
Customizing contour plots part 2https://undocumentedmatlab.com/blog/customizing-contour-plots-part2 https://undocumentedmatlab.com/blog/customizing-contour-plots-part2#comments Sun, 12 Nov 2017 11:03:37 +0000 https://undocumentedmatlab.com/?p=7149

Related posts:
1. Draggable plot data-tips Matlab's standard plot data-tips can be customized to enable dragging, without being limitted to be adjacent to their data-point. ...
2. Customizing contour plots Contour labels, lines and fill patches can easily be customized in Matlab HG2. ...
3. Matlab’s HG2 mechanism HG2 is presumably the next generation of Matlab graphics. This article tries to explore its features....
4. getundoc – get undocumented object properties getundoc is a very simple utility that displays the hidden (undocumented) properties of a specified handle object....

]]>
A few weeks ago a user posted a question on Matlab’s Answers forum, asking whether it is possible to display contour labels in the same color as their corresponding contour lines. In today’s post I’ll provide some insight that may assist users with similar customizations in other plot types.

Matlab does not provide, for reasons that escape my limited understanding, documented access to the contour plot’s component primitives, namely its contour lines, labels and patch faces. Luckily however, these handles are accessible (in HG2, i.e. R2014b onward) via undocumented hidden properties aptly named EdgePrims, TextPrims and FacePrims, as I explained in a previous post about contour plots customization, two years ago.

[X,Y,Z] = peaks; [C,hContour] = contour(X,Y,Z, 'ShowText','on', 'LevelStep',1);

The result is the screenshot on the left:

Standard Matlab contour labels

Customized Matlab contour labels

In order to update the label colors (to get the screenshot on the right), we create a short updateContours function that updates the TextPrims color to their corresponding EdgePrims color:

### The updateContours() function

function updateContours(hContour) % Update the text label colors drawnow % very important! levels = hContour.LevelList; labels = hContour.TextPrims; % undocumented/unsupported lines = hContour.EdgePrims; % undocumented/unsupported for idx = 1 : numel(labels) labelValue = str2double(labels(idx).String); lineIdx = find(abs(levels-labelValue)<10*eps, 1); % avoid FP errors using eps labels(idx).ColorData = lines(lineIdx).ColorData; % update the label color %labels(idx).Font.Size = 8; % update the label font size end drawnow % optional end

Note that in this function we don’t directly equate the numeric label values to the contour levels’ values: this would work well for integer values but would fail with floating-point ones. Instead I used a very small 10*eps tolerance in the numeric comparison.

Also note that I was careful to call drawnow at the top of the update function, in order to ensure that EdgePrims and TextPrims are updated when the function is called (this might not be the case before the call to drawnow). The final drawnow at the end of the function is optional: it is meant to reduce the flicker caused by the changing label colors, but it can be removed to improve the rendering performance in case of rapidly-changing contour plots.

Finally, note that I added a commented line that shows we can modify other label properties (in this case, the font size from 10 to 8). Feel free to experiment with other label properties.

### Putting it all together

The final stage is to call our new updateContours function directly, immediately after creating the contour plot. We also want to call updateContours asynchronously whenever the contour is redrawn, for example, upon a zoom/pan event, or when one of the relevant contour properties (e.g., LevelStep or *Data) changes. To do this, we add a callback listener to the contour object’s [undocumented] MarkedClean event that reruns our updateContours function:

[X,Y,Z] = peaks; [C,hContour] = contour(X,Y,Z, 'ShowText','on', 'LevelStep',1);   % Update the contours immediately, and also whenever the contour is redrawn updateContours(hContour); addlistener(hContour, 'MarkedClean', @(h,e)updateContours(hContour));

### Contour level values

As noted in my comment reply below, the contour lines (hContour.EdgePrims) correspond to the contour levels (hContour.LevelList).

For example, to make all negative contour lines dotted, you can do the following:

[C,hContour] = contour(peaks, 'ShowText','on', 'LevelStep',1); drawnow set(hContour.EdgePrims(hContour.LevelList<0), 'LineStyle', 'dotted');

Customized Matlab contour lines

As I noted on my previous post on contour plot customization, I am marking this article as “High risk of breaking in future Matlab versions“, not because of the basic functionality (being important enough I don’t presume it will go away anytime soon) but because of the property names: TextPrims, EdgePrims and FacePrims don’t seem to be very user-friendly property names. So far MathWorks has been very diligent in making its object properties have meaningful names, and so I assume that when the time comes to expose these properties, they will be renamed (perhaps to TextHandles, EdgeHandles and FaceHandles, or perhaps LabelHandles, LineHandles and FillHandles). For this reason, even if you find out in some future Matlab release that TextPrims, EdgePrims and FacePrims don’t exist, perhaps they still exist and simply have different names. Note that these properties have not changed their names or functionality in the past 3 years, so while it could well happen next year, it could also remain unchanged for many years to come. The exact same thing can be said for the MarkedClean event.

### Professional assistance anyone?

As shown by this and many other posts on this site, a polished interface and functionality is often composed of small professional touches, many of which are not exposed in the official Matlab documentation for various reasons. So if you need top-quality professional appearance/functionality in your Matlab program, or maybe just a Matlab program that is dependable, robust and highly-performant, consider employing my consulting services.

]]>
https://undocumentedmatlab.com/blog/customizing-contour-plots-part2/feed 6

Related posts:
1. uiundo – Matlab’s undocumented undo/redo manager The built-in uiundo function provides easy yet undocumented access to Matlab's powerful undo/redo functionality. This article explains its usage....
2. Matlab’s HG2 mechanism HG2 is presumably the next generation of Matlab graphics. This article tries to explore its features....
3. New information on HG2 More information on Matlab's new HG2 object-oriented handle-graphics system...
4. Pinning annotations to graphs Annotation object can be programmatically set at, and pinned-to, plot axes data points. ...

]]>
Back in 2010, I posted about Matlab’s undocumented feature function. One of the features that I mentioned was 'HotLinks'. A few days ago I had an occasion to remember this feature when a StackOverflow user complained that the headers of table outputs in the Matlab console appear with HTML tags (<strong>) in his diary output. He asked whether it was possible to turn off this automated headers markup.

There are several ways this problem can be solved, ranging from creating a custom table display function, to modifying the table’s internal disp method (%matlabroot%/toolbox/matlab/datatypes/@tabular/disp.m), to using this method’s second optional argument (disp(myTable,false)). Note that simply subclassing the table class to overload disp() will not work because the table class is Sealed, but we could instead subclass table‘s superclass (tabular) just like table does.

Inside the disp.m method mentioned above, the headers markup is controlled (around line 45, depending on your Matlab release) by matlab.internal.display.isHot. Unfortunately, there is no corresponding setHot() method, nor corresponding m- or p-code that can be inspected. But the term “Hot” rang a bell, and then I remembered my old post about the HotLinks feature, which is apparently reflected by matlab.internal.display.isHot.

feature('HotLinks',false); % temporarily disable bold headers and hyperlinks (matlab.internal.display.isHot=false) disp(myTable) myTable % this calls disp() implicitly feature('HotLinks',true); % restore the standard behavior (markup displayed, matlab.internal.display.isHot=true)

Searching for “isHot” or “HotLinks” under the Matlab installation folder, we find that this feature is used in hundreds of places (the exact number depends on your installed toolboxes). The general use appears to be to disable/enable output of hyperlinks to the Matlab console, such as when you display a Matlab class, when its class name is hyperlinked and so is the “Show all properties” message at the bottom. But in certain cases, such as for the table output above, the feature is also used to determine other types of markup (bold headers in this case).

>> feature('HotLinks',0) % temporarily disable bold headers and hyperlinks >> groot ans = Graphics Root with properties:   CurrentFigure: [0×0 GraphicsPlaceholder] ScreenPixelsPerInch: 96 ScreenSize: [1 1 1366 768] MonitorPositions: [2×4 double] Units: 'pixels'   Use GET to show all properties >> feature('HotLinks',1) % restore the standard behavior (markup displayed) >> groot ans = Graphics Root with properties:   CurrentFigure: [0×0 GraphicsPlaceholder] ScreenPixelsPerInch: 96 ScreenSize: [1 1 1366 768] MonitorPositions: [2×4 double] Units: 'pixels'   Show all properties

There’s nothing really earth shuttering in all this, but the HotLinks feature could be useful when outputting console output into a diary file. Of course, if diary would have automatically stripped away markup tags we would not need to resort to such hackery. Then again, this is not the only problem with diary, which is long-overdue an overhaul.

]]>
Tips for accelerating Matlab performancehttps://undocumentedmatlab.com/blog/tips-for-accelerating-matlab-performance https://undocumentedmatlab.com/blog/tips-for-accelerating-matlab-performance#comments Thu, 05 Oct 2017 18:25:06 +0000 http://undocumentedmatlab.com/?p=7099

Related posts:
1. Performance: scatter vs. line In many circumstances, the line function can generate visually-identical plots as the scatter function, much faster...
2. Plot LimInclude properties The plot objects' XLimInclude, YLimInclude, ZLimInclude, ALimInclude and CLimInclude properties are an important feature, that has both functional and performance implications....
3. Plot performance Undocumented inner plot mechanisms can significantly improve plotting performance ...
4. Performance: accessing handle properties Handle object property access (get/set) performance can be significantly improved using dot-notation. ...

]]>
I’m proud to report that MathWorks has recently posted my article “Tips for Accelerating MATLAB Performance” in their latest newsletter digest (September 2017). This article is an updated and expanded version of my post about consulting work that I did for the Crustal Dynamics Research Group at Harvard University, where I helped speed-up a complex Matlab-based GUI by a factor of 50-500 (depending on the specific feature). You can read the full detailed technical article here.

Crustal dynamics visualization GUI

Featuring an article on the official newsletter by a non-MathWorker is rare. Doing this with someone like myself who has a reputation for undocumented aspects, and a consultancy business that potentially competes with theirs, is certainly not obvious. I take this to be a sign that despite the possible drawbacks of publishing my article, MathWorks felt that it provided enough value to the Matlab user community to merit the risk. I applaud MathWorks for this, and thank them for the opportunity of being featured in their official newsletter and conferences. I do not take it for granted in the least.

The newsletter article provides multiple ideas of improving the run-time performance for file I/O and graphics. Many additional techniques for improving Matlab’s performance can be found under the Performance tag in this blog, as well as in my book “Accelerating MATLAB Performance” (CRC Press, 2014, ISBN 978-1482211290).

I am offering a couple of webinars about various ways to improve Matlab’s run-time performance:

Both the webinar videos and their corresponding slide-decks are available for download. The webinars content is based on onsite training courses that I presented at multiple client locations (details).

Email me if you would like additional information on the webinars or my consulting, or to inquire regarding an onsite training course.

]]>
https://undocumentedmatlab.com/blog/tips-for-accelerating-matlab-performance/feed 2
Faster csvwrite/dlmwritehttps://undocumentedmatlab.com/blog/faster-csvwrite-dlmwrite https://undocumentedmatlab.com/blog/faster-csvwrite-dlmwrite#comments Tue, 03 Oct 2017 15:00:05 +0000 http://undocumentedmatlab.com/?p=7080

Related posts:
1. Parsing mlint (Code Analyzer) output The Matlab Code Analyzer (mlint) has a lot of undocumented functionality just waiting to be used. ...
2. Explicit multi-threading in Matlab part 3 Matlab performance can be improved by employing POSIX threads in C/C++ code. ...
3. Plot LimInclude properties The plot objects' XLimInclude, YLimInclude, ZLimInclude, ALimInclude and CLimInclude properties are an important feature, that has both functional and performance implications....
4. Preallocation performance Preallocation is a standard Matlab speedup technique. Still, it has several undocumented aspects. ...

]]>
Matlab’s builtin functions for exporting (saving) data to output files are quite sub-optimal (as in slowwwwww…). I wrote a few posts about this in the past (how to improve fwrite performance, and save performance). Today I extend the series by showing how we can improve the performance of delimited text output, for example comma-separated (CSV) or tab-separated (TSV/TXT) files.

The basic problem is that Matlab’s dlmwrite function, which can either be used directly, or via the csvwrite function which calls it internally, is extremely inefficient: It processes each input data value separately, in a non-vectorized loop. In the general (completely non-vectorized) case, each data value is separately converted into a string, and is separately sent to disk (using fprintf). In the specific case of real data values with simple delimiters and formatting, row values are vectorized, but in any case the rows are processed in a non-vectorized loop: A newline character is separately exported at the end of each row, using a separate fprintf call, and this has the effect of flushing the I/O to disk each and every row separately, which is of course disastrous for performance. The output file is indeed originally opened in buffered mode (as I explained in my fprintf performance post), but this only helps for outputs done within the row – the newline output at the end of each row forces an I/O flush regardless of how the file was opened. In general, when you read the short source-code of dlmwrite.m you’ll get the distinct feeling that it was written for correctness and maintainability, and some focus on performance (e.g., the vectorization edge-case). But much more could be done for performance it would seem.

This is where Alex Nazarovsky comes to the rescue.

Alex was so bothered by the slow performance of csvwrite and dlmwrite that he created a C++ (MEX) version that runs about enormously faster (30 times faster on my system). He explains the idea in his blog, and posted it as an open-source utility (mex-writematrix) on GitHub.

Usage of Alex’s utility is very easy:

mex_WriteMatrix(filename, dataMatrix, textFormat, delimiter, writeMode);

where the input arguments are:

• filename – full path name for file to export
• dataMatrix – matrix of numeric values to be exported
• textFormat – format of output text (sprintf format), e.g. '%10.6f'
• delimiter – delimiter, for example ',' or ';' or char(9) (=tab)
• writeMode – 'w+' for rewriting file; 'a+' for appending (note the lowercase: uppercase will crash Matlab!)

Here is a sample run on my system, writing a simple CSV file containing 1K-by-1K data values (1M elements, ~12MB text files):

>> data = rand(1000, 1000); % 1M data values, 8MB in memory, ~12MB on disk   >> tic, dlmwrite('temp1.csv', data, 'delimiter',',', 'precision','%10.10f'); toc Elapsed time is 28.724937 seconds.   >> tic, mex_WriteMatrix('temp2.csv', data, '%10.10f', ',', 'w+'); toc % 30 times faster! Elapsed time is 0.957256 seconds.

Alex’s mex_WriteMatrix function is faster even in the edge case of simple formatting where dlmwrite uses vectorized mode (in that case, the file is exported in ~1.2 secs by dlmwrite and ~0.9 secs by mex_WriteMatrix, on my system).

### Trapping Ctrl-C interrupts

Alex’s mex_WriteMatrix code includes another undocumented trick that could help anyone else who uses a long-running MEX function, namely the ability to stop the MEX execution using Ctrl-C. Using Ctrl-C is normally ignored in MEX code, but Wotao Yin showed how we can use the undocumented utIsInterruptPending() MEX function to monitor for user interrupts using Ctrl-C. For easy reference, here is a copy of Wotao Yin’s usage example (read his webpage for additional details):

/* A demo of Ctrl-C detection in mex-file by Wotao Yin. Jan 29, 2010. */   #include "mex.h"   #if defined (_WIN32) #include <windows.h> #elif defined (__linux__) #include <unistd.h> #endif   #ifdef __cplusplus extern "C" bool utIsInterruptPending(); #else extern bool utIsInterruptPending(); #endif   void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]) { int count = 0; while(1) { #if defined(_WIN32) Sleep(1000); /* Sleep one second */ #elif defined(__linux__) usleep(1000*1000); /* Sleep one second */ #endif   mexPrintf("Count = %d\n", count++); /* print count and increase it by 1 */ mexEvalString("drawnow;"); /* flush screen output */   if (utIsInterruptPending()) { /* check for a Ctrl-C event */ mexPrintf("Ctrl-C Detected. END\n\n"); return; } if (count == 10) { mexPrintf("Count Reached 10. END\n\n"); return; } } }

### Matlab performance webinars

Next week I will present live online webinars about numerous other ways to improve Matlab’s run-time performance:

These live webinars will be 3.5 hours long, starting at 10am EDT (7am PDT, 3pm UK, 4pm CET, 7:30pm IST, time in your local timezone), with a short break in the middle. The presentations content will be based on onsite training courses that I presented at multiple client locations (details). A recording of the webinars will be available for anyone who cannot join the live events.

Additional Matlab performance tips can be found under the Performance tag in this blog, as well as in my book “Accelerating MATLAB Performance“.

Email me if you would like additional information on the webinars, or an onsite training course, or about my consulting.

]]>
https://undocumentedmatlab.com/blog/faster-csvwrite-dlmwrite/feed 5
Runtime code instrumentationhttps://undocumentedmatlab.com/blog/runtime-code-instrumentation https://undocumentedmatlab.com/blog/runtime-code-instrumentation#comments Thu, 28 Sep 2017 13:36:17 +0000 http://undocumentedmatlab.com/?p=7063

Related posts:
1. Creating a simple UDD class This article explains how to create and test custom UDD packages, classes and objects...
2. Customizing uiundo This article describes how Matlab's undocumented uiundo undo/redo manager can be customized...
3. Introduction to UDD UDD classes underlie many of Matlab's handle-graphics objects and functionality. This article introduces these classes....
4. Getting default HG property values Matlab has documented how to modify default property values, but not how to get the full list of current defaults. This article explains how to do this. ...

]]>
I regularly follow the MathWorks Pick-of-the-Week (POTW) blog. In a recent post, Jiro Doke highlighted Per Isakson’s tracer4m utility. Per is an accomplished Matlab programmer, who has a solid reputation in the Matlab user community for many years. His utility uses temporary conditional breakpoints to enable users to trace calls to their Matlab functions and class methods. This uses a little-known trick that I wish to highlight in this post.

tracer4m utility uses conditional breakpoints that evaluate but never become live

Matlab breakpoints are documented and supported functionality, and yet their documented use is typically focused at interactive programming in the Matlab editor, or as interactive commands that are entered in the Matlab console using the set of db* functions: dbstop, dbclear, dbstatus, dbstack etc. However, nothing prevents us from using these db* functions directly within our code.

For example, the dbstack function can help us diagnose the calling tree for the current function, in order to do action A if one of the calling ancestors was FunctionX, or to do action B otherwise (for example, to avoid nested recursions).

Similarly, we could add a programmatic call to dbstop in order to stop at a certain code location downstream (for debugging), if a certain condition happens upstream.

Per extended this idea very cleverly in tracer4m: conditional breakpoints evaluate a string in run-time: if the result is true (non-zero) then the code run is stopped at that location, but if it’s false (or zero) then the code run continues normally. To instrument calls to specific functions, Per created a function tracer() that logs the function call (using dbstack) and always returns the value false. He then dynamically created a string that contains a call to this new function and used the dbstop function to create a conditional breakpoint based on this function, something similar to this:

dbstop('in', filename, 'at', location, 'if', 'tracer()');

We can use this same technique for other purposes. For example, if we want to do some action (not necessarily log – perhaps do something else) when a certain code point is reached. The benefit here is that we don’t need to modify the code at all – we’re adding ad-hoc code pieces using the conditional breakpoint mechanism without affecting the source code. This is particularly useful when we do not have access to the source code (such as when it’s compiled or write-protected). All you need to do is to ensure that the instrumentation function always returns false so that the breakpoint does not become live and for code execution to continue normally.

The tracer4m utility is quite sophisticated in the sense that it uses mlint and smart regexp to parse the code and know which functions/methods occur on which line numbers and have which type (more details). In this sense, Per used undocumented functionality. I’m certain that Jiro was not aware of the dependency on undocumented features when he posted about the utility, so please don’t take this to mean that Jiro or MathWorks officially support this or any other undocumented functionality. Undocumented aspects are often needed to achieve top functionality, and I’m happy that the POTW blog highlights utilities based on their importance and merit, even if they do happen to use some undocumented aspect.

tracer4m‘s code also contains references to the undocumented profiler option -history, but this is not in fact used by the code itself, only in comments. I use this feature in my profile_history utility, which displays the function call/timing history in an interactive GUI window. This utility complements tracer4m by providing a lot more information, but this can result in a huge amount of information for large and/or long-running programs. In addition, tracer4m has the benefit of only logging those functions/methods that the user finds useful, rather than all the function call, which enables easier debugging when the relevant code area is known. In short, I wish I had known about tracer4m when I created profile_history. Now that I know about it, maybe I’ll incorporate some of its ideas into profile_history in order to make it more useful. Perhaps another moral of this is that we should actively monitor the POTW blog, because true gems are quite often highlighted there.

Function call timeline profiling

For anyone who missed the announcement in my previous post, I’m hosting a series of live webinars on advanced Matlab topics in the upcoming 2 weeks – I’ll be happy if you would join.

]]>
https://undocumentedmatlab.com/blog/runtime-code-instrumentation/feed 3
User-defined tab completions – take 2https://undocumentedmatlab.com/blog/user-defined-tab-completions-take-2 https://undocumentedmatlab.com/blog/user-defined-tab-completions-take-2#comments Wed, 12 Jul 2017 13:00:30 +0000 http://undocumentedmatlab.com/?p=6961

Related posts:
1. Setting desktop tab completions The Matlab desktop's Command-Window tab-completion can be customized for user-defined functions...
2. Class object tab completion & improper field names Tab completions and property access can be customized for user-created Matlab classes. ...
3. Recovering previous editor state Recovering the previous state of the Matlab editor and its loaded documents is possible using a built-in backup config file. ...
4. The HotLinks feature feature('HotLinks') can be used to temporarily disable hyperlinks and other markups in the Matlab console. ...

]]>
Back in 2010, I posted about Matlab’s undocumented mechanism for setting Matlab desktop tab-completions. That mechanism used a couple of internal files (TC.xml and TC.xsd) to describe the various possible options that are auto-completed (or displayed in a small tooltip window) when the user clicks the <Tab> key on partially-entered function input parameters.

Using TabComplete for user-defined functions

Unfortunately, this mechanism apparently broke in R2016a and was replaced with a new mechanism, as explained below.

The new mechanism relies on a file called functionSignatures.json which exists in every single folder that contains Matlab files that have functions whose input parameters ought to be tab-completable.

The new mechanism offers far greater versatility and flexability in defining the input types and inter-relationsships compared to the old TC.*-based mechanism. Another important benefit is that we can now add custom user-defined functionSignatures.json files to our user folders, next to our m-files, without having to modify any Matlab system file.

Note that you may need to restart Matlab the first time that you create a functionSignatures.json file. But once it’s created, you can modify it within a Matlab session and the changes take effect immediately.

Note: Credit for first posting about this new mechanism goes to Nicholas Mati. I’ve known about this new mechanism for over a year, but I never found the time to write about it until now, so Nicholas gets credit for breaking the scoop. The discussion below uses and expands Nicholas’ original post.

### Syntax

The functionSignatures.json file has the general form:

{ "FunctionName1": { "key1":"val1", "key2":"val2", "keyn":"valn" }, "FunctionName2": { "key1":"val1", "key2":"val2", "keyn":"valn" } }

A number of keys are supported including “platform“, “setsAns“, “inputs“, and “outputs“, although inputs and outputs are by far the most common (and presumably only inputs are relevant for tab-completion). These keys take an array of (or a single) object value(s). The objects typically take one of the following forms:

{"name":"variable_name", "kind":"kind_option", "type":"string_or_array_of_type"} {"mutuallyExclusiveGroup": [ ... ] } {"name":"varargin", "kind":"optional", "multiplicity":"append"}

The value for “kind” can be “required”, “optional”, “positional”, “flag”, “namevalue” or “platform” (and perhaps a few other lesser-used kinds):

• required” means that the specified input is mandatory
• optional” means that it can be added or omitted
• positional” means that it’s an optional input but if it is specified then it must appear at the specified position relative to the previous (earlier) inputs
• flag” means that it’s an optional input flag, from a predefined list of one or more single-token strings. For example, in regexp(s1,s2,'once') the last input arg ('once') is such a flag.
• namevalue” means that it follows Matlab’s standard practice of using P-V pairs (parameter name followed by its value). For example, func('propName',propValue)
• platform” indicates that this input is only available on the specified platform(s)

These “kind”s are all explained below.

The value for “type” can be a string such as “char” or “numeric” or “filepath”, or a more complicated JSON array (see below).

In addition to “name”, “kind” and “type”, we can also define a “default” value (e.g. "default":"false") and a “display” string. While these are [currently] not used by Desktop tab-completion, they might be used by other components such as the JIT compiler or the Editor, if not today then perhaps in a future release.

Note that while pure JSON format does not accept comments, Matlab’s functionSignatures.json does accept C++-style comments, as discovered by Heiko in a comment below. To add a comment, simply add // comment text at the end of any line, or /* comment text */ anywhere within a line.

### Usage examples

Multiple examples of functionSignatures.json files can be found in subfolders of %matlabroot%/toolbox/matlab. For example, here’s the tab-completion definition for the visdiff function, which displays a visual comparison between two files, and resides in %matlabroot%/toolbox/shared/comparisons/functionSignatures.json:

{ "visdiff": { "inputs": [ {"name":"filename1", "kind":"required", "type":"filepath"}, {"name":"filename2", "kind":"required", "type":"filepath"}, {"name":"type", "kind":"positional", "type":"choices={'text', 'binary'}"} ] } }

As can be seen in this example, the first and second inputs are expected to be a filename, whereas the third input is one of the two predefined strings ‘text’ or ‘binary’. This third input has “kind”:”positional”, meaning that it is optional, but if it is provided then it must be in the 3rd position and cannot appear sooner. Moreover, if the user specifies any input argument to the “right” of a positional input, then the positional argument becomes required, not optional.

Whereas a “positional” parameter has a specific position in the args list (#3 in the case of visdiff above), an “optional” parameter may appear anywhere in the list of inputs.

Here’s a more complex example, for the built-in regexprep function (in %matlabroot%/toolbox/matlab/strfun/functionSignatures.json). This example shows how to limit the input to certain data types and how to specify optional input flags with pre-defined choices:

"regexprep": { "inputs": [ {"name":"str", "kind":"required", "type":[["char"], ["cell"], ["string"]]}, {"name":"expression", "kind":"required", "type":[["char"], ["cell"], ["string"]]}, {"name":"replace", "kind":"required", "type":[["char"], ["cell"], ["string"]]}, {"name":"optMatch", "kind":"flag", "display":"", "type":[["char", "choices={'all','once'}"], ["numeric", "scalar"]], "default":"'all'"}, {"name":"optWarnings", "kind":"flag", "display":"", "type":["char", "choices={'nowarnings','warnings'}"], "default":"'nowarnings'"}, {"name":"optCase", "kind":"flag", "display":"", "type":["char", "choices={'matchcase','ignorecase','preservecase'}"], "default":"'matchcase'"}, {"name":"optEmptyMatch", "kind":"flag", "display":"", "type":["char", "choices={'noemptymatch','emptymatch'}"], "default":"'noemptymatch'"}, {"name":"optDotAll", "kind":"flag", "display":"", "type":["char", "choices={'dotall','dotexceptnewline'}"], "default":"'dotall'"}, {"name":"optStringAnchors", "kind":"flag", "display":"", "type":["char", "choices={'stringanchors','lineanchors'}"], "default":"'stringanchors'"}, {"name":"optSpacing", "kind":"flag", "display":"", "type":["char", "choices={'literalspacing','freespacing'}"], "default":"'literalspacing'"} ], "outputs": [ {"name":"newStr", "type":[["char"], ["cell"], ["string"]]} ] },

Here’s an even more complex example, this time for the codegen function (in %matlabroot%/toolbox/coder/matlabcoder/functionSignatures.json, part of the Matlab Coder toolbox). This example shows how to limit the filenames to certain extensions and how to specify name-value input pairs:

"codegen": { "inputs": [ {"name":"compile_only", "kind":"flag", "type":"choices={'-c'}"}, {"name":"config_flag", "kind":"flag", "type":"choices={'-config:mex','-config:lib','-config:dll','-config:exe','-config:hdl'}"}, {"name":"debug", "kind":"flag", "type":"choices={'-g'}"}, {"name":"report", "kind":"flag", "type":"choices={'-report'}"}, {"name":"launchreport", "kind":"flag", "type":"choices={'-launchreport'}"}, {"name":"file", "kind":"flag", "type":"filepath=*.m,*.mlx,*.c,*.cpp,*.h,*.o,*.obj,*.a,*.so,*.lib,*.tmf", "multiplicity":"append"}, {"name":"-d", "kind":"namevalue", "type":"folderpath"}, {"name":"-I", "kind":"namevalue", "type":"folderpath"}, {"name":"-globals", "kind":"namevalue"}, {"name":"-o", "kind":"namevalue", "type":[["char"], ["filepath"]]}, {"name":"-O", "kind":"namevalue", "type":"choices={'enable:inline','disable:inline','enable:blas','disable:blas','enable:openmp','disable:openmp'}"}, {"name":"-args", "kind":"namevalue", "type":[["identifier=variable"], ["char"]]}, {"name":"-config", "kind":"namevalue", "type":[["identifier=variable"], ["char"]]}, {"name":"verbose", "kind":"flag", "type":"choices={'-v'}"}, {"name":"singleC", "kind":"flag", "type":"choices={'-singleC'}"}, {"name":"-test", "kind":"namevalue", "type":"identifier=function"} ] },

### Argument types

As noted above, we use "type":... to specify the expected data type of each parameter. This can be a simple string such as “char”, “cellstr”, “numeric”, “table”, “categorical”, “filepath”, “folderpath”, “matlabpath”, class name, or a more complicated JSON array. For example:

• "type":["numeric","scalar"]
• "type":["numeric","numel=3",">=4"]
• "type":[["char"], ["cellstr"], ["numeric"], ["logical","vector"]]
• "type":[["char", "choices={'-ascii'}"]]
• "type":[["filepath"], ["matlabpath=*.m,*.mlx"], ["char"]]
• "type":"identifier=variable,function,localfunction,package,classdef"
• "type":"matlab.graphics.axis.Axes"
• "type":"choices={'yes','no','maybe'}"

We can even specify on-the-fly Matlab computation that returns a cell-array of values, for example a list of available fonts via "type":"choices=listfonts". A more complex example is the definition of the rmfield function, where the possible input choices for the second input arg (highlighted) depend on the struct that is provided in the first input arg (by running the fieldnames function on it):

"rmfield": { "inputs": [ {"name":"s", "kind":"required", "type":"struct"}, {"name":"field", "kind":"required", "type":"choices=fieldnames(s)"} ], "outputs": [ {"name":"s", "type":"struct"} ] },

### Alternative inputs

Multiple alternative inputs can be specified in the functionSignatures.json file. The easiest way to do so is to simply create multiple different definitions for the same function, one beneath the other. Matlab’s tab-completion parser is smart enough to combine those definitions and proceed with the most appropriate one based on the user-entered inputs.

For example, in the same Coder file above we find 6 alternative definitions. If (for example) we start typing coder('-ecoder', and click <Tab>, Matlab would automatically auto-complete the second input to “false”, and then another <Tab> click would set the third input to the required ‘-new’ parameter (see highlighted lines below):

... "coder": { "inputs": [ {"name":"projectname", "kind":"required", "type":"filepath=*.prj"} ] }, "coder": { "inputs": [ {"name":"-open", "kind":"namevalue", "type":"filepath=*.prj"} ] }, "coder": { "inputs": [ {"name":"-build", "kind":"namevalue", "type":"filepath=*.prj"} ] }, "coder": { "inputs": [ {"name":"-new", "kind":"namevalue", "type":[["filepath=*.prj"], ["char"]]} ] }, "coder": { "inputs": [ {"name":"ecoderFlag", "kind":"required", "type":"choices={'-ecoder'}"}, {"name":"ecoderValue", "kind":"required", "type":[["logical"], ["choices={'false'}"]]}, {"name":"newFlag", "kind":"required", "type":"choices={'-new'}"}, {"name":"newvalue", "kind":"required", "type":[["filepath=*.prj"], ["char"]]} ] }, "coder": { "inputs": [ {"name":"tocodeFlag", "kind":"required", "type":"choices={'-tocode'}"}, {"name":"tocodevalue", "kind":"required", "type":"filepath=*.prj"}, {"mutuallyExclusiveGroup": [ [], [ {"name":"scriptFlag", "kind":"required", "type":"choices={'-script'}"}, {"name":"scriptname", "kind":"required", "type":[["filepath=*.m"], ["char"]]} ] ] } ] }

This example also shows, in the last definition for the coder function, another mechanism for specifying alternative inputs, using “mutuallyExclusiveGroup” (aka “MEGs”). A MEG is defined using an array of options, enclosed in square brackets ([]). Each of the MEG options is exclusive to each of the others, meaning that we can only work with one of them and not the others. This is equivalent to duplicating the definition as we saw above, and saves us some copy-paste (in some cases a lot of copy-pastes, especially with multiple and/or nested MEGs). However, MEGs have a major drawback of reduced readability. I believe that in most cases we only have a single MEG and few input args, and in such cases it makes more sense to use repeated function defs rather than a MEG. The Matlab signature files contain numerous usage examples for either of these two mechanisms.

### Platform dependencies

If a specific function (or a specific signature variant) depends on the running platform, this can be specified via the “platform” directive. For example, the winopen function only works on Windows, but not on Linux/Mac. Its corresponding signature definition is:

"winopen": { "platform":"win32,win64", "inputs": [ {"name":"filename", "kind":"required", "type":"filepath"}, {"name":"varargin", "kind":"optional", "multiplicity":"append"} ] }

Platform dependence could also be specified at the parameter level, not just the entire function level. For example, in the xlsread function (defined in %matlabroot%/toolbox/matlab/iofun/functionSignatures.json), we see that the usage variant xlsread(filename,-1) is only available on Windows (note that the numeric value is defined as "<=-1", not necessarily -1), and so is the “functionHandle” input (which is called processFcn in the documentation – for some reason that escapes me the names of many input args do not match in the documentation and functionSignature):

"xlsread": { "inputs": [ {"name":"filename", "kind":"required", "type":"filepath=*.xls,*.xlsx,*.xlsb,*.csv"}, {"mutuallyExclusiveGroup": [ {"name":"openExcel", "kind":"required", "display":"", "type":["numeric", "<=-1"], "platform":"win64,win32"}, {"name":"xlRange", "kind":"required", "type":["char", "@(x) isempty(x) || ~isempty(strfind(x, ':'))"], "default":"''"}, [ {"name":"sheet", "kind":"positional", "type":[["char", "choices=matlab.internal.language.introspective.tabcompletion.xlsread_vsheet(filename)"], ["numeric", ">=1"]], "default":"1"}, {"name":"xlRange", "kind":"positional", "type":"char", "default":"''"}, {"name":"basic", "kind":"positional", "display":"", "type":["char", "choices={'basic',''}"]}, {"name":"functionHandle", "kind":"positional", "type":"function_handle", "platform":"win64,win32"} ] ] } ], ...

### Parsing errors

The new mechanism is not very user-friendly when you get something wrong. In the best case, it issues a cryptic error message (see below), and in the worst case it simply ignores the changes and the user has no idea why the new custom tab-completion is not working as intended.

The most likely causes of such problems are:

• The most common problem is that you placed the functionSignatures.json file in a different folder than the Matlab function. For example, if the myFunction() function is defined in myFunction.m, then the tab-completion of this function MUST be located in a functionSignatures.json file that resides in the same folder, not anywhere else on the Matlab path. In other words, the Matlab path is NOT relevant for tab-completion.
• Your functionSignatures.json file does not follow the [extremely strict] syntax rules above, to the letter. For example, forgetting the top or final curly braces, forgetting a comma or adding an extra one, or not closing all brackets/braces properly.
• You mistyped one or more of the input parameters, types or options.

In case of a parsing error, you’d see a red error message on the Matlab console the next time that you try to use tab-completion:

Error parsing JSON data; Boost reports "(189): expected ',' or ']'".

Unfortunately the error message only tells us the problematic line location within the functionSignatures.json file, but not the file’s location, so if we haven’t recently edited this file we’d need to find it in the relevant folder. For example:

edit(fullfile(fileparts(which('myFunction')), 'functionSignatures.json')

Moreover, when a JSON syntax error (such as the one above) occurs, the entire file is not parsed, not just the definition that caused the error.

Another limitation of tab-completion is that it does not work while the main Matlab thread is working (e.g., during a uiwait or waitfor). This may be somewhat misleading since most editor/debugging actions do work.

Arguably, this new tab-completion mechanism could be made more programmer-friendly. Perhaps this will improve in a future Matlab release.

For a related mechanism, see my article on tab-completion for class properties and methods from 2014, which is apparently still relevant and functional.

]]>
https://undocumentedmatlab.com/blog/user-defined-tab-completions-take-2/feed 6
Matlab compilation quirks – take 2https://undocumentedmatlab.com/blog/matlab-compilation-quirks-take-2 https://undocumentedmatlab.com/blog/matlab-compilation-quirks-take-2#respond Wed, 31 May 2017 18:00:42 +0000 http://undocumentedmatlab.com/?p=6919

Related posts:
1. Matlab compiler bug and workaround Both the Matlab compiler and the publish function have errors when parsing block-comments in Matlab m-code. ...
2. UDD Properties UDD provides a very convenient way to add customizable properties to existing Matlab object handles...
3. Disabling menu entries in deployed docked figures Matlab's standard menu items can and should be removed from deployed docked figures. This article explains how. ...
4. Handle Graphics Behavior HG behaviors are an important aspect of Matlab graphics that enable custom control of handle functionality. ...

]]>
Once again I would like to welcome guest blogger Hanan Kavitz of Applied Materials. Hanan posted a couple of guest posts here over the past few years, including a post last year about quirks with Matlab-compiled DLLs. Today Hanan will follow up on that post by discussing several additional quirks that they have encountered with Matlab compilations/deployment.

### Don’t fix it, if it ain’t broke…

In Applied Materials Israel (PDC) we use Matlab code for both algorithm development and deployment (production). As part of the dev-ops build system, which builds our product software versions, we build Matlab artifacts (binaries) from the Matlab source code.

A typical software version has several hundreds Matlab artifacts that are automatically rebuilt on a daily basis, and we have many such versions – totaling many thousands of compilations each day.

This process takes a long time, so we were looking for a way to make it more efficient.

The idea that we chose to implement sounds simple – take a single binary module in any software version (Ex. foo.exe – Matlab-compiled exe) and check it: if the source code for this module has not changed since the last compilation then simply don’t compile it, just copy it from previous software version repository. Since most of our code doesn’t change daily (some of it hasn’t changed in years), we can skip the compilation time of most binaries and just copy them from some repository of previously compiled binaries.

In a broader look, avoiding lengthy compilations cycles by not compiling unchanged code is a common programming practice, implemented by all modern compilers. For example, the ‘make’ utility uses a ‘makefile’ to check the time stamps of all dependencies of every object file in order to decide which object requires recompilation. In reality, this is not always the best solution as time stamps may be incorrect, but it works well in the vast majority of cases.

Coming back to Matlab, now comes the hard part – how could our build system know that nothing has changed in module X and that something has changed in module Y? How does it even know which source files it needs to ensure didn’t change?

The credit for the idea goes to my manager, Lior Cohen, as follows: You can actually check the dependency of a given binary after compilation. The basis of the solution is that a Matlab executable is in fact a compressed (zip) file. The idea is then to:

1. Compile the binary once
2. Unzip the binary and “see” all your dependencies (source files are encrypted and resources are not, but we only need the list of file names – not their content).
3. Now build a list of all your dependency files and compute the CRC value of each from the source control. Save it for the next time you are required to compile this module.
4. In the next compilation cycle, find this dependency list, review it, dependency source file at a time and make sure CRC of the dependency hasn’t changed since last time.
5. If no dependency CRC has changed, then copy the binary from the repository of previous software version, without compiling.
6. Otherwise, recompile the binary and rebuild the CRC list of all dependencies again, in preparation for the next compilation cycle.

That’s it! That simple? Well… not really – the reality is a bit more complex since there are many other dependencies that need to be checked. Some of them are:

1. Did the requested Matlab version of the binary change since the last compilation?
2. Did the compilation instructions themselves (we have a sort of ‘makefile’) change?

Basically, I implemented a policy that if anything changed, or if the dependency check itself failed, then we don’t take any chances and just compile this binary. Keeping in mind that this dependencies check and file copying is much faster than a Matlab compilation, we save a lot of actual compilation time using this method.

Bottom line: Given a software version containing hundreds of compilation instructions to execute and assuming not much has changed in the version (which is often the case), we skip over 90% of compilations altogether and only rebuild what really changed. The result is a version build that takes about half an hour, instead of many hours. Moreover, since the compilation process is working significantly less, we get fewer failures, fewer stuck or crashed mcc processes, and [not less importantly] less maintenance required by me.

Note that in our implementation we rely on the undocumented fact that Matlab binaries are in fact compressed zip archives. If and when a future Matlab release will change the implementation such that the binaries will no longer be zip archives, another way will need to be devised in order to ensure the consistency of the target executable with its dependent source files.

### Don’t kill it, if it ain’t bad…

I want to share a very weird issue I investigated over a year ago when using Matlab compiled exe. It started with a user showed me a Matlab compiled exe that didn’t run – I’m not talking about a regular Matlab exception: the process was crashing with an MS Windows popup window popping, stating something very obscure.

It was a very weird behavior that I couldn’t explain – the compiler seemed to work well but the compiled executable process kept crashing. Compiling completely different code showed the same behavior.

This issue has to do with the system compiler configuration that is being used. As you might know, when installing the Matlab compiler, before the first compilation is ever made, the user has to state the C compiler that the Matlab compiler should use in its compilation process. This is done by command ‘mbuild –setup’. This command asks the users to choose the C compiler and saves the configuration (batch file back then, xml in the newer versions of Matlab) in the user’s prefdir folder. At the time we were using Microsoft Visual C++ compiler 9.0 SP1.

The breakthrough in the investigation came when I ran mcc command with –verbose flag, which outputs much more compilation info than I would typically ever want… I discovered that although the target executable file had been created, a post compilation step failed to execute, while issuing a very cryptic error message:

mt.exe : general error c101008d: Failed to write the updated manifest to the resource of file “…”. Access is denied.

cryptic compilation error (click to zoom)

The failure was in one of the ‘post link’ commands in the configuration batch file – something obscure such as this:

set POSTLINK_CMDS2=mt.exe -outputresource: %MBUILD_OUTPUT_FILE_NAME%;%MANIFEST_RESOURCE% -manifest "%MANIFEST_FILE_NAME%"

This line of code takes an XML manifest file and inserts it into the generated binary file (additional details).

If you open a valid R2010a (and probably other old versions as well) Matlab-generated exe in a text editor you can actually see a small XML code embedded in it, while in a non-functioning exe I could not see this XML code.

So why would this command fail?

It turned out, as funny as it sounds, to be an antivirus issue – our IT department updated its antivirus policies and this ‘post link’ command suddenly became an illegal operation. Once our IT eased the policy, this command worked well again and the compiled executables stopped crashing, to our great joy.

]]>
https://undocumentedmatlab.com/blog/matlab-compilation-quirks-take-2/feed 0
GUI formatting using HTMLhttps://undocumentedmatlab.com/blog/gui-formatting-using-html https://undocumentedmatlab.com/blog/gui-formatting-using-html#comments Wed, 05 Apr 2017 20:26:44 +0000 http://undocumentedmatlab.com/?p=6877

Related posts:
1. Icon images & text in Matlab uicontrols HTML can be used to add image icons to Matlab listbox and popup (drop-down) controls. ...
2. Spicing up Matlab uicontrol tooltips Matlab uicontrol tooltips can be spiced-up using HTML and CSS, including fonts, colors, tables and images...
3. Multi-line uitable column headers Matlab uitables can present long column headers in multiple lines, for improved readability. ...
4. Rich-contents log panel Matlab listboxes and editboxes can be used to display rich-contents HTML-formatted strings, which is ideal for log panels. ...

]]>
As I’ve mentioned several times in the past, HTML can be used for simple formatting of GUI controls, including font colors/sizes/faces/angles. With a bit of thought, HTML (and some CSS) can also be used for non-trivial formatting, that would otherwise require the use of Java, such as text alignment, background color, and using a combination of text and icons in the GUI control’s contents.

### Alignment

For example, a question that I am often asked (latest example) is whether it is possible to left/center/right align the label within a Matlab button, listbox or table. While Matlab does not (yet) have properties that control alignment in uicontrols, we can indeed use HTML for this. There’s a catch though: if we simply tried to use <div align="left">…, it will not work. No error will be generated but we will not see any visible left-alignment. The reason is that internally, the text is contained within a snugly-fitting box. Aligning anything within a tight-fitting box obviously has no effect.

To solve the problem, we need to tell Matlab (or rather, the HTML interpreter used by the underlying Java control) to widen this internal box. One way to do this is to specify the width of the div tag, which can be enormous in order to span the entire available apace (<div width="999px" align="left">…). Another method is to simulate a simple HTML table that contains a single cell that holds the text, and then tell HTML the table cell’s width:

hButton.String = '<html><tr><td width=9999 align=left>Left-aligned'; % left-align within a button hTable.Data{2,1} = '<html><tr><td width=9999 align=right>And right'; % right-align within a specific uitable cell

Centered (default) and right-aligned button labels

Non-default alignment of uitable cells

I discussed the specific aspect of uicontrol content alignment in another post last year.

### Background color

The same problem (and solution) applies to background colors: if we don’t enlarge the snugly-fitting internal bounding-box, any HTML bgcolor that we specify would only be shown under the text (i.e., within the internal box’s confines). In order to display bgcolor across the entire control/cell width, we need to enlarge the internal box’s width (the align and bgcolor tags can of course be used together):

hButton.String = '<html><tr><td width=9999 bgcolor=#ffff00>Yellow'; % bgcolor within a button hTable.Data{2,1} = '<html><tr><td width=9999 bgcolor=#ffff00>Yellow'; % bgcolor within a specific uitable cell

### CSS

We can also use simple CSS, which provides more formatting customizability than plain HTML:

hTable.Data{2,1} = '<html><tr><td width=9999 style="background-color:yellow">Yellow';

HTML/CSS formatting is a poor-man’s hack. It is very crude compared to the numerous customization options available via Java. However, it does provide a reasonable solution for many use-cases, without requiring any Java. I discussed the two approaches for uitable cell formatting in this post.

### [Non-]support in uifigures

Important note: HTML formatting is NOT [yet] supported by the new web-based uifigures. While uifigures can indeed be hacked with HTML/CSS content (details), this is not an easy task. Since it should be trivially easy for MathWorks to enable HTML content in the new web-based uifigures, I implore anyone who uses HTML in their Matlab GUI to let MathWorks know about it so that they could prioritize this R&D effort into an upcoming Matlab release. You can send an email to George.Caia at mathworks.com, who apparently handles such aspects in MathWorks’ R&D efforts to transition from Java-based GUIs to web-based ones. In my previous post I spotlit MathWorks user-feedback surveys about users’ use of Java GUI aspects, aimed in order to migrate as many of the use-cases as possible onto the new web-based framework. HTML/CSS support is a natural by-product of the fact that Matlab’s non-web-based GUI is based on Java Swing components (that inherently support HTML/CSS). But unfortunately the MathWorks surveys are specific to the javacomponent function and the figure’s JavaFrame property. In other words, many users might be using undocumented Java aspects by simply using HTML content in their GUI, without ever realizing it or using javacomponent. So I think that in this case a simple email to George.Caia at mathworks.com to let him know how you’re using HTML would be more useful. Maybe one day MathWorks will be kind enough to post a similar survey specific to HTML support, or maybe one day they’s just add the missing HTML support, if only to be done with my endless nagging.

p.s. – I am well aware that we can align and bgcolor buttons in AppDesigner. But we can’t do this with individual table/listbox cells, and in general we can’t use HTML within uifigures without extensive hacks. I merely used the simple examples of button and uitable cell formatting in today’s post to illustrate the issue. So please don’t get hung up on the specifics, but rather on the broader issue of HTML support in uifigures.

And in the meantime, for as long as non-web-based GUI is still supported in Matlab, keep on enjoying the benefits that HTML/CSS provides.

### Automated bug-fix emails

In an unrelated matter, I wish to express my Kudos to the nameless MathWorkers behind the scenes who, bit by bit, improve Matlab and the user experience: Over the years I’ve posted a few times my frustrations with the opaqueness of MathWorks’ bug-reporting mechanism. One of my complaints was that users who file bugs are not notified when a fix or workaround becomes available. That at least seems to have been fixed now. I just received a seemingly-automated email notifying me that one of the bugs that I reported a few years ago has been fixed. This is certainly a good step in the right direction, so thank you!

Happy Passover/Easter to all!

]]>
https://undocumentedmatlab.com/blog/gui-formatting-using-html/feed 6

Related posts:
1. uiundo – Matlab’s undocumented undo/redo manager The built-in uiundo function provides easy yet undocumented access to Matlab's powerful undo/redo functionality. This article explains its usage....
2. Undocumented Profiler options part 3 An undocumented feature of the Matlab Profiler can report call history timeline - part 3 of series. ...
3. Undocumented Profiler options part 4 Several undocumented features of the Matlab Profiler can make it much more useful - part 4 of series. ...
4. Pinning annotations to graphs Annotation object can be programmatically set at, and pinned-to, plot axes data points. ...

]]>
Matlab’s license function returns the primary license number/ID used by Matlab, but no information about the various toolboxes that may be installed. The ver function returns a bit more information, listing the version number and installation date of installed toolboxes (even user toolboxes, such as my IB-Matlab toolbox). However, no additional useful information is provided beyond that:

>> license ans = 123456 % actual number redacted   >> ver ---------------------------------------------------------------------------------------------------- MATLAB Version: 9.1.0.441655 (R2016b) MATLAB License Number: 123456 Operating System: Microsoft Windows 7 Professional Version 6.1 (Build 7601: Service Pack 1) Java Version: Java 1.7.0_60-b19 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode ---------------------------------------------------------------------------------------------------- MATLAB Version 9.1 (R2016b) Curve Fitting Toolbox Version 3.5.4 (R2016b) Database Toolbox Version 7.0 (R2016b) Datafeed Toolbox Version 5.4 (R2016b) Financial Instruments Toolbox Version 2.4 (R2016b) Financial Toolbox Version 5.8 (R2016b) GUI Layout Toolbox Version 2.2.1 (R2015b) Global Optimization Toolbox Version 3.4.1 (R2016b) IB-Matlab - Matlab connector to InteractiveBrokers Version 1.89 Expires: 1-Apr-2018 Image Processing Toolbox Version 9.5 (R2016b) MATLAB Coder Version 3.2 (R2016b) MATLAB Report Generator Version 5.1 (R2016b) Optimization Toolbox Version 7.5 (R2016b) Parallel Computing Toolbox Version 6.9 (R2016b) Statistical Graphics Toolbox Version 1.2 Statistics and Machine Learning Toolbox Version 11.0 (R2016b)   >> v = ver v = 1×16 struct array with fields: Name Version Release Date   >> v(1) ans = struct with fields:   Name: 'Curve Fitting Toolbox' Version: '3.5.4' Release: '(R2016b)' Date: '25-Aug-2016'   >> v(8) ans = struct with fields:   Name: 'IB-Matlab - Matlab connector to InteractiveBrokers' Version: '1.89' Release: 'Expires: 1-Apr-2018' Date: '02-Feb-2017'

It is sometimes useful to know which license number “owns” which product/toolbox, and the expiration date is associated with each of them. Unfortunately, there is no documented way to retrieve this information in Matlab – the only documented way is to go to your account section on the MathWorks website and check there.

Luckily, there is a simpler way that can be used to retrieve additional information, from right inside Matlab, using matlab.internal.licensing.getFeatureInfo:

>> all_data = matlab.internal.licensing.getFeatureInfo all_data = 23×1 struct array with fields: feature expdate keys license_number entitlement_id   >> all_data(20) ans = struct with fields:   feature: 'optimization_toolbox' expdate: '31-mar-2018' keys: 0 license_number: '123456' entitlement_id: '1409891'   >> all_data(21) ans = struct with fields:   feature: 'optimization_toolbox' expdate: '07-mar-2017' keys: 0 license_number: 'DEMO' entitlement_id: '3749959'

As can be seen in this example, I have the Optimization toolbox licensed under my main Matlab license (123456 [actual number redacted]) until 31-mar-2018, and also licensed under a trial (DEMO) license that expires in 3 weeks. As long as a toolbox has any future expiration date, it will continue to function, so in this case I’m covered until March 2018.

We can also request information about a specific toolbox (“feature”):

>> data = matlab.internal.licensing.getFeatureInfo('matlab') data = 3×1 struct array with fields: feature expdate keys license_number entitlement_id   >> data(1) data = struct with fields:   feature: 'matlab' expdate: '31-mar-2018' keys: 0 license_number: '123456' entitlement_id: '1409891'

The drawback of this functionality is that it only provides information about MathWorks’ toolbox, not any user-provided toolboxes (such as my IB-Matlab connector, or MathWorks’ own GUI Layout toolbox). Also, some of the toolbox names may be difficult to understand (“gads_toolbox” apparently stands for the Global Optimization Toolbox, for example):

>> {all_data.feature} ans = 1×23 cell array Columns 1 through 4 'curve_fitting_toolbox' 'database_toolbox' 'datafeed_toolbox' 'distrib_computing_toolbox' Columns 5 through 8 'distrib_computing_toolbox' 'excel_link' 'fin_instruments_toolbox' 'financial_toolbox' Columns 9 through 15 'gads_toolbox' 'gads_toolbox' 'image_toolbox' 'image_toolbox' 'matlab' 'matlab' 'matlab' Columns 16 through 20 'matlab_coder' 'matlab_coder' 'matlab_report_gen' 'matlab_report_gen' 'optimization_toolbox' Columns 21 through 23 'optimization_toolbox' 'optimization_toolbox' 'statistics_toolbox'

A related undocumented builtin function is matlab.internal.licensing.getLicInfo:

% Information on a single toolbox/product: >> matlab.internal.licensing.getLicInfo('matlab') ans = struct with fields:   license_number: {'123456' 'Prerelease' 'T3749959'} expiration_date: {'31-mar-2018' '30-sep-2016' '07-mar-2017'}   % Information on multiple toolboxes/products: >> matlab.internal.licensing.getLicInfo({'matlab', 'image_toolbox'}) % cell array of toolbox/feature names ans = 1×2 struct array with fields: license_number expiration_date   % The full case-insensitive names of the toolboxes can also be used: >> matlab.internal.licensing.getLicInfo({'Matlab', 'Image Processing toolbox'}) ans = 1×2 struct array with fields: license_number expiration_date   % And here's how to get the full list (MathWorks products only): >> v=ver; data=matlab.internal.licensing.getLicInfo({v.Name}) data = 1×16 struct array with fields: license_number expiration_date

I have [still] not found any way to associate a user toolbox/product (such as my IB-Matlab) in a way that will report it in a unified manner with the MathWorks products. If anyone finds a way to do this, please do let me know.

p.s. – don’t even think of asking questions or posting comments on this website related to illegal uses or hacks of the Matlab license…

]]>
Quirks with parfor vs. forhttps://undocumentedmatlab.com/blog/quirks-with-parfor-vs-for https://undocumentedmatlab.com/blog/quirks-with-parfor-vs-for#comments Thu, 05 Jan 2017 17:15:48 +0000 http://undocumentedmatlab.com/?p=6821

Related posts:
1. Matlab mex in-place editing Editing Matlab arrays in-place can be an important technique for optimizing calculations. This article shows how to do it using Mex. ...
2. Preallocation performance Preallocation is a standard Matlab speedup technique. Still, it has several undocumented aspects. ...
3. Array resizing performance Several alternatives are explored for dynamic array growth performance in Matlab loops. ...
4. Matlab’s internal memory representation Matlab's internal memory structure is explored and discussed. ...

]]>
A few months ago, I discussed several tips regarding Matlab’s parfor command, which is used by the Parallel Computing Toolbox (PCT) for parallelizing loops. Today I wish to extend that post with some unexplained oddities when using parfor, compared to a standard for loop.

### Data serialization quirks

Dimitri Shvorob may not appear at first glance to be a prolific contributor on Matlab Central, but from the little he has posted over the years I regard him to be a Matlab power-user. So when Dimitri reports something, I take it seriously. Such was the case several months ago, when he contacted me regarding very odd behavior that he saw in his code: the for loop worked well, but the parfor version returned different (incorrect) results. Eventually, Dimitry traced the problem to something originally reported by Dan Austin on his Fluffy Nuke It blog.

The core issue is that if we have a class object that is used within a for loop, Matlab can access the object directly in memory. But with a parfor loop, the object needs to be serialized in order to be sent over to the parallel workers, and deserialized within each worker. If this serialization/deserialization process involves internal class methods, the workers might see a different version of the class object than the one seen in the serial for loop. This could happen, for example, if the serialization/deserialization method croaks on an error, or depends on some dynamic (or random) conditions to create data.

In other words, when we use data objects in a parfor loop, the data object is not necessarily sent “as-is”: additional processing may be involved under the hood that modify the data in a way that may be invisible to the user (or the loop code), resulting in different processing results of the parallel (parfor) vs. serial (for) loops.

For additional aspects of Matlab serialization/deserialization, see my article from 2 years ago (and its interesting feedback comments).

### Data precision quirks

The following section was contributed by guest blogger Lior Perlmuter-Shoshany, head algorithmician at a private equity fund.

In my work, I had to work with matrixes in the order of 109 cells. To reduce the memory footprint (and hopefully also improve performance), I decided to work with data of type single instead of Matlab’s default double. Furthermore, in order to speed up the calculation I use parfor rather than for in the main calculation. In the end of the run I am running a mini for-loop to see the best results.

What I discovered to my surprise is that the results from the parfor and for loop variants is not the same!

The following simplified code snippet illustrate the problem by calculating a simple standard-deviation (std) over the same data, in both single– and double-precision. Note that the loops are ran with only a single iteration, to illustrate the fact that the problem is with the parallelization mechanism (probably the serialization/deserialization parts once again), not with the distribution of iterations among the workers.

clear rng('shuffle','twister');   % Prepare the data in both double and single precision arr_double = rand(1,100000000); arr_single = single(arr_double);   % No loop - direct computation std_single0 = std(arr_single); std_double0 = std(arr_double);   % Loop #1 - serial for loop std_single = 0; std_double = 0; for i=1 std_single(i) = std(arr_single); std_double(i) = std(arr_double); end   % Loop #2 - parallel parfor loop par_std_single = 0; par_std_double = 0; parfor i=1 par_std_single(i) = std(arr_single); par_std_double(i) = std(arr_double); end   % Compare results of for loop vs. non-looped computation isForSingleOk = isequal(std_single, std_single0) isForDoubleOk = isequal(std_double, std_double0)   % Compare results of single-precision data (for vs. parfor) isParforSingleOk = isequal(std_single, par_std_single) parforSingleAccuracy = std_single / par_std_single   % Compare results of double-precision data (for vs. parfor) isParforDoubleOk = isequal(std_double, par_std_double) parforDoubleAccuracy = std_double / par_std_double

Output example :

isForSingleOk = 1 % <= true (of course!) isForDoubleOk = 1 % <= true (of course!)   isParforSingleOk = 0 % <= false (odd!) parforSingleAccuracy = 0.73895227413361 % <= single-precision results are radically different in parfor vs. for   isParforDoubleOk = 0 % <= false (odd!) parforDoubleAccuracy = 1.00000000000021 % <= double-precision results are almost [but not exactly] the same in parfor vs. for

From my testing, the larger the data array, the bigger the difference is between the results of single-precision data when running in for vs. parfor.

In other words, my experience has been that if you have a huge data matrix, it’s better to parallelize it in double-precision if you wish to get [nearly] accurate results. But even so, I find it deeply disconcerting that the results are not exactly identical (at least on R2015a-R2016b on which I tested) even for the native double-precision .

Hmmm… bug?

### Upcoming travels – Zürich & Geneva

I will shortly be traveling to clients in Zürich and Geneva, Switzerland. If you are in the area and wish to meet me to discuss how I could bring value to your work with some advanced Matlab consulting or training, then please email me (altmany at gmail):

• Zürich: January 15-17
• Geneva: January 18-21

Happy new year everybody!

]]>
https://undocumentedmatlab.com/blog/quirks-with-parfor-vs-for/feed 7
Afterthoughts on implicit expansionhttps://undocumentedmatlab.com/blog/afterthoughts-on-implicit-expansion https://undocumentedmatlab.com/blog/afterthoughts-on-implicit-expansion#comments Wed, 30 Nov 2016 20:28:44 +0000 http://undocumentedmatlab.com/?p=6750

Related posts:
1. tic / toc – undocumented option Matlab's built-in tic/toc functions have an undocumented option enabling multiple nested clockings...
2. Plot LimInclude properties The plot objects' XLimInclude, YLimInclude, ZLimInclude, ALimInclude and CLimInclude properties are an important feature, that has both functional and performance implications....
3. Matrix processing performance Matrix operations performance is affected by internal subscriptions in a counter-intuitive way....
4. Performance: accessing handle properties Handle object property access (get/set) performance can be significantly improved using dot-notation. ...

]]>
Matlab release R2016b introduced implicit arithmetic expansion, which is a great and long-awaited natural expansion of Matlab’s arithmetic syntax (if you are still unaware of this or what it means, now would be a good time to read about it). This is a well-documented new feature. The reason for today’s post is that this new feature contains an undocumented aspect that should very well have been documented and even highlighted.

The undocumented aspect that I’m referring to is the fact that code that until R2016a produced an error, in R2016b produces a valid result:

% R2016a >> [1:5] + [1:3]' Error using + Matrix dimensions must agree.   % R2016b >> [1:5] + [1:3]' ans = 2 3 4 5 6 3 4 5 6 7 4 5 6 7 8 

This incompatibility is indeed documented, but not where it matters most (read on).

I first discovered this feature by chance when trying to track down a very strange phenomenon with client code that produced different numeric results on R2015b and earlier, compared to R2016a Pre-release. After some debugging the problem was traced to a code snippet in the client’s code that looked something like this (simplified):

% Ensure compatible input data try dataA + dataB; % this will (?) error if dataA, dataB are incompatible catch dataB = dataB'; end

The code snippet relied on the fact that incompatible data (row vs. col) would error when combined, as it did up to R2015b. But in R2016a Pre-release it just gave a valid numeric matrix, which caused numerically incorrect results downstream in the code. The program never crashed, so everything appeared to be in order, it just gave different numeric results. I looked at the release notes and none of the mentioned release incompatibilities appeared relevant. It took me quite some time, using side-by-side step-by-step debugging on two separate instances of Matlab (R2015b and R2016aPR) to trace the problem to this new feature.

This implicit expansion feature was removed from the official R2016a release for performance reasons. This was apparently fixed in time for R2016b’s release.

I’m totally in favor of this great new feature, don’t get me wrong. I’ve been an ardent user of bsxfun for many years and (unlike many) have even grown fond of it, but I still find the new feature to be better. I use it wherever there is no significant performance penalty, a need to support older Matlab releases, or a possibility of incorrect results due to dimensional mismatch.

### So what’s my point?

What I am concerned about is that I have not seen the new feature highlighted as a potential backward compatibility issue in the documentation or the release notes. Issues of far lesser importance are clearly marked for their backward incompatibility in the release notes, but not this important major change. A simple marking of the new feature with the warning icon () and in the “Functionality being removed or changed” section would have saved my client and me a lot of time and frustration.

MathWorks are definitely aware of the potential problems that the new feature might cause in rare use cases such as this. As Steve Eddins recently noted, there were plenty of internal discussions about this very thing. MathWorks were careful to ensure that the feature’s benefits far outweigh its risks (and I concur). But this also highlights the fact that MathWorks were fully aware that in some rare cases it might indeed break existing code. For those cases, I believe that they should have clearly marked the incompatibility implications in the release notes and elsewhere.

I have several clients who scour Matlab’s release notes before each release, trying to determine the operational risk of a Matlab upgrade. Having a program that returns different results in R2016b compared to R2016a, without being aware of this risk, is simply unacceptable to them, and leaves users with a disinclination to upgrade Matlab, to MathWorks’ detriment.

MathWorks in general are taking a very serious methodical approach to compatibility issues, and are clearly investing a lot of energy in this (a recent example). It’s too bad that sometimes this chain is broken. I find it a pity, and think that this can still be corrected in the online doc pages. If and when this is fixed, I’ll be happy to post an addendum here.

In my humble opinion from the backbenches, increasing the transparency on compatibility issues and open bugs will increase user confidence and result in greater adoption and upgrades of Matlab. Just my 2 cents…

Today MathWorks added the following compatibility warning to the release notes (R2016b, Mathematics section, first item) – thanks for listening MathWorks

]]>
https://undocumentedmatlab.com/blog/afterthoughts-on-implicit-expansion/feed 12
Icon images & text in Matlab uicontrolshttps://undocumentedmatlab.com/blog/icon-images-in-matlab-uicontrols https://undocumentedmatlab.com/blog/icon-images-in-matlab-uicontrols#comments Wed, 28 Sep 2016 10:28:04 +0000 http://undocumentedmatlab.com/?p=6687

Related posts:
1. Spicing up Matlab uicontrol tooltips Matlab uicontrol tooltips can be spiced-up using HTML and CSS, including fonts, colors, tables and images...
2. Rich-contents log panel Matlab listboxes and editboxes can be used to display rich-contents HTML-formatted strings, which is ideal for log panels. ...
3. Aligning uicontrol contents Matlab uicontrols can often be customized using plain HTML/CSS, without need for advanced Java. ...
4. GUI integrated browser control A fully-capable browser component is included in Matlab and can easily be incorporated in regular Matlab GUI applications. This article shows how....

]]>
One of my consulting clients recently asked me if I knew any builtin Matlab GUI control that could display a list of colormap names alongside their respective image icons, in a listbox or popup menu (drop-down/combo-box):

Matlab listbox (left) & popup menu (right) with icon images

My initial thought was that this should surely be possible, since Colormap is a documented figure property, that should therefore be listed inside the inspector window, and should therefore have an associated builtin Java control for the dropdown (just like other inspector controls, which are part of the com.mathworks.mlwidgets package, or possibly as a standalone control in the com.mathworks.mwswing package). To my surprise it turns out that for some unknown reason MathWorks neglected to add the Colormap property (and associated Java controls) to the inspector. This property is fully documented and all, just like Color and other standard figure properties, but unlike them Colormap can only be modified programmatically, not via the inspector window. Matlab does provide the related colormapeditor function and associated dialog window, but I would have expected a simple drop-down of the standard builtin colormaps to be available in the inspector. Anyway, this turned out to be a dead-end.

It turns out that we can relatively easily implement the requested listbox/combo-box using a bit of HTML magic, as I explained last week. The basic idea is for each of the listbox/combobox items to be an HTML string that contains both an <img> tag for the icon and the item label text. For example, such a string might contain something like this (parula is Matlab’s default colormap in HG2, starting in R2014b):

<html><img src="http://www.mathworks.com/help/matlab/ref/colormap_parula.png">parula

parula colormap image

Of course, it would be a bit inefficient for each of the icons to be fetched from the internet. Luckily, the full set of Matlab documentation is typically installed on the local computer as part of the standard Matlab installation, beneath the docroot folder (e.g., C:\Program Files\Matlab\R2016b\help). In our specific case, the parula colormap image is located in:

imageFilename = [docroot, '/matlab/ref/colormap_parula.png']

Note that for a local image to be accepted by HTML, it needs to follow certain conventions. In our case, the HTML string for displaying the above image is:

<html><img src="file:///C:/Program%20Files/Matlab/R2016b/help/matlab/ref/colormap_parula.png">parula

Warning: it’s easy when dealing with HTML images in Matlab to get the format confused, resulting in a red-x icon. I discussed this issue some 4 years ago, which is still relevant.

How can we get the list of available builtin colormaps? The standard Matlab way of doing this would be something like this:

>> possibleColormaps = set(gcf,'Colormap') possibleColormaps = {}

but as we can see, for some unknown reason (probably another MathWorks omission), Matlab does not list the names of its available builtin colormaps.

Fortunately, all the builtin colormaps have image filenames that follow the same convention, which make it easy to get this list by simply listing the names of the relevant files, from which we can easily create the necessary HTML strings:

>> iconFiles = dir([docroot, '/matlab/ref/colormap_*.png']);   >> colormapNames = regexprep({iconFiles.name}, '.*_(.*).png', '\$1') colormapNames = Columns 1 through 9 'autumn' 'bone' 'colorcube' 'cool' 'copper' 'flag' 'gray' 'hot' 'hsv' Columns 10 through 18 'jet' 'lines' 'parula' 'pink' 'prism' 'spring' 'summer' 'white' 'winter'   >> htmlStrings = strcat('<html><img width=200 height=10 src="file:///C:/Program%20Files/Matlab/R2016a/help/matlab/ref/colormap_', colormapNames', '.png">', colormapNames') str = '<html><img width=200 height=10 src="file:///C:/Program%20Files/Matlab/R2016a/help/matlab/ref/colormap_autumn.png">autumn' '<html><img width=200 height=10 src="file:///C:/Program%20Files/Matlab/R2016a/help/matlab/ref/colormap_bone.png">bone' '<html><img width=200 height=10 src="file:///C:/Program%20Files/Matlab/R2016a/help/matlab/ref/colormap_colorcube.png">colorcube' ...   >> hListbox = uicontrol(gcf, 'Style','listbox', 'Units','pixel', 'Pos',[10,10,270,200], 'String',htmlStrings); >> hPopup = uicontrol(gcf, 'Style','popup', 'Units','pixel', 'Pos',[10,500,270,20], 'String',htmlStrings);

…which results in the screenshots at the top of this post.

Note how I scaled the images to 10px high (so that the labels would be shown and not cropped vertically) and 200px wide (so that it becomes narrower than the default 434px). There’s really no need in this case for the full 434×27 image size – such flat images scale very nicely, even when their aspect ratio is not preserved. You can adjust the height and width values for a best fit with you GUI.

Unfortunately, it seems that HTML strings are not supported in the new web-based uifigure controls. This is not really Matlab’s fault because the way to customize labels in HTML controls is via CSS: directly embedding HTML code in labels does not work (it’s a Java-Swing feature, not a browser feature). I really hope that either HTML or CSS processing will be enabled for web-based uicontrol in a future Matlab release, because until that time uifigure uicontrols will remain seriously deficient compared to standard figure uicontrols. Until then, if we must use uifigures and wish to customize our labels or listbox items, we can directly access the underlying web controls, as Iliya explained here.

A blog reader recently complained that I’m abusing Swing and basically making Matlab work in unnatural ways, “something it was never meant to be“. I feel that using HTML as I’ve shown last week and in this post would fall under the same category in his eyes. To him and to others who complain I say that I have absolutely no remorse about doing this. When I purchase anything I have the full rights (within the scope of the license) to adapt it in whatever way fits my needs. As a software developer and manager for over 25 years, I’ve developed in dozens of programming languages and environments, and I still enjoy [ab]using Matlab. Matlab is a great environment to get things done quickly and if this sometimes requires a bit of HTML or Java hacks that make some people cringe, then that’s their problem, not mine – I’m content with being able to do in Matlab [nearly] everything I want, quickly, and move on to the next project. As long as it gets the job done, that’s fine by me. If this makes me more of an engineer than a computer scientist, then so be it.

On the flip side, I say to those who claim that Matlab is lacking in this or that aspect, that in most likelihood the limitation is only in their minds, not in Matlab – we can do amazing stuff with Matlab if we just open our minds, and possibly use some undocumented hacks. I’m not saying that Matlab has no limitations, I’m just saying that in most cases they can be overcome if we took the time and trouble to look for a solution. Matlab is a great tool and yet many people are not aware of its potential. Blaming Matlab for its failings is just an easy excuse in many cases. Of course, MathWorks could help my crusade on this subject by enabling useful features such as easy GUI component customizations…

On this sad day, I wish you all Shanah Tova!

]]>
https://undocumentedmatlab.com/blog/icon-images-in-matlab-uicontrols/feed 5