Presumed future risk – Undocumented Matlab https://undocumentedmatlab.com Charting Matlab's unsupported hidden underbelly Thu, 19 Jul 2018 20:59:48 +0000 en-US hourly 1 https://wordpress.org/?v=4.4.1 Multi-threaded Mexhttps://undocumentedmatlab.com/blog/multi-threaded-mex https://undocumentedmatlab.com/blog/multi-threaded-mex#respond Wed, 18 Jul 2018 14:56:44 +0000 https://undocumentedmatlab.com/?p=7771
 
Related posts:
  1. Explicit multi-threading in Matlab part 3 Matlab performance can be improved by employing POSIX threads in C/C++ code. ...
  2. Faster csvwrite/dlmwrite The speed of the builtin csvwrite, dlmwrite functions can be improved dramatically. ...
  3. Matlab mex in-place editing Editing Matlab arrays in-place can be an important technique for optimizing calculations. This article shows how to do it using Mex. ...
  4. Parsing mlint (Code Analyzer) output The Matlab Code Analyzer (mlint) has a lot of undocumented functionality just waiting to be used. ...
 
]]>
I was recently asked by a consulting client to help speed up a Matlab process. Accelerating MATLAB Performance Quite often there are various ways to improve the run-time, and in this particular case it turned out that the best option was to convert the core Matlab processing loop into a multi-threaded Mex function, while keeping the rest (vast majority of program code) in easy-to-maintain Matlab. This resulted in a 160x speedup (25 secs => 0.16 secs). Some of this speedup is attributed to C-code being faster in general than Matlab, another part is due to the multi-threading, and another due to in-place data manipulations that avoid costly memory access and re-allocations.

In today’s post I will share some of the insights relating to this MEX conversion, which could be adapted for many other similar use-cases. Additional Matlab speed-up techniques can be found in other performance-related posts on this website, as well in my book Accelerating MATLAB Performance.

There are quite a few online resources about creating Mex files, so I will not focus on this aspect. I’ll assume that the reader is already familiar with the concept of using Mex functions, which are simply dynamically-linked libraries that have a predefined entry-function syntax and predefined platform-specific extension. Instead, I’ll focus on how to create and debug a multi-threaded Mex function, so that it runs in parallel on all CPU cores.

The benefit of multi-threading is that threads are very light-weight objects, that have minimal performance and memory overheads. This contrasts to multi-tasking, which is what the Parallel Computing Toolbox currently does: launches duplicate copies of the entire Matlab engine process (“headless workers”) and then manages and coordinates the tasks to split up the processing work. Multi-tasking should be avoided wherever we can employ light-weight multi-threading instead. Unfortunately, Matlab does not currently have the ability to explicitly multi-thread Matlab code. But we can still use explicit multi-threading by invoking code in other languages, as I’ve already shown for Java, C# (and .NET in general), and C/C++. Today’s article will expand on the latter post (the one about C/C++ multi-threading), by showing a general framework for making a multi-threaded C-based Mex function.

There are several alternative implementation of threads. On non-Windows machines, POSIX threads (“pthreads”) are a de-facto standard; on Windows, which pthreads can still be used, they generally use native Windows threads under the hood, and we can use these native threads directly.

I have uploaded a file called max_in_place to the Matlab File Exchange. This function serves as an example showing a generic multi-threaded Mex function. A compiled version of this Mex file for Win64 is also included in the File Exchange entry, and you can run it directly on a Win64 machine.

The usage in Matlab is as follows (note how matrix1 is updated in-place):

>> matrix1 = rand(4)
matrix1 =
      0.89092      0.14929      0.81428      0.19664
      0.95929      0.25751      0.24352      0.25108
      0.54722      0.84072      0.92926      0.61604
      0.13862      0.25428      0.34998      0.47329
 
>> matrix2 = rand(4)
matrix2 =
      0.35166      0.91719      0.38045      0.53081
      0.83083      0.28584      0.56782      0.77917
      0.58526       0.7572     0.075854      0.93401
      0.54972      0.75373      0.05395      0.12991
 
>> max_in_place(matrix1, matrix2)
 
>> matrix1
matrix1 =
      0.89092      0.91719      0.81428      0.53081
      0.95929      0.28584      0.56782      0.77917
      0.58526      0.84072      0.92926      0.93401
      0.54972      0.75373      0.34998      0.47329

Source code and discussion

The pseudo-code for the MEX function is as follows:

mexFunction():
   validate the input/output args
   quick bail-out if empty inputs
   get the number of threads N from Matlab's maxNumCompThreads function
   if N == 1
       run main_loop directly
   else
       split input matrix #1 into N index blocks
       assign start/end index for each thread
       create and launch N new threads that run main_loop
       wait for all N threads to complete
       free the allocated memory resources
   end

Here’s the core source-code of this function, which was adapted from original work by Dirk-Jan Kroon:

/*====================================================================
 *
 * max_in_place.c  updates a data matrix in-place with the max value
 *                 of the matrix and a 2nd matrix of the same size
 *
 * The calling syntax is:
 *
 *		max_in_place(matrix1, matrix2)
 *
 * matrix1 will be updated with the maximal values from corresponding
 * indices of the 2 matrices
 *
 * Both inputs must be double 2D real non-sparse matrices of same size
 *
 * Yair Altman 2018-07-18
 * http://UndocumentedMatlab.com/blog/multi-threaded-mex
 *
 * Adapted from original work by Dirk-Jan Kroon
 * http://mathworks.com/matlabcentral/profile/authors/1097878-dirk-jan-kroon
 *
 *==================================================================*/
 
#include <math.h>
#include "mex.h"
 
/* undef needed for LCC compiler */
#undef EXTERN_C
#ifdef _WIN32
    #include <windows.h>
    #include <process.h>
#else
    #include <pthread.h>
#endif
 
/* Input Arguments */
#define	hMatrix1	prhs[0]
#define	hMatrix2	prhs[1]
 
/* Macros */
#if !defined(MAX)
#define	MIN(A, B)	((A) < (B) ? (A) : (B))
#endif
 
/* Main processing loop function */
void main_loop(const mxArray *prhs[], int startIdx, int endIdx)
{
    /* Assign pointers to the various parameters */
    double *p1 = mxGetPr(hMatrix1);
    double *p2 = mxGetPr(hMatrix2);
 
    /* Loop through all matrix coordinates */
    for (int idx=startIdx; idx<=endIdx; idx++)
    {
        /* Update hMatrix1 with the maximal value of hMatrix1,hMatrix2 */
        if (p1[idx] < p2[idx]) {
            p1[idx] = p2[idx];
        }
    }
}
 
/* Computation function in threads */
#ifdef _WIN32
  unsigned __stdcall thread_func(void *ThreadArgs_) {
#else
  void thread_func(void *ThreadArgs_) {
#endif
    double **ThreadArgs = ThreadArgs_;  /* void* => double** */
    const mxArray** prhs = (const mxArray**) ThreadArgs[0];
 
    int ThreadID = (int) ThreadArgs[1][0];
    int startIdx = (int) ThreadArgs[2][0];
    int endIdx   = (int) ThreadArgs[3][0];
    /*mexPrintf("Starting thread #%d: idx=%d:%d\n", ThreadID, startIdx, endIdx); */
 
    /* Run the main processing function */
    main_loop(prhs, startIdx, endIdx);
 
    /* Explicit end thread, helps to ensure proper recovery of resources allocated for the thread */
    #ifdef _WIN32
        _endthreadex( 0 );
        return 0;
    #else
        pthread_exit(NULL);
    #endif
}
 
/* validateInputs function here... */
 
/* Main entry function */
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
 
{
    /* Validate the inputs */
    validateInputs(nlhs, plhs, nrhs, prhs);
 
    /* Quick bail-out in the trivial case of empty inputs */
    if (mxIsEmpty(hMatrix1))  return;
 
    /* Get the number of threads from the Matlab engine (maxNumCompThreads) */
    mxArray *matlabCallOut[1] = {0};
    mxArray *matlabCallIn[1]  = {0};
    mexCallMATLAB(1, matlabCallOut, 0, matlabCallIn, "maxNumCompThreads");
    double *Nthreadsd = mxGetPr(matlabCallOut[0]);
    int Nthreads = (int) Nthreadsd[0];
 
    /* Get the number of elements to process */
    size_t n1 = mxGetNumberOfElements(hMatrix1);
 
    if (Nthreads == 1) {
 
        /* Process the inputs directly (not via a thread) */
        main_loop(prhs, 0, n1-1);
 
    } else {  /* multi-threaded */
 
        /* Allocate memory for handles of worker threads */
        #ifdef _WIN32
            HANDLE    *ThreadList = (HANDLE*)   malloc(Nthreads*sizeof(HANDLE));
        #else
            pthread_t *ThreadList = (pthread_t*)malloc(Nthreads*sizeof(pthread_t));
        #endif
 
        /* Allocate memory for the thread arguments (attributes) */
        double **ThreadID, **ThreadStartIdx, **ThreadEndIdx, ***ThreadArgs;
        double *ThreadID1, *ThreadStartIdx1, *ThreadEndIdx1, **ThreadArgs1;
 
        ThreadID       = (double **) malloc( Nthreads* sizeof(double *) );
        ThreadStartIdx = (double **) malloc( Nthreads* sizeof(double *) );
        ThreadEndIdx   = (double **) malloc( Nthreads* sizeof(double *) );
        ThreadArgs     = (double ***)malloc( Nthreads* sizeof(double **) );
 
        /* Launch the requested number of threads */
        int i;
        int threadBlockSize = ceil( ((double)n1) / Nthreads );
        for (i=0; i<Nthreads; i++)
        {
            /* Create thread ID */
            ThreadID1 = (double *)malloc( 1* sizeof(double) );
            ThreadID1[0] = i;
            ThreadID[i] = ThreadID1;
 
            /* Compute start/end indexes for this thread */
            ThreadStartIdx1 = (double *) malloc( sizeof(double) );
            ThreadStartIdx1[0] = i * threadBlockSize;
            ThreadStartIdx[i] = ThreadStartIdx1;
 
            ThreadEndIdx1 = (double *) malloc( sizeof(double) );
            ThreadEndIdx1[0] = MIN((i+1)*threadBlockSize, n1) - 1;
            ThreadEndIdx[i] = ThreadEndIdx1;
 
            /* Prepare thread input args */
            ThreadArgs1 = (double **) malloc( 4* sizeof(double*) );
            ThreadArgs1[0] = (double *) prhs;
            ThreadArgs1[1] = ThreadID[i];
            ThreadArgs1[2] = ThreadStartIdx[i];
            ThreadArgs1[3] = ThreadEndIdx[i];
 
            ThreadArgs[i] = ThreadArgs1;
 
            /* Launch the thread with its associated args */
            #ifdef _WIN32
                ThreadList[i] = (HANDLE)_beginthreadex(NULL, 0, &thread_func, ThreadArgs[i], 0, NULL);
            #else
                pthread_create ((pthread_t*)&ThreadList[i], NULL, (void *) &thread_func, ThreadArgs[i]);
            #endif
        }
 
        /* Wait for all the treads to finish working */
        #ifdef _WIN32
            for (i=0; i<Nthreads; i++) { WaitForSingleObject(ThreadList[i], INFINITE); }
            for (i=0; i<Nthreads; i++) { CloseHandle( ThreadList[i] ); }
        #else
            for (i=0; i<Nthreads; i++) { pthread_join(ThreadList[i],NULL); }
        #endif
 
        /* Free the memory resources allocated for the threads */
        for (i=0; i<Nthreads; i++)
        {
            free(ThreadArgs[i]);
            free(ThreadID[i]);
            free(ThreadStartIdx[i]);
            free(ThreadEndIdx[i]);
        }
 
        free(ThreadArgs);
        free(ThreadID );
        free(ThreadStartIdx);
        free(ThreadEndIdx);
        free(ThreadList);
    }
 
    return;
}

This file also includes a validateInputs function. I did not include it in the code snippet above for brevity; you can read it directly in the FEX entry (max_in_place.c). This function checks that there are exactly 0 output and 2 input args, that the input args are real non-sparse matrices and that they have the same number of elements.

Note that the threads run a generic thread_func function, which in turn runs the main_loop function with the thread’s specified startIndex, endIndex values. When this function completes, the thread ends itself explicitly, to ensure resource cleanup.

Also note how the thread code is using pthreads on non-Windows (!defined(_WIN32)) machines, and native Windows threads otherwise. This means that the same MEX source-code could be used on all Matlab platforms.

The important thing to note about this framework is that we no longer need to worry about the thread plumbing. If we wish to adapt this code for any other processing, we just need to modify the main_loop function with the new processing logic. In addition, you may wish to modify the validateInputs function based on your new setup of input/output args.

A few caveats:

  • On Windows machines with R2017b or older, we simply compile using mex max_in_place.c; on non-Windows we might need to add the –lpthread flag to link the pthreads library, depending on your specific compiler.
  • On R2018a or newer on all platforms, due to MEX’s new interleaved-complex memory format, we would need to compile with the -R2017b flag if we wish to use mexGetPr, as in the sample code above (in R2018a’s new data model, the corresponding function is mxGetDoubles). Note that updating data in-place becomes more difficult with the new MEX API, so if you want to preserve the performance boost that in-place data manipulation provides, it may be better to stick with the legacy data memory model.
  • The sample code above splits the data between the threads based on the first input matrix’s size. Instead, you may consider sending to the MEX function the loop indexes as extra input args, and then splitting those up between the threads.
  • In this specific implementation of max_in_place, I have updated the data locations directly. This is generally discouraged and risky, because it conflicts with Matlab’s standard Copy-on-Write mechanism. For example, if we assign the input to any other Matlab variable(s) before calling max_in_place, then that other variable(s) will also get their data updated. If we do not want this side-effect, we should mxUnshareArray the input matrix1, and return the resulting matrix as an output of the MEX function (plhs[0]).

Speed-up tips

The core logic in the specific case that I was asked to optimize was something similar to this:

main_process:
    initialize output matrix
    loop z over all slices in a 3D data matrix
        temp_data = data(:,:,z);
        temp_data = process(temp_data);
        output = max(output, temp_data);
    end z loop

The initial speed-up attempt was naturally focused on the process and max functions. Converting them to a MEX function improved the speed by a factor of ~8, but the resulting run-time (4-5 secs) was still too slow for real-time use. The reason that we did not see a larger speed-up was, I believe, due to several reasons:

  • temp_data was small enough such that the overheads associated with creating and then disposing separate threads were significant compared to the processing time of each thread.
  • temp_data was small enough such that each thread processed a relatively small portion of the memory, in contrast to single-threaded processing that accesses memory in larger blocks, more efficiently.
  • In each iteration of the z loop, the overheads associated with calling the MEX function, handling input variables and validation, creating/disposing threads, and allocating/deallocating memory for temp_data, were repeatedly paid.

So, while the profiling result showed that 98% of the time was spent in the MEX function (which would seem to indicate that not much additional speedup can be achieved), in fact the MEX function was under-performing because of the inefficiencies involved in repeatedly creating threads to process small data chunks. It turned out that running in single-thread mode was actually somewhat faster than multi-threaded mode.

I then moved the entire z loop (entire main_process) into the MEX function, where the threads were split to process separate adjacent blocks of z slices (i.e. different blocks of the z loop). This way, the MEX function was called, the inputs validated, and threads created/disposed only once for the entire process, making this overhead negligible compared to each thread’s processing time. Moreover, each thread now processed the entire temp_data belonging to its z slice, so memory access was more efficient, reducing the memory I/O wait time and improving the overall processing time. Additional benefits were due to the fact that some variables could be reused within each thread across loop iterations, minimizing memory allocations and deallocations. The overall effect was to reduce the total run-time down to ~0.16 secs, a 160x speedup compared to the original (25 secs). As my client said: “You sped up [the application] from practically useless to clinically useful.”

The lesson: try to minimize MEX invocation and thread creation/disposal overheads, and let the threads process as large adjacent memory blocks as possible.

Debugging MEX files

When debugging multi-threaded MEX functions, I find that it’s often easier to run the function in single-threaded mode to debug the core logic, and once this is ready we can switch back multi-threading. This can easily be done by setting the number of threads outside the MEX function using Matlab’s builtin maxNumCompThreads function:

Nthreads = maxNumCompThreads(1);  % temporarily use only 1 thread for debugging
max_in_place(matrix1, matrix2);
maxNumCompThreads(Nthreads);      % restore previous value
%maxNumCompThreads('automatic');  % alternative

Once debugging is done and the MEX function works properly, we should remove the maxNumCompThreads calls, so that the MEX function will use the regular number of Matlab computational threads, which should be the same as the number of cores: feature(‘numCores’).

I typically like to use Eclipse as my IDE for non-Matlab code (Java, C/C++ etc.). Unfortunately, there’s a problem attaching Eclipse to Matlab processes (which is necessary for interactive MEX debugging) if you’re using any recent (post-2015) version of MinGW and Eclipse. This problem is due to a known Eclipse bug, as user Lithe pointed out. The workaround is to install an old version of MinGW, *in addition* to your existing MinGW version. Reportedly, only versions 4.9.1 or older of MinGW include gdb 7.8 (which is still supported by Eclipse), whereas newer versions of MinGW include a newer gdb that is not supported. Download and install such an old MinGW version in a separate folder from your more-modern compiler. Don’t update your MEX to use the older MinGW – just tell Eclipse to use the version of gdb in the old MinGW bin/ folder when you set up a debug configuration for debugging your MEX files.

Once you have a compatible gdb, and ask Eclipse to attach to a process, the processes list will finally appear (it won’t with an incompatible gdb). Use feature('getPID') to get your Matlab process ID, which can then used to attach to the relevant process in the Eclipse Attach-to-process window. For example, if your Matlab’s PID is 4321, then the Matlab process will be called “Program – 4321” in Eclipse’s processes list.

I wish that MathWorks would update their official Answer and their MinGW support package on File Exchange to include this information, because without it debugging on Eclipse becomes impossible. Eclipse is so powerful, easy-to-use and ubiquitous that it’s a shame for most users not to be able to work with it just because the workarounds above are not readily explained.

N.B. If you don’t like Eclipse, you can also use Visual Studio Code (VS Code), as Andy Campbell recently explained in the MathWorks Developers’ blog.

Consulting

Do you have any Matlab code that could use a bit (or a lot) of speeding-up? If so, please contact me for a private consulting offer. I can’t promise to speed up your code by a similar factor of 160x, but you never know…

]]>
https://undocumentedmatlab.com/blog/multi-threaded-mex/feed 0
Plot legend customizationhttps://undocumentedmatlab.com/blog/plot-legend-customization https://undocumentedmatlab.com/blog/plot-legend-customization#comments Thu, 12 Jul 2018 14:11:40 +0000 https://undocumentedmatlab.com/?p=7744
 
Related posts:
  1. Plot LineSmoothing property LineSmoothing is a hidden and undocumented plot line property that creates anti-aliased (smooth unpixelized) lines in Matlab plots...
  2. Multi-column (grid) legend This article explains how to use undocumented axes listeners for implementing multi-column plot legends...
  3. Controlling plot data-tips Data-tips are an extremely useful plotting tool that can easily be controlled programmatically....
  4. getundoc – get undocumented object properties getundoc is a very simple utility that displays the hidden (undocumented) properties of a specified handle object....
 
]]>
Three years ago I explained how we can use a couple of undocumented hidden properties of the legend in order to add a legend title (the legend object had no Title property back then – this was only added in a later Matlab release, perhaps as a result of my post). Today I will expand on that article by explaining the plot legend’s internal graphics hierarchy, how we can access each of these components, and then how this information could be used to customize the separate legend components. Note that the discussion today is only relevant for HG2 legends (i.e. R2014b or newer).

Let’s start with a simple Matlab plot with a legend:

hold all; 
hLine1 = plot(1:5); 
hLine2 = plot(2:6); 
hLegend = legend([hLine1,hLine2], 'Location','SouthEast');
hLegend.Title.String = 'MyLegend';

Standard Matlab legend

Standard Matlab legend

This legend is composed of the following visible internal components, which can be customized separately:
Matlab legend components

Id in screenshotAccessed viaObject typeDescriptionImportant properties
1hLegend.TitleTextTitle of the legendVisible, String, Color, FontSize, FontWeight.
2hLegend.TitleSeparatorLineStripSeparator line between title and legend entries. Only appears when title is set.Visible, LineStyle, LineWidth, ColorData (4×1 uint8)
3hLegend.BoxEdgeLineLoopBox (border) line around the entire legend (including title)Visible, LineStyle, LineWidth, ColorData (4×1 uint8)
4hLegend.EntryContainer.NodeChildren(2)LegendEntryEntry row in the legend, corresponding to hLine1Icon, Label, Object (line object in main axes)
5hLegend.EntryContainer.NodeChildren(1)LegendEntryEntry row in the legend, corresponding to hLine2Icon, Label, Object (line object in main axes)
6hLegend.EntryContainer.NodeChildren(1).LabelTextLabel of legend entryVisible, String, Color, FontSize, FontWeight
7hLegend.EntryContainer.NodeChildren(1).IconLegendIconIcon/marker of legend entryVisible, Transform.Children.Children (LineStrip object)

A pivotal object of the legend group are the LegendEntry items, one per legend row:

>> hLegendEntry = hLegend.EntryContainer.NodeChildren(1);
>> get(hLegendEntry)
              Children: [3×1 Graphics]
                 Color: [0 0 0]
                 Dirty: 0
             FontAngle: 'normal'
              FontName: 'Helvetica'
              FontSize: 8
            FontWeight: 'normal'
      HandleVisibility: 'on'
               HitTest: 'on'
                  Icon: [1×1 LegendIcon]
                 Index: 0
           Interpreter: 'tex'
                 Label: [1×1 Text]
            LayoutInfo: [1×1 matlab.graphics.illustration.legend.ItemLayoutInfo]
                Legend: [1×1 Legend]
              Listener: [1×1 event.listener]
                Object: [1×1 Line]
               Overlay: [1×1 TriangleStrip]
          OverlayAlpha: 0.65
                Parent: [1×1 Group]
           PeerVisible: 'on'
         PickableParts: 'visible'
              Selected: 'off'
    SelectionHighlight: 'on'
               Visible: 'on'
       VisibleListener: [1×1 event.proplistener]

Each LegendEntry contains a back-reference to the original graphics object. In my example above, hLegend.EntryContainer.NodeChildren(2).Object == hLine1, and hLegend.EntryContainer.NodeChildren(2).Object == hLine1. Note how the default legend entries order is the reverse of the order of creation of the original graphics objects. Naturally, we can modify this order by creating the legend py passing it an array of handles that is ordered differently (see the documentation of the legend function).

To get all the original graphic objects together, in a single array, we could use one of two mechanisms (note the different order of the returned objects):

% Alternative #1
>> [hLegend.EntryContainer.NodeChildren.Object]'
ans = 
  2×1 Line array:
 
  Line    (data2)
  Line    (data1)
 
% Alternative #2
>> hLegend.PlotChildren
ans = 
  2×1 Line array:
 
  Line    (data1)
  Line    (data2)

For some reason, accessing the displayed graphic line in LegendEntry‘s Icon is not simple. For example, the LineStrip object that corresponds to hLine2 can be gotten via:

hLegendEntry = hLegend.EntryContainer.NodeChildren(1);
hLegendIconLine = hLegendEntry.Icon.Transform.Children.Children;  % a LineStrip object in our example

I assume that this was done to enable non-standard icons for patches and other complex objects (in which case the displayed icon would not necessarily be a LineStrip object). In the case of a line with markers, for example, hLegendIconLine would be an array of 2 objects: a LineStrip object and a separate Marker object. Still, I think that a direct reference in a hLegend.EntryContainer.NodeChildren(1).Icon property would have helped in 99% of all cases, so that we wouldn’t need to pass through the Transform object.

Anyway, once we have this object reference(s), we can modify its/their properties. In the case of a LineStrip this includes LineStyle, LineWidth, ColorData (4×1 uint8), and VertexData (which controls position/length):

>> get(hLegendIconLine(end))  % LineStrip
          AlignVertexCenters: 'on'
             AmbientStrength: 0.3
                ColorBinding: 'object'
                   ColorData: [4×1 uint8]
                   ColorType: 'truecolor'
             DiffuseStrength: 0.6
            HandleVisibility: 'on'
                     HitTest: 'off'
                       Layer: 'middle'
                     LineCap: 'none'
                    LineJoin: 'round'
                   LineStyle: 'solid'
                   LineWidth: 0.5
               NormalBinding: 'none'
                  NormalData: []
                      Parent: [1×1 Group]
               PickableParts: 'visible'
    SpecularColorReflectance: 1
            SpecularExponent: 10
            SpecularStrength: 0.9
                   StripData: []
                     Texture: [0×0 GraphicsPlaceholder]
                  VertexData: [3×2 single]
               VertexIndices: []
                     Visible: 'on'
       WideLineRenderingHint: 'software'

and in the presense of markers:

>> get(hLegendIconLine(1))  % Marker
    EdgeColorBinding: 'object'
       EdgeColorData: [4×1 uint8]
       EdgeColorType: 'truecolor'
    FaceColorBinding: 'object'
       FaceColorData: []
       FaceColorType: 'truecolor'
    HandleVisibility: 'on'
             HitTest: 'off'
               Layer: 'middle'
           LineWidth: 0.5
              Parent: [1×1 Group]
       PickableParts: 'visible'
                Size: 6
         SizeBinding: 'object'
               Style: 'circle'
          VertexData: [3×1 single]
       VertexIndices: []
             Visible: 'on'

An additional undocumented legend property that is of interest is ItemTokenSize. This is a 2-element numeric array specifying the minimal size of the legend entries’ icon and label. By default hLegend.ItemTokenSize == [30,18], but we can either expand or shrink the icons/labels by setting different values. For example:

hLegend.ItemTokenSize == [10,1];  % shrink legend icons and labels

Note that regardless of the amount that we specify, the actual amount that will be used will be such that all legend labels appear.
Fun: try playing with negative values for the icon and the label and see what happens :-)

Have you come across any other interesting undocumented aspect of Matlab legends? If so, then please share it in a comment below.

]]>
https://undocumentedmatlab.com/blog/plot-legend-customization/feed 3
Sliders in Matlab GUI – part 2https://undocumentedmatlab.com/blog/sliders-in-matlab-gui-part-2 https://undocumentedmatlab.com/blog/sliders-in-matlab-gui-part-2#comments Thu, 05 Jul 2018 11:40:56 +0000 https://undocumentedmatlab.com/?p=7728
 
Related posts:
  1. Context-Sensitive Help Matlab has a hidden/unsupported built-in mechanism for easy implementation of context-sensitive help...
  2. Customizing uitree This article describes how to customize Matlab GUI tree controls created using the undocumented uitree function...
  3. Date selection components The JIDE package, pre-bundled in Matlab, contains several GUI controls for selecting dates - this article explains how they can be used...
  4. uitree This article describes the undocumented Matlab uitree function, which displays data in a GUI tree component...
 
]]>
Exactly 3 years ago I posted about various alternatives for embedding sliders in Matlab GUI. Today I will follow up on that post with a description of yet another undocumented builtin alternative – controllib.widget.Slider. A summary of the various alternatives can be seen in the following screenshot:

Slider alternatives in Matlab GUI

Slider alternatives in Matlab GUI

The controllib.widget.Slider component is a class in Matlab’s internal controllib package (last week I discussed a different utility function in this package, controllib.internal.util.hString2Char).

controllib.widget.Slider accepts 3 input arguments: containing figure handle, position in pixels, and data values. For example:

>> hSlider = controllib.widget.Slider(gcf, [10,10,100,50], 5:25)
hSlider = 
  Slider with properties:
 
        Data: [6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25]
       Index: 11
       Value: 15
    FontSize: 8
    Position: [10 10 100 50]

This creates an invisible axes at the specified figure location and displays graphic axes objects that provide the appearance of the slider. When you move the slider’s knob, or click its centerline or arrows (“Steppers”), the slider’s value changes accordingly.

You can attach a callback function to the slider as follows:

myCallback = @(h,e) disp(h.Value);  % as an example
addlistener(hSlider, 'ValueChanged', myCallback);

Note that controllib.widget.Slider is based on pure-Matlab code and fully-supported functionality. The Matlab source code is available (%matlabroot%/toolbox/shared/controllib/graphics/+controllib/+widget/Slider.m) and quite readable. So while it does not actually work with the new web-based uifigures, is should be relatively easy to adapt the code so that this component could be displayed in such uifigures.

Below is a script to recreate the screenshot above. Note the two alternative mechanisms for setting properties (Java setter-method notation, and HG set notation):

hFig = figure('Color','w');
 
% 1. controllib.widget.Slider
hSlider1 = controllib.widget.Slider(hFig, [10,10,100,50], 1:20);
hSlider1.Value = 12;
 
% 2. uicontrol
hSlider2 = uicontrol('style','slider', 'units','pixels', 'pos',[10,80,100,20], 'Min',0', 'Max',20, 'Value',12);
 
% 3. JScrollBar
jSlider3 = javaObjectEDT(javax.swing.JScrollBar);
jSlider3.setOrientation(jSlider3.HORIZONTAL);  % Java setter-method notation
set(jSlider3, 'VisibleAmount',1, 'Minimum',0, 'Maximum',20, 'Value',12);  % HG set notation
[hSlider3, hContainer3] = javacomponent(jSlider3, [10,130,100,20], hFig);
 
% 4. JSlider #1
jSlider4 = javaObjectEDT(javax.swing.JSlider(0,20,12))
jSlider4.setBackground(java.awt.Color.white);  % Java setter-method notation
set(jSlider4, 'MinorTickSpacing',1, 'MajorTickSpacing',4, 'SnapToTicks',true, 'PaintLabels',true);  % HG set notation
[hSlider4, hContainer4] = javacomponent(jSlider4, [10,180,100,30], hFig);
 
% 5. JSlider #2
jSlider5 = javaObjectEDT(javax.swing.JSlider(0,20,12))
jSlider5.setBackground(java.awt.Color.white);  % Java setter-method notation
jSlider5.setPaintTicks(true);
set(jSlider5, 'MinorTickSpacing',1, 'MajorTickSpacing',4, 'SnapToTicks',true, 'PaintLabels',true);  % HG set notation
[hSlider5, hContainer5] = javacomponent(jSlider5, [10,230,100,40], hFig);

For additional details regarding the other slider alternatives, please refer to my earlier post on this subject.

Have you ever used another interesting utility or class in Matlab’s builtin packages? If so, please tell us about it in a comment below.

]]>
https://undocumentedmatlab.com/blog/sliders-in-matlab-gui-part-2/feed 4
String/char compatibilityhttps://undocumentedmatlab.com/blog/string-char-compatibility https://undocumentedmatlab.com/blog/string-char-compatibility#comments Thu, 28 Jun 2018 12:57:59 +0000 https://undocumentedmatlab.com/?p=7710
 
Related posts:
  1. Undocumented mouse pointer functions Matlab contains several well-documented functions and properties for the mouse pointer. However, some very-useful functions have remained undocumented and unsupported. This post details their usage....
  2. Matlab layout managers: uicontainer and relatives Matlab contains a few undocumented GUI layout managers, which greatly facilitate handling GUI components in dynamically-changing figures....
  3. Tab panels – uitab and relatives This article describes several undocumented Matlab functions that support tab-panels...
  4. uitree This article describes the undocumented Matlab uitree function, which displays data in a GUI tree component...
 
]]>
In numerous functions that I wrote over the years, some input arguments were expected to be strings in the old sense, i.e. char arrays for example, 'on' or 'off'. Matlab release R2016b introduced the concept of string objects, which can be created using the string function or [starting in R2017a] double quotes ("on").

The problem is that I have numerous functions that supported the old char-based strings but not the new string objects. If someone tries to enter a string object ("on") as input to a function that expects a char-array ('on'), in many cases Matlab will error. This by itself is very unfortunate – I would have liked everything to be fully backward-compatible. But unfortunately this is not the case: MathWorks did invest effort in making the new strings backward-compatible to some degree (for example, graphic object property names/values and many internal functions that now accept either form as input). However, backward compatibility of strings is not 100% perfect.

In such cases, the only solution is to make the function accept both forms (char-arrays and string objects), for example, by type-casting all such inputs as char-arrays using the builtin char function. If we do this at the top of our function, then the rest of the function can remain unchanged. For example:

function test(stage)
   if isa(stage,'string')      stage = char(stage);   end   % from this point onward, we don't need to worry about string inputs - any such strings will become plain-ol' char-arrays
 
   switch stage
      case 'stage 1', ...
      case 'stage 2', ...
      ...
   end
end

That was simple enough. But what if our function expects complex inputs (cell-arrays, structs etc.) that may contain strings in only some of their cells/fields?

Luckily, Matlab contains an internal utility function that can help us: controllib.internal.util.hString2Char. This function, whose Matlab source-code is available (%matlabroot%/toolbox/shared/controllib/general/+controllib/+internal/+util/hString2Char.m) recursively scans the input value and converts any string object into the corresponding char-array, leaving all other data-types unchanged. For example:

>> controllib.internal.util.hString2Char({123, 'char-array', "a string"})
ans =
  1×3 cell array
    {[123]}    {'char-array'}    {'a string'}
 
>> controllib.internal.util.hString2Char(struct('a',"another string", 'b',pi))
ans = 
  struct with fields:
    a: 'another string'
    b: 3.14159265358979

In order to keep our code working not just on recent releases (that support strings and controllib.internal.util.hString2Char) but also on older Matlab releases (where they did not exist), we simply wrap the call to hString2Char within a trycatch block. The adaptation of our function might then look as follows:

function test(varargin)
   try varargin = controllib.internal.util.hString2Char(varargin); catch, end   % from this point onward, we don't need to worry about string inputs - any such strings will become plain-ol' char-arrays
   ...
end

Note that controllib.internal.util.hString2Char is a semi-documented function: it contains a readable internal help section (accessible via help controllib.internal.util.hString2Char), but not a doc-page. Nor is this function mentioned anywhere in Matlab’s official documentation. I think that this is a pity, because it’s such a useful little helper function.

]]>
https://undocumentedmatlab.com/blog/string-char-compatibility/feed 15
Blocked wait with timeout for asynchronous eventshttps://undocumentedmatlab.com/blog/blocked-wait-with-timeout-for-asynchronous-events https://undocumentedmatlab.com/blog/blocked-wait-with-timeout-for-asynchronous-events#respond Sun, 13 May 2018 20:22:08 +0000 https://undocumentedmatlab.com/?p=7620
 
Related posts:
  1. Matlab-Java memory leaks, performance Internal fields of Java objects may leak memory - this article explains how to avoid this without sacrificing performance. ...
  2. File deletion memory leaks, performance Matlab's delete function leaks memory and is also slower than the equivalent Java function. ...
  3. Converting Java vectors to Matlab arrays Converting Java vectors to Matlab arrays is pretty simple - this article explains how....
  4. The Java import directive The import function can be used to clarify Java code used in Matlab. ...
 
]]>
Readers of this website may have noticed that I have recently added an IQML section to the website’s top menu bar. IQML is a software connector that connects Matlab to DTN’s IQFeed, a financial data-feed of live and historic market data. IQFeed, like most other data-feed providers, sends its data in asynchronous messages, which need to be processed one at a time by the receiving client program (Matlab in this case). I wanted IQML to provide users with two complementary modes of operation:

IQML's IQFeed-Matlab connectivity

  • Streaming (asynchronous, non-blocking) – incoming server data is processed by internal callback functions in the background, and is made available for the user to query at any later time.
  • Blocking (synchronously waiting for data) – in this case, the main Matlab processing flows waits until the data arrives, or until the specified timeout period has passed – whichever comes first.

Implementing streaming mode is relatively simple in general – all we need to do is ensure that the underlying connector object passes the incoming server messages to the relevant Matlab function for processing, and ensure that the user has some external way to access this processed data in Matlab memory (in practice making the connector object pass incoming data messages as Matlab callback events may be non-trivial, but that’s a separate matter – read here for details).

In today’s article I’ll explain how we can implement a blocking mode in Matlab. It may sound difficult but it turns out to be relatively simple.

I had several requirements/criteria for my blocked-wait implementation:

  1. Compatibility – It had to work on all Matlab platforms, and all Matlab releases in the past decade (which rules out using Microsoft Dot-NET objects)
  2. Ease-of-use – It had to work out-of-the-box, with no additional installation/configuration (which ruled out using Perl/Python objects), and had to use a simple interface function
  3. Timeout – It had to implement a timed-wait, and had to be able to tell whether the program proceeded due to a timeout, or because the expected event has arrived
  4. Performance – It had to have minimal performance overhead

The basic idea

The basic idea is to use Matlab’s builtin waitfor, as I explained in another post back in 2012. If our underlying connector object has some settable boolean property (e.g., Done) then we can set this property in our event callback, and then waitfor(object,'Done'). The timeout mechanism is implemented using a dedicated timer object, as follows:

% Wait for data updates to complete (isDone = false if timeout, true if event has arrived)
function isDone = waitForDone(object, timeout)
    % Initialize: timeout flag = false
    object.setDone(false);
 
    % Create and start the separate timeout timer thread
    hTimer = timer('TimerFcn',@(h,e)object.setDone(true), 'StartDelay',timeout);
    start(hTimer);
 
    % Wait for the object property to change or for timeout, whichever comes first
    waitfor(object,'Done');
 
    % waitfor is over - either because of timeout or because the data changed
    % To determine which, check whether the timer callback was activated
    isDone = isvalid(hTimer) && hTimer.TasksExecuted == 0;
 
    % Delete the timer object
    try stop(hTimer);   catch, end
    try delete(hTimer); catch, end
 
    % Return the flag indicating whether or not timeout was reached
end  % waitForDone

where the event callback is responsible for invoking object.setDone(true) when the server data arrives. The usage would then be similar to this:

requestDataFromServer();
 
if isBlockingMode
   % Blocking mode - wait for data or timeout (whichever comes first)
   isDone = waitForDone(object, 10.0);  % wait up to 10 secs for data to arrive
   if ~isDone  % indicates a timeout
      fprintf(2, 'No server data has arrived within the specified timeout period!\n')
   end
else
   % Non-blocking (streaming) mode - continue with regular processing
end

Using a stand-alone generic signaling object

But what can we do if we don’t have such a Done property in our underlying object, or if we do not have programmatic access to it?

We could create a new non-visible figure and then waitfor one of its properties (e.g. Resize). The property would be initialized to 'off', and within both the event and timer callbacks we would set it to 'on', and then waitfor(hFigure,'Resize','on'). However, figures, even if non-visible, are quite heavy objects in terms of both memory, UI resources, and performance.

It would be preferable to use a much lighter-weight object, as long as it abides by the other criteria above. Luckily, there are numerous such objects in Java, which is bundled in every Matlab since 2000, on every Matlab platform. As long as we choose a small Java object that has existed forever, we should be fine. For example, we could use a simple javax.swing.JButton and its boolean property Enabled:

hSemaphore = handle(javax.swing.JButton);  % waitfor() expects a handle() object, not a "naked" Java reference
 
% Wait for data updates to complete (isDone = false if timeout, true if event has arrived)
function isDone = waitForDone(hSemaphore, timeout)
    % Initialize: timeout flag = false
    hSemaphore.setEnabled(false);
 
    % Create and start the separate timeout timer thread
    hTimer = timer('TimerFcn',@(h,e)hSemaphore.setEnabled(true), 'StartDelay',timeout);
    start(hTimer);
 
    % Wait for the object property to change or for timeout, whichever comes first
    waitfor(hSemaphore,'Enabled');
 
    % waitfor is over - either because of timeout or because the data changed
    % To determine which, check whether the timer callback was activated
    isDone = isvalid(hTimer) && hTimer.TasksExecuted == 0;
 
    % Delete the timer object
    try stop(hTimer);   catch, end
    try delete(hTimer); catch, end
 
    % Return the flag indicating whether or not timeout was reached
end  % waitForDone

In this implementation, we would need to pass the hSemaphore object handle to the event callback, so that it would be able to invoke hSemaphore.setEnabled(true) when the server data has arrived.

Under the hood, note that Enabled is not a true “property” of javax.swing.JButton, but rather exposes two simple public getter/setter methods (setEnabled() and isEnabled()), which Matlab interprets as a “property”. For all intents and purposes, in our Matlab code we can treat Enabled as a property of javax.swing.JButton, including its use by Matlab’s builtin waitfor function.

There is a light overhead to this: on my laptop the waitfor function returns ~20 mSecs after the invocation of hSemaphore.setEnabled(true) in the timer or event callback – in many cases this overhead is negligible compared to the networking latencies for the blocked data request. When event reactivity is of utmost importance, users can always use streaming (non-blocking) mode, and process the incoming data events immediately in a callback.

Of course, it would have been best if MathWorks would add a timeout option and return value to Matlab’s builtin waitfor function, similar to my waitForDone function – this would significantly simplify the code above. But until this happens, you can use waitForDone pretty-much as-is. I have used similar combinations of blocking and streaming modes with multiple other connectors that I implemented over the years (Interactive Brokers, CQG, Bloomberg and Reuters for example), and the bottom line is that it actually works well in practice.

Let me know if you’d like me to assist with your own Matlab project or data connector, either developing it from scratch or improving your existing code. I will be visiting Boston and New York in early June and would be happy to meet you in person to discuss your specific needs.

]]>
https://undocumentedmatlab.com/blog/blocked-wait-with-timeout-for-asynchronous-events/feed 0
Speeding-up builtin Matlab functions – part 2https://undocumentedmatlab.com/blog/speeding-up-builtin-matlab-functions-part-2 https://undocumentedmatlab.com/blog/speeding-up-builtin-matlab-functions-part-2#comments Sun, 06 May 2018 16:26:19 +0000 https://undocumentedmatlab.com/?p=7533
 
Related posts:
  1. Speeding-up builtin Matlab functions – part 1 Built-in Matlab functions can often be profiled and optimized for improved run-time performance. This article shows a typical example. ...
  2. Solving a MATLAB bug by subclassing Matlab's Image Processing Toolbox's impoint function contains an annoying bug that can be fixed using some undocumented properties....
  3. Docs of old Matlab releases MathWorks recently posted archived documentation for many previous Matlab releases...
  4. Array resizing performance Several alternatives are explored for dynamic array growth performance in Matlab loops. ...
 
]]>
Last week I showed how we can speed-up built-in Matlab functions, by creating local copies of the relevant m-files and then optimizing them for improved speed using a variety of techniques.Accelerating MATLAB Performance Today I will show another example of such speed-up, this time of the Financial Toolbox’s maxdrawdown function, which is widely used to estimate the relative risk of a trading strategy or asset. One might think that such a basic indicator would be optimized for speed, but experience shows otherwise. In fact, this function turned out to be the main run-time performance hotspot for one of my clients. The vast majority of his code was optimized for speed, and he naturally assumed that the built-in Matlab functions were optimized as well, but this was not the case. Fortunately, I was able to create an equivalent version that was 30-40 times faster (!), and my client remains a loyal Matlab fan.

In today’s post I will show how I achieved this speed-up, using different methods than the ones I showed last week. A combination of these techniques can be used in a wide range of other Matlab functions. Additional speed-up techniques can be found in other performance-related posts on this website, as well in my book Accelerating MATLAB Performance.

Profiling

As I explained last week, the first step in speeding up any function is to profile its current run-time behavior using Matlab’s builtin Profiler tool, which can either be started from the Matlab Editor toolstrip (“Run and Time”) or via the profile function.

The profile report for the client’s function showed that it had two separate performance hotspots:

  1. Code that checks the drawdown format (optional 2nd input argument) against a set of allowed formats
  2. Main code section that iteratively loops over the data-series values to compute the maximal drawdown

In order top optimize the code, I copied %matlabroot%/toolbox/finance/finance/maxdrawdown.m to a local folder on the Matlab path, renaming the file (and the function) maxdrawdn, in order to avoid conflict with the built-in version.

Optimizing input args pre-processing

The main problem with the pre-processing of the optional format input argument is the string comparisons, which are being done even when the default format is used (which is by far the most common case). String comparison are often more expensive than numerical computations. Each comparison by itself is very short, but when maxdrawdown is run in a loop (as it often is), the run-time adds up.

Here’s a snippet of the original code:

if nargin < 2 || isempty(Format)
    Format = 'return';
end
if ~ischar(Format) || size(Format,1) ~= 1
    error(message('finance:maxdrawdown:InvalidFormatType'));
end
choice = find(strncmpi(Format,{'return','arithmetic','geometric'},length(Format)));
if isempty(choice)
    error(message('finance:maxdrawdown:InvalidFormatValue'));
end

An equivalent code, which avoids any string processing in the common default case, is faster, simpler and more readable:

if nargin < 2 || isempty(Format)
    choice = 1;
elseif ~ischar(Format) || size(Format,1) ~= 1
    error(message('finance:maxdrawdown:InvalidFormatType'));
else
    choice = find(strncmpi(Format,{'return','arithmetic','geometric'},length(Format)));
    if isempty(choice)
        error(message('finance:maxdrawdown:InvalidFormatValue'));
    end
end

The general rule is that whenever you have a common case, you should check it first, avoiding unnecessary processing downstream. Moreover, for improved run-time performance (although not necessarily maintainability), it is generally preferable to work with numbers rather than strings (choice rather than Format, in our case).

Vectorizing the main loop

The main processing loop uses a very simple yet inefficient iterative loop. I assume that the code was originally developed this way in order to assist debugging and to ensure correctness, and that once it was ready nobody took the time to also optimize it for speed. It looks something like this:

MaxDD = zeros(1,N);
MaxDDIndex = ones(2,N);
...
if choice == 1   % 'return' format
    MaxData = Data(1,:);
    MaxIndex = ones(1,N);
    for i = 1:T
        MaxData = max(MaxData, Data(i,:));
        q = MaxData == Data(i,:);
        MaxIndex(1,q) = i;
        DD = (MaxData - Data(i,:)) ./ MaxData;
        if any(DD > MaxDD)
            p = DD > MaxDD;
            MaxDD(p) = DD(p);
            MaxDDIndex(1,p) = MaxIndex(p);
            MaxDDIndex(2,p) = i;
        end
    end
else             % 'arithmetic' or 'geometric' formats
    ...

This loop can relatively easily be vectorized, making the code much faster, and arguably also simpler, more readable, and more maintainable:

if choice == 3
    Data = log(Data);
end
MaxDDIndex = ones(2,N);
MaxData = cummax(Data);
MaxIndexes = find(MaxData==Data);
DD = MaxData - Data;
if choice == 1	% 'return' format
    DD = DD ./ MaxData;
end
MaxDD = cummax(DD);
MaxIndex2 = find(MaxDD==DD,1,'last');
MaxIndex1 = MaxIndexes(find(MaxIndexes<=MaxIndex2,1,'last'));
MaxDDIndex(1,:) = MaxIndex1;
MaxDDIndex(2,:) = MaxIndex2;
MaxDD = MaxDD(end,:);

Let’s make a short run-time and accuracy check – we can see that we achieved a 31-fold speedup (YMMV), and received the exact same results:

>> data = rand(1,1e7);
 
>> tic, [MaxDD1, MaxDDIndex1] = maxdrawdown(data); toc  % builtin Matlab function
Elapsed time is 7.847140 seconds.
 
>> tic, [MaxDD2, MaxDDIndex2] = maxdrawdn(data); toc  % our optimized version
Elapsed time is 0.253130 seconds.
 
>> speedup = round(7.847140 / 0.253130)
speedup =
    31
 
>> isequal(MaxDD1,MaxDD2) && isequal(MaxDDIndex1,MaxDDIndex2)  % check accuracy
ans =
  logical
   1

Disclaimer: The code above seems to work (quite well in fact) for a 1D data vector. You’d need to modify it a bit to handle 2D data – the returned maximal drawdown are still computed correctly but not the returned indices, due to their computation using the find function. This modification is left as an exercise for the reader…

Very similar code could be used for the corresponding maxdrawup function. Although this function is used much less often than maxdrawdown, it is in fact widely used and very similar to maxdrawdown, so it is surprising that it is missing in the Financial Toolbox. Here is the corresponding code snippet:

% Code snippet for maxdrawup (similar to maxdrawdn)
MaxDUIndex = ones(2,N);
MinData = cummin(Data);
MinIndexes = find(MinData==Data);
DU = Data - MinData;
if choice == 1	% 'return' format
    DU = DU ./ MinData;
end
MaxDU = cummax(DU);
MaxIndex = find(MaxDD==DD,1,'last');
MinIndex = MinIndexes(find(MinIndexes<=MaxIndex,1,'last'));
MaxDUIndex(1,:) = MinIndex;
MaxDUIndex(2,:) = MaxIndex;
MaxDU = MaxDU(end,:);

Similar vectorization could be applied to the emaxdrawdown function. This too is left as an exercise for the reader…

Conclusions

Matlab is composed of thousands of internal functions. Each and every one of these functions was meticulously developed and tested by engineers, who are after all only human. Whereas supreme emphasis is always placed with Matlab functions on their accuracy, run-time performance sometimes takes a back-seat. Make no mistake about this: code accuracy is almost always more important than speed (an exception are cases where some accuracy may be sacrificed for improved run-time performance). So I’m not complaining about the current state of affairs.

But when we run into a specific run-time problem in our Matlab program, we should not despair if we see that built-in functions cause slowdown. We can try to avoid calling those functions (for example, by reducing the number of invocations, or limiting the target accuracy, etc.), or optimize these functions in our own local copy, as I’ve shown last week and today. There are multiple techniques that we could employ to improve the run time. Just use the profiler and keep an open mind about alternative speed-up mechanisms, and you’d be half-way there.

Let me know if you’d like me to assist with your Matlab project, either developing it from scratch or improving your existing code, or just training you in how to improve your Matlab code’s run-time/robustness/usability/appearance. I will be visiting Boston and New York in early June and would be happy to meet you in person to discuss your specific needs.

]]>
https://undocumentedmatlab.com/blog/speeding-up-builtin-matlab-functions-part-2/feed 8
Speeding-up builtin Matlab functions – part 1https://undocumentedmatlab.com/blog/speeding-up-builtin-matlab-functions-part-1 https://undocumentedmatlab.com/blog/speeding-up-builtin-matlab-functions-part-1#comments Sun, 29 Apr 2018 09:46:29 +0000 https://undocumentedmatlab.com/?p=7509
 
Related posts:
  1. Speeding-up builtin Matlab functions – part 2 Built-in Matlab functions can often be profiled and optimized for improved run-time performance. This article shows a typical example. ...
  2. Solving a MATLAB bug by subclassing Matlab's Image Processing Toolbox's impoint function contains an annoying bug that can be fixed using some undocumented properties....
  3. Matrix processing performance Matrix operations performance is affected by internal subscriptions in a counter-intuitive way....
  4. Improving save performance There are many different ways of improving Matlab's standard save function performance. ...
 
]]>
A client recently asked me to assist with his numeric analysis function – it took 45 minutes to run, which was unacceptable (5000 runs of ~0.55 secs each).Accelerating MATLAB Performance The code had to run in 10 minutes or less to be useful. It turns out that 99% of the run-time was taken up by Matlab’s built-in fitdist function (part of the Statistics Toolbox), which my client was certain is already optimized for maximal performance. He therefore assumed that to get the necessary speedup he must either switch to another programming language (C/Java/Python), and/or upgrade his computer hardware at considerable expense, since parallelization was not feasible in his specific case.

Luckily, I was there to assist and was able to quickly speed-up the code down to 7 minutes, well below the required run-time. In today’s post I will show how I did this, which is relevant for a wide variety of other similar performance issues with Matlab. Many additional speed-up techniques can be found in other performance-related posts on this website, as well in my book Accelerating MATLAB Performance.

Profiling

The first step in speeding up any function is to profile its current run-time behavior using Matlab’s builtin Profiler tool, which can either be started from the Matlab Editor toolstrip (“Run and Time”) or via the profile function.

The profile report for the client’s function showed that 99% of the time was spent in the Statistics Toolbox’s fitdist function. Drilling into this function in the profiling report, we see onion-like functions that processed input parameters, ensured data validation etc. The core processing is done inside a class that is unique to each required distribution (e.g., prob.StableDistribution, prob.BetaDistribution etc.) that is invoked within fitdist using an feval call, based on the distribution name that was specified by the user.

In our specific case, the external onion layers of sanity checks were unnecessary and could be avoided. In general, I advise not to discard such checks, because you never know whether future uses might have a problem with outlier data or input parameters. Moreover, in the specific case of fitdist they take only a very minor portion of the run-time (this may be different in other cases, such as the ismember function that I described years ago, where the sanity checks have a significant run-time impact compared to the core processing in the internal ismembc function).

However, since we wanted to significantly improve the total run-time and this was spent within the distribution class (prob.StableDistribution in the case of my client), we continue to drill-down into this class to determine what can be done.

It turns out that prob.StableDistribution basically does 3 things in its main fit() method:

  1. pre-process the input data (prob.ToolboxFittableParametricDistribution.processFitArgs() and .removeCensoring() methods) – this turned out to be unnecessary in my client’s data, but has little run-time impact.
  2. call stablefit() in order to get fitting parameters – this took about half the run-time
  3. call stablelike() in order to get likelihood data – this took about half the run-time as well
  4. call prob.StableDistribution.makeFitted() to create a probability-distribution object returned to the caller – this also took little run-time that was not worth bothering about.

The speed-up improvement process

With user-created code we could simply modify our code in-place. However, a more careful process is necessary when modifying built-in Matlab functions (either in the core Matlab distribution or in one of the toolboxes).

The basic idea here is to create a side-function that would replicate the core processing of fitdist. This is preferable to modifying Matlab’s installation files because we could then reuse the new function in multiple computers, without having to mess around in the Matlab installation (which may not even be possible if we do not have admin privileges). Also, if we ever upgrade our Matlab we won’t need to remember to update the installed files (and obviously retest).

I called the new function fitdist2 and inside it I initially placed only the very core functionality of prob.StableDistribution.fit():

% fitdist2 - fast replacement for fitdist(data,'stable')
% equivalent to pd = prob.StableDistribution.fit(data);
function pd = fitdist2(data)
    % Bypass pre-processing (not very important)
    [cens,freq,opt] = deal([]);
    %[data,cens,freq,opt] = prob.ToolboxFittableParametricDistribution.processFitArgs(data);
    %data = prob.ToolboxFittableParametricDistribution.removeCensoring(data,cens,freq,'stable');
 
    % Main processing in stablefit(), stablelike()
    params = stablefit(data,0.05,opt);
    [nll,cov] = stablelike(params,data);
 
    % Combine results into the returned probability distribution object
    pd = prob.StableDistribution.makeFitted(params,nll,cov,data,cens,freq);
end

If we try to run this as-is, we’d see errors because stablefit() and stablelike() are both sub-functions within %matlabroot%/toolbox/stats/stats/+prob/StableDistribution.m. So we copy these sub-functions (and their dependent subfunctions infoMtxCal(), intMle(), tabpdf(), neglog_pdf(), stable_nloglf(), varTrans) to the bottom of our fitdist2.m file, about 400 lines in total.

We also remove places that call checkargs(…) since that seems to be unnecessary – if you want to keep it then add the checkargs() function as well.

Now we re-run our code, after each speedup iteration verifying that the returned pd object returned by our fitdist2 is equivalent to the original object returned by fitdist.

Speeding-up stablefit()

A new profiling run shows that the vast majority of the time in stablefit() is spent in two lines:

  1. s = load('private/StablePdfTable.mat');
  2. [parmhat,~,err,output] = fminsearch(@(params)stable_nloglf(x,params),phi0,options);

The first of these lines is reloading a static data file. The very same static data file is later reloaded in stablelike(). Both of these data-loads is done in every single invocation of fitdist, so if we have 5000 data fits, we load the same static data file 10,000 times! This is certainly not indicative of good programming practice. It would be much faster to reload the static data once into memory, and then use this cached data for the next 9,999 invocation. Since the original authors of StableDistribution.m seem to like single-character global variables (another bad programming practice, for multiple reasons), we’ll follow their example (added lines are highlighted):

persistent s  % this should have a more meaningful name (but at least is not global...)!if isempty(s)    fit_path = fileparts(which('fitdist'));    s = load([fit_path '/private/StablePdfTable.mat']);
    a = s.a;
    b = s.b;
    xgd = s.xgd;
    p = s.p;
end

In order to speed-up the second line (that calls fminsearch), we can reduce the tolerances used by this function, by updating the options structure passed to it, so that we use tolerances of 1e-3 rather than the default 1e-6 (in our specific case this resulted in acceptable errors of ~0.1%). Specifically, we modify the code from this:

function [parmhat,parmci] = stablefit(x,alpha,options)
...
if nargin < 3 || isempty(options)
    options = statset('stablefit');
else
    options = statset(statset('stablefit'),options);
end
 
% Maximize the log-likelihood with respect to the transformed parameters
[parmhat,~,err,output] = fminsearch(@(params)stable_nloglf(x,params),phi0,options);
...
end

to this (note that the line that actually calls fminsearch remains unchanged):

function [parmhat,parmci] = stablefit(x,alpha,unused_options)...
persistent optionsif isempty(options)    options = statset('stablefit');
    options.TolX   = 1e-3;    options.TolFun = 1e-3;    options.TolBnd = 1e-3;end
 
% Maximize the log-likelihood with respect to the transformed parameters
[parmhat,~,err,output] = fminsearch(@(params)stable_nloglf(x,params),phi0,options);
...
end

The fminsearch internally calls tabpdf() repeatedly. Drilling down in the profiling report we see that it recomputes a griddedInterpolant object that is essentially the same for all iterations (and therefore a prime candidate for caching), and also that it uses the costly cubic interpolation rather than a more straight-forward linear interpolation:

function y = tabpdf(x,alpha,beta)
...
persistent G  % this should have a more meaningful name (but at least is not global...)!if isempty(G)    G = griddedInterpolant({b, a, xgd}, p, 'linear','none');  % 'linear' instead of 'cubic'end%G = griddedInterpolant({b, a, xgd}, p, 'cubic','none');  % original
y = G({beta,alpha,x});
...

These cases illustrate two important speedup technique categories: caching data in order to reduce the number of times that a certain code hotspot is being run, and modifying the parameters/inputs in order to reduce the individual run-time of the hotspot code. Variations of these techniques form the essence of effective speedup and can often be achieved by just reflecting on the problem and asking yourself two questions:

  1. can I reduce the number of times that this code is being run? and
  2. can I reduce the run-time of this specific code section?

Additional important speed-up categories include parallelization, vectorization and algorithmic modification. These are sometimes more difficult programmatically and riskier in terms of functional equivalence, but may be required in case the two main techniques above are not feasible. Of course, we can always combine these techniques, we don’t need to choose just one or the other.

Speeding-up stablelike()

We now turn our attentions to stablelike(). As for the loaded static file, we could simply use the cached s to load the data in order to avoid repeated reloading of the data from disk. But it turns out that this data is actually not used at all inside the function (!) so we just comment-out the old code:

%s = load('private/StablePdfTable.mat');
%a = s.a;
%b = s.b;
%xgd = s.xgd;
%p = s.p;

Think about this – the builtin Matlab code loads a data file from disk and then does absolutely nothing with it – what a waste!

Another important change is to reduce the run-time of the integral function, which is called thousands of times within a double loop. We do this by reducing the tolerances specified in the integral call from 1e-6:

F(i,j) = integral(@(x)infoMtxCal(x,params,step,i,j),-Inf,Inf,'AbsTol',1e-6,'RelTol',1e-4); % original
F(i,j) = integral(@(x)infoMtxCal(x,params,step,i,j),-Inf,Inf,'AbsTol',1e-3,'RelTol',1e-3); % new

You can see that once again these two cases follow the two techniques that I mentioned above: we reduced the number of times that we load the data file (to 0 in our case), and we improved the run-time of the individual integral calculation by reducing its tolerances.

Conclusions

The final result of applying the techniques above was a 6-fold speedup, reducing the total run-time from 45 minutes down to 7 minutes. I could probably have improved the run-time even further, but since we reached our target run-time I stopped there. The point after all was to make the code usable, not to reach a world speed record.

In my next article I will present another example of dramatically improving the run-time speed of a built-in Matlab function, this time a very commonly-used function in the Financial Toolbox that I was able to speed-up by a factor of 40.

Matlab releases improve continuously, so hopefully my techniques above (or alternatives) would find their way into the builtin Matlab functions, making them faster than today, out-of-the-box.

Until this happens, we should not lose hope when faced with a slow Matlab function, even if it is a built-in/internal one, as I hope to have clearly illustrated today, and will also show in my next article. Improving the performance is often easy. In fact, it took me much longer to write this article than it was to improve my client’s code…

Let me know if you’d like me to assist with your Matlab project, either developing it from scratch or improving your existing code, or just training you in how to improve your Matlab code’s run-time/robustness/usability/appearance. I will be visiting Boston and New York in early June and would be happy to meet you in person to discuss your specific needs.

]]>
https://undocumentedmatlab.com/blog/speeding-up-builtin-matlab-functions-part-1/feed 6
Spicing up the Matlab Editorhttps://undocumentedmatlab.com/blog/spicing-up-the-matlab-editor https://undocumentedmatlab.com/blog/spicing-up-the-matlab-editor#respond Thu, 29 Mar 2018 21:34:20 +0000 https://undocumentedmatlab.com/?p=7472
 
Related posts:
  1. Non-textual editor actions The UIINSPECT utility can be used to expand EditorMacro capabilities to non-text-insertion actions. This is how:...
  2. R2009b keyboard bindings The new Matlab release R2009b includes the ability to customize keyboard bindings for the editor and Command Window. However, there are still some uses for the EditorMacro utility and its...
  3. Accessing the Matlab Editor The Matlab Editor can be accessed programmatically, for a wide variety of possible uses - this article shows how....
  4. Customizing help popup contents The built-in HelpPopup, available since Matlab R2007b, has a back-door that enables displaying arbitrary text, HTML and URL web-pages....
 
]]>
I’d like to introduce guest blogger Andreas Justin, who will discuss some way-cool features in his Editor Plugin utility. Many of his feature implementations are not Editor-specific and can be reused in other Matlab-Desktop applications, for example dockable panels, and integration with Matlab’s main Preferences window.

Note: I will be traveling to the USA in June, and to Spain in August. If you would like me to visit your location for onsite consulting or training, then please let me know.

Happy Easter/Passover!

Overview

Editor-plugin's cool logo

Editor-plugin's cool logo

Compared to other IDE like IntellIJ, Eclipse and many more, Matlab’s editor seems somewhat outdated. Especially writing Object-Oriented code in Matlab is kind of a hassle. To make Matlab more user friendly, I’ve written a Java app that adds important features to the editor – Features such as navigating inside Class-code and in Inherited members; Searching through methods and instantly jumping to desired location; Reopening an editor that was closed by accident; Storing bookmarks between Matlab sessions; and Live Templates using commands directly written in the editor, replaced by pre-defined text.

The default Keyboard shortcuts listed below for the features can be customized. Most variables can be customized as well (I will point out which variables are not [yet] customizable).

Most GUIs have a search field. Within this search field you can move the list or the tree up and down using arrow keys, or hit <escape> to return to editor. These search fields allow you to enter regular expressions to limit results shown in list or tree. Also, most GUIs are dockable.

The Editor-Plugin utility is open-source. It is available on GitHub and also mirrored on the Matlab File Exchange. A detailed setup guide is provided on the utility’s wiki section in GitHub.

If you discover any problem or have any suggestion for improvement, please visit the utility’s Issues section on GitHub, where all open/closed issues can be tracked and discussed.

A brief overview of some of the features is presented below. For a detailed explanation of these and other features (which are not listed below), please review the Features section of the utility’s wiki (you guessed it: on GitHub…).

Editing

  • Delete / duplicate linesCTRL + SHIFT + Y or D allows you to delete or duplicate current line.
    Deleting/duplicating complete lines
  • Move lines up or downCTRL + ALT + UP or DOWN allows you to move selected lines up or down.
    Moving complete lines
  • Live (auto-replace) templates – Live Templates are editor commands you can design to insert predefined code. Here’s an example for the command %this% (delivered within the package). When you type a command into the editor the string will get replaced by the predefined text. This predefined text may include variables depending on what you want to achieve. %this% was designed to insert the fully qualified name of the current class you’re in (or function, or script).
    Auto-replace template
  • Clipboard StackCTRL + SHIFT + V opens the Clipboard Stack, where the last 10 copied/cut text are stored and can be directly inserted into the current editor position. The Clipboard Stack only stores text copied from editor.

  • Auto switch current folder and detail viewer – Switching editor tabs will update the Desktop’s current folder and detail viewer. This behavior can be directly changed in the detail viewer bar.
    Use the new icons in the Desktop’s Current Folder panel: Toggle detail viewer to toggle detail viewer and Toggle following the current editor file to toggle following the current editor file.
    Follow current editor file
  • File Structure – I personally use this one the most: CTRL + F12 will show a GUI that let you search methods and properties including inherited ones.
    File structure analysis
  • Navigation History – Every editor file that is opened is stored in the navigation history. Up to 50 editor file paths are stored. If you have a 5-Button Mouse, you can navigate through previous location (backward and forward).
  • Recently ClosedCTRL + SHIFT + T will show a GUI that allows you to reopen closed editors this or last session.
  • BookmarksCTRL + SHIFT + F2 will show a GUI that allows you to name, search and delete bookmarks. Matlab has this nice feature to delete bookmarks after closing the editor. This feature will store all bookmarks. If an editor or Matlab is closed and opened later. All Bookmarks will be restored.
    This feature has some issues though. The most obvious first: if the source file has been changed outside of Matlab, the bookmark does not get updated, and may point to the wrong line, or gets deleted. Also on some Systems the default shortcut does not work. But there are workarounds.

Other useful features

  • Dockable Windows – As mentioned before, most GUIs are dockable.
    Dockable Windows
  • Preferences – Preferences panel integrated in Matlab’s main Preferences window.
    Preferences panel integrated in Matlab's main Preferences window
  • Execute current lineSHIFT + F9 allows you to execute the current line. This is equivalent to selecting the entire line, then clicking F9, and is similar in concept to the editor’s built-in ability to execute the current cell-block.
    Execute current line
  • VarDiff – Select two variables in the workspace and compare them using Matlab’s internal comparison feature (adding a custom hook to the Workspace context-menu was discussed by Yair back in 2010).
    Variables comparison integrated in the Workspace browser
  • KeyPressListener – Allows you to create custom keyboard shortcuts for your own functions. A similar functionality was discussed by Yair back in 2009.

As noted above, a detailed explanation of these and other features is provided in the Features section of the utility’s wiki. If you discover any problem or have any suggestion for improvement, please visit the utility’s Issues section on GitHub, where issues can be tracked and discussed.

]]>
https://undocumentedmatlab.com/blog/spicing-up-the-matlab-editor/feed 0
Auto-scale image colorshttps://undocumentedmatlab.com/blog/auto-scale-image-colors https://undocumentedmatlab.com/blog/auto-scale-image-colors#comments Wed, 21 Feb 2018 18:06:23 +0000 https://undocumentedmatlab.com/?p=7334
 
Related posts:
  1. Introduction to UDD UDD classes underlie many of Matlab's handle-graphics objects and functionality. This article introduces these classes....
  2. Multi-column (grid) legend This article explains how to use undocumented axes listeners for implementing multi-column plot legends...
  3. UDD Events and Listeners UDD event listeners can be used to listen to property value changes and other important events of Matlab objects...
  4. Customizing axes part 3 – Backdrop Matlab HG2 axes can be customized in many different ways. This article explains some of the undocumented aspects. ...
 
]]>
I deal extensively in image processing in one of my consulting projects. The images are such that most of the interesting features are found in the central portion of the image. However, the margins of the image contain z-values that, while not interesting from an operational point-of-view, cause the displayed image’s color-limits (axes CLim property) to go wild. An image is worth a thousand words, so check the following raw image (courtesy of Flightware, Inc.), displayed by the following simple script:

hImage = imagesc(imageData); colormap(gray); colorbar;

Raw image with default Matlab CLim

Raw image with default Matlab CLim

Rescaling the axes color-limits

As you can see, this image is pretty useless for human-eye analysis. The reason is that while all of the interesting features in the central portion of the image have a z-value of ~-6, the few pixels in the margins that have a z-value of 350+ screw up the color limits and ruin the perceptual resolution (image contrast). We could of course start to guess (or histogram the z-values) to get the interesting color-limit range, and then manually set hAxes.CLim to get a much more usable image:

hAxes = hImage.Parent; hAxes.CLim = [-7.5,-6];

Raw image with a custom CLim

Raw image with a custom CLim

Auto-scaling the axes color-limits

Since the z-values range and distribution changes between different images, it would be better to automatically scale the axes color-limits based on an analysis of the image. A very simple technique for doing this is to take the 5%,95% or 10%,90% percentiles of the data, clamping all outlier data pixels to the extreme colors. If you have the Stats Toolbox you can use the prctile function for this, but if not (or even if you do), here’s a very fast alternative that automatically scales the axes color limits based on the specified threshold (a fraction between 0-0.49):

% Rescale axes CLim based on displayed image portion's CData
function rescaleAxesClim(hImage, threshold)
    % Get the displayed image portion's CData
    CData = hImage.CData;
    hAxes = hImage.Parent;
    XLim = fix(hAxes.XLim);
    YLim = fix(hAxes.YLim);
    rows = min(max(min(YLim):max(YLim),1),size(CData,1)); % visible portion
    cols = min(max(min(XLim):max(XLim),1),size(CData,2)); % visible portion
    CData = CData(unique(rows),unique(cols));
    CData = CData(:);  % it's easier to work with a 1d array
 
    % Find the CLims from this displayed portion's CData
    CData = sort(CData(~isnan(CData)));  % or use the Stat Toolbox's prctile()
    thresholdVals = [threshold, 1-threshold];
    thresholdIdxs = fix(numel(CData) .* thresholdVals);
    CLim = CData(thresholdIdxs);
 
    % Update the axes
    hAxes.CLim = CLim;
end

Note that a threshold of 0 uses the full color range, resulting in no CLim rescaling at all. At the other extreme, a threshold approaching 0.5 reduces the color-range to a single value, basically reducing the image to an unusable B/W (rather than grayscale) image. Different images might require different thresholds for optimal contrast. I believe that a good starting point for the threshold is a value of 0.10, which corresponds to the 10-90% range of CData values.

Dynamic auto-scaling of axes color-limits

This is very nice for the initial image display, but if we zoom-in, or pan a sub-image around, or update the image in some way, we would need to repeat calling this rescaleAxesClim() function every time the displayed image portion changes, otherwise we might still get unusable images. For example, if we zoom into the image above, we will see that the color-limits that were useful for the full image are much less useful on the local sub-image scale. The first (left) image uses the static custom color limits [-7.5,-6] above (i.e., simply zooming-in on that image, without modifying CLim again); the second (right) image is the result of repeating the call to rescaleAxesClim(), which improves the image contrast:

Zoomed-in image with a custom static CLim

Zoomed-in image with a custom static CLim

Zoomed-in image with a re-applied custom CLim

Zoomed-in image with a re-applied custom CLim

We could in theory attach the rescaleAxesClim() function as a callback to the zoom and pan functions (that provide such callback hooks). However, we would still need to remember to manually call this function whenever we modify the image or its containing axes programmatically.

A much simpler way is to attach our rescaleAxesClim() function as a callback to the image’s undocumented MarkedClean event:

% Instrument image: add a listener callback to rescale upon any image update
addlistener(hImage, 'MarkedClean', @(h,e)rescaleAxesClim(hImage,threshold));

In order to avoid callback recursion (potentially caused by modifying the axes CLim within the callback), we need to add a bit of code to the callback that prevents recursion/reentrancy (details). Here’s one simple way to do this:

% Rescale axes CLim based on displayed image portion's CData
function rescaleAxesClim(hImage, threshold)
    % Check for callback reentrancy
    inCallback = getappdata(hImage, 'inCallback');
    if ~isempty(inCallback), return, end
    try
        setappdata(hImage, 'inCallback',1);  % prevent reentrancy
 
        % Get the displayed image portion's CData
        ...  (copied from above)
 
        % Update the axes
        hAx.CLim = CLim;
        drawnow; pause(0.001);  % finish all graphic updates before proceeding
    catch
    end
    setappdata(hImage, 'inCallback',[]);  % reenable this callback
end

The result of this dynamic automatic color-scaling can be seen below:

Zoomed-in image with dynamic CLim

Zoomed-in image with dynamic CLim

autoScaleImageCLim utility

I have created a small utility called autoScaleImageCLim, which includes all the above, and automatically sets the specified input image(s) to use auto color scaling. Feel free to download this utility from the Matlab File Exchange. Here are a few usage examples:

autoScaleImageCLim()           % auto-scale the current axes' image
autoScaleImageCLim(hImage,5)   % auto-scale image using 5%-95% CData limits
autoScaleImageCLim(hImage,.07) % auto-scale image using 7%-93% CData limits

(note that the hImage input parameter can be an array of image handles)

Hopefully one day the so-useful MarkedClean event will become a documented and fully-supported event for all HG objects in Matlab, so that we won’t need to worry that it might not be supported by some future Matlab release…

]]>
https://undocumentedmatlab.com/blog/auto-scale-image-colors/feed 1
Adding custom properties to GUI objectshttps://undocumentedmatlab.com/blog/adding-custom-properties-to-gui-objects https://undocumentedmatlab.com/blog/adding-custom-properties-to-gui-objects#comments Thu, 15 Feb 2018 12:39:35 +0000 https://undocumentedmatlab.com/?p=7324
 
Related posts:
  1. Matlab and the Event Dispatch Thread (EDT) The Java Swing Event Dispatch Thread (EDT) is very important for Matlab GUI timings. This article explains the potential pitfalls and their avoidance using undocumented Matlab functionality....
  2. Tab panels – uitab and relatives This article describes several undocumented Matlab functions that support tab-panels...
  3. uitree This article describes the undocumented Matlab uitree function, which displays data in a GUI tree component...
  4. Borderless button used for plot properties A borderless button can be used to add unobtrusive functionality to plot axes...
 
]]>
Matlab objects have numerous built-in properties (some of them publicly-accessible/documented and others not, but that’s a different story). For various purposes, it is sometimes useful to attach custom user-defined properties to such objects. While there was never a fully-documented way to do this, most users simply attached such properties as fields in the UserData property or the object’s [hidden] ApplicationData property (accessible via the documented setappdata/getappdata functions).

An undocumented way to attach actual new user-defined properties to objects such as GUI handles or Java references has historically (in HG1, up to R2014a) been to use the undocumented schema.prop function, as I explained here. As I wrote in that post, in HG2 (R2014b onward), we can use the fully-documented addprop function to add new custom properties (and methods) to such objects. What is still NOT documented, as far as I could tell, is that all of Matlab’s builtin handle graphics objects indirectly inherit the dynamicprops class, which allows this. The bottom line is that we can dynamically add custom properties in run-time to any HG object, without affecting any other object. In other words, the new properties will only be added to the handles that we specifically request, and not to any others.

All this is important, because for some unexplained reason that escapes my understanding, MathWorks chose to seal its classes, thus preventing users to extend them with sub-classes that contain the new properties. So much frustration could have been solved if MathWorks would simply remove the Sealed class meta-property from its classes. Then again, I’d have less to blog about in that case…

Anyway, why am I rehashing old news that I have already reported a few years ago?

Well, first, because my experience has been that this little tidbit is [still] fairly unknown by Matlab developers. Secondly, I happened to run into a perfect usage example a short while ago that called for this solution: a StackExchange user asked whether it is possible to tell a GUI figure’s age, in other words the elapsed time since the figure was created. The simple answer would be to use setappdata with the creation date whenever we create a figure. However, a “cleaner” approach seems to be to create new read-only properties for the figure’s CreationTime and Age:

First, create a small Matlab function as follows, that attaches the CreationTime property to a figure:

function setCreationTime(hFig,varargin)
   hProp = addprop(hFig,'CreationTime');
   hFig.CreationTime = now;
   hProp.SetAccess = 'private';  % make property read-only after setting its initial value
 
   hProp = addprop(hFig,'Age');
   hProp.GetMethod = @(h,e) etime(datevec(hFig.CreationTime), clock);  % compute on-the-fly
   hProp.SetAccess = 'private';  % make property read-only
end

Now assign this function as the default CreateFcn callback function for all new figures from now on:

set(0,'DefaultFigureCreateFcn',@setCreationTime)

That’s it – you’re done! Whenever a new figure will be created from now on, it will have two custom read-only properties: CreationTime and Age.

For example:

>> newFig = figure;
>> newFig.CreationTime
ans =
      737096.613706748
 
>> ageInDays = now - newFig.CreationTime
ageInDays = 
       0.0162507836846635
>> ageDuration = duration(ageInDays*24,0,0)
ageDuration = 
  duration
   00:23:24
>> ageString = datestr(ageInDays, 'HH:MM:SS.FFF')
ageString = 
    '00:23:24.068'
 
>> ageInSecs = newFig.Age
ageInSecs =
       1404.06771035492

Note that an alternative way to set the computed property Age would have been to set its value to be an anonymous function, but this would have necessitated invoking it with parenthesis (as in: ageInSecs = newFig.Age()). By setting the property’s GetMethod meta-property we avoid this need.

Keen readers will have noticed that the mechanism that I outlined above for the Age property/method can also be used to add custom user methods. For example, we can create a new custom property named refresh that would be read-only and have a GetMethod which is the function handle of the function that refreshes the object in some way.

Do you have any special uses for custom user-defined properties/methods in your program? or perhaps you have a use-case that might show MathWorks why sub-classing the built-in classes might improve your work? if so, then please place a comment about it below. If enough users show MathWorks why this is important, then maybe it will be fixed in some future release.

]]>
https://undocumentedmatlab.com/blog/adding-custom-properties-to-gui-objects/feed 5
IP address input controlhttps://undocumentedmatlab.com/blog/ip-address-input-control https://undocumentedmatlab.com/blog/ip-address-input-control#comments Wed, 31 Jan 2018 16:16:12 +0000 https://undocumentedmatlab.com/?p=7313
 
Related posts:
  1. FindJObj GUI – display container hierarchy The FindJObj utility can be used to present a GUI that displays a Matlab container's internal Java components, properties and callbacks....
  2. FindJObj – find a Matlab component’s underlying Java object The FindJObj utility can be used to access and display the internal components of Matlab controls and containers. This article explains its uses and inner mechanism....
  3. Uitable sorting Matlab's uitables can be sortable using simple undocumented features...
  4. Tri-state checkbox Matlab checkboxes can easily be made to support tri-state functionality....
 
]]>
A few weeks ago, a user posted a question on Matlab Answers, asking whether it is possible to implement a text input control that accepts and validates an IP address (for example, ‘192.168.1.101’). While doing this using purely documented Matlab code is indeed possible (for those of us who are masochistically inclined and/or have nothing else to do with their spare time), a very simple-to-use and polished-looking solution is to use an undocumented built-in Matlab control.

The solution is based on the fact that Matlab comes with a huge set of professional Java-based controls by JideSoft, bundled in various JAR libraries within the %matlabroot%/java/jarext/jide Matlab installation folder. For our specific purposes (an IP-address entry/display control), we are interested in the com.jidesoft.field.IPTextField control (online documentation), which is part of the JIDE Grids library (%matlabroot%/java/jarext/jide/jide-grids.jar). We can use it as follows:

jIPField = com.jidesoft.field.IPTextField('255.255.255.0');  % set default IP
[jIPField, hContainer] = javacomponent(jIPField, [10,10,120,20], hParent);  % hParent: panel/figure handle

IPTextField control in a Matlab GUI

IPTextField control in a Matlab GUI

You can modify the position/size of the text-field in the javacomponent call above, or by modifying the Position / Units properties of the returned hContainer.

We can retrieve the IP text/numeric values using:

vals = jIPField.getValue';         % 1x4 uint32 array => [255,255,255,0]
vals = cell(jIPField.getRawText)'; % 1x4 string cells => {'255','255','255','0'} 
ip   = char(jIPField.getText);     % entire IP string => '255.255.255.0'

The IPTextField component auto-validates the IP values, ensuring that the displayed IP is always valid (for example, IP components cannot be negative or larger than 255). The component has many other features, including the ability to enable/disable, color or format the IP components etc.

We can set a callback function to process user changes, by setting the component’s *StateChangedCallback* property, for example:

jIPField.StateChangedCallback = @(jComponent,jEventData) disp(jComponent.getValue');

The JIDE libraries that come with Matlab contain numerous other similarly-useful components, including date/time/font/color/file/folder selectors, calendars in various formats, credit-card fields, and many more.

For more information about using the javacomponent function and handling Java components in Matlab GUI, see other posts on this website – particularly those marked with the “JIDE” tag.

Additional discussion of JIDE’s combo-boxes, and JIDE controls in general, is available in Chapter 5 of my Matlab-Java Programming book.

If you need to integrate professional-looking controls such as these in your Matlab GUI, consider hiring my consulting services.

Caution

Remember that JIDE evolves with Matlab, and so JIDE’s online documentation, which refers to the latest JIDE version, may be partially inapplicable if you use an old Matlab version. In any case, Matlab releases always lag the latest JIDE release by at least a year (e.g., Matlab R2017b still uses JIDE v3.4.1 that was released in June 2012 – MathWorks used to update the bundled JIDE libraries to newer versions, but for some reason has stopped doing that in 2013). The older your Matlab, the more such inconsistencies that you may find. For example, I believe that DateSpinnerComboBox only became available around R2010b; similarly, some control properties behave differently (or are missing altogether) in different releases. To determine the version of JIDE that you are currently using in Matlab, run the following (the result can then be compared to JIDE’s official change-log history):

>> com.jidesoft.utils.Lm.getProductVersion
ans =
3.4.1

Note that JIDE is a commercial product. We may not use it without JIDESoft’s permission outside the Matlab environment. It is my understanding however, that we can freely use it within Matlab. Note that this is not legal advise as I am an engineer, not a lawyer. If you have any licensing questions, contact sales@jidesoft.com.

Also note that all of JIDE’s controls use the Java-based figures (that are created using GUIDE or the figure function), and will not work on the new web-based uifigures (created using App Designer or the uifigure function). There is currently no corresponding IP entry/display control for web-based GUIs, and since there is [still] no way to integrate external Javascript/CSS libraries in uifigures, the only resort is to use a plain-vanilla edit-box. If MathWorks would simply open a hook to integrate external JS/CSS libraries, that would enable users to use 3rd-party libraries that have such custom controls and MathWorks would then not need to spend a huge amount of effort to develop multiple UI control variants.

]]>
https://undocumentedmatlab.com/blog/ip-address-input-control/feed 3
Customizing axes tick labelshttps://undocumentedmatlab.com/blog/customizing-axes-tick-labels https://undocumentedmatlab.com/blog/customizing-axes-tick-labels#comments Wed, 24 Jan 2018 13:38:26 +0000 https://undocumentedmatlab.com/?p=7304
 
Related posts:
  1. HG2 update HG2 appears to be nearing release. It is now a stable mature system. ...
  2. Performance: accessing handle properties Handle object property access (get/set) performance can be significantly improved using dot-notation. ...
  3. Customizing axes rulers HG2 axes can be customized in numerous useful ways. This article explains how to customize the rulers. ...
  4. Plot legend title Titles to plot legends are easy to achieve in HG1 (R2014a or earlier), but much more difficult in HG2 (R2014b or newer). ...
 
]]>
In last week’s post, I discussed various ways to customize bar/histogram plots, including customization of the tick labels. While some of the customizations that I discussed indeed rely on undocumented properties/features, many Matlab users are not aware that tick labels can be individually customized, and that this is a fully documented/supported functionality. This relies on the fact that the default axes TickLabelInterpreter property value is 'tex', which supports a wide range of font customizations, individually for each label. This includes any combination of symbols, superscript, subscript, bold, italic, slanted, face-name, font-size and color – even intermixed within a single label. Since tex is the default interpreter, we don’t need any special preparation – simply set the relevant X/Y/ZTickLabel string to include the relevant tex markup.

To illustrate this, have a look at the following excellent answer by user Ubi on Stack Overflow:

Axes with Tex-customized tick labels

Axes with Tex-customized tick labels

plot(1:10, rand(1,10))
ax = gca;
 
% Simply color an XTickLabel
ax.XTickLabel{3} = ['\color{red}' ax.XTickLabel{3}];
 
% Use TeX symbols
ax.XTickLabel{4} = '\color{blue} \uparrow';
 
% Use multiple colors in one XTickLabel
ax.XTickLabel{5} = '\color[rgb]{0,1,0}green\color{orange}?';
 
% Color YTickLabels with colormap
nColors = numel(ax.YTickLabel);
cm = jet(nColors);
for i = 1:nColors
    ax.YTickLabel{i} = sprintf('\\color[rgb]{%f,%f,%f}%s', cm(i,:), ax.YTickLabel{i});
end

In addition to 'tex', we can also set the axes object’s TickLabelInterpreter to 'latex' for a Latex interpreter, or 'none' if we want to use no string interpretation at all.

As I showed in last week’s post, we can control the gap between the tick labels and the axle line, using the Ruler object’s undocumented TickLabelGapOffset, TickLabelGapMultiplier properties.

Also, as I explained in other posts (here and here), we can also control the display of the secondary axle label (typically exponent or units) using the Ruler’s similarly-undocumented SecondaryLabel property. Note that the related Ruler’s Exponent property is documented/supported, but simply sets a basic exponent label (e.g., '\times10^{6}' when Exponent==6) – to set a custom label string (e.g., '\it\color{gray}Millions'), or to modify its other properties (position, alignment etc.), we should use SecondaryLabel.

]]>
https://undocumentedmatlab.com/blog/customizing-axes-tick-labels/feed 5
Customizing histogram plotshttps://undocumentedmatlab.com/blog/customizing-histogram-plots https://undocumentedmatlab.com/blog/customizing-histogram-plots#respond Wed, 17 Jan 2018 20:41:15 +0000 https://undocumentedmatlab.com/?p=7292
 
Related posts:
  1. getundoc – get undocumented object properties getundoc is a very simple utility that displays the hidden (undocumented) properties of a specified handle object....
  2. Customizing axes part 3 – Backdrop Matlab HG2 axes can be customized in many different ways. This article explains some of the undocumented aspects. ...
  3. Customizing axes part 4 – additional properties Matlab HG2 axes can be customized in many different ways. This article explains some of the undocumented aspects. ...
  4. Plot line transparency and color gradient Static and interpolated (gradient) colors and transparency can be set for plot lines in HG2. ...
 
]]>
Earlier today, I was given the task of displaying a histogram plot of a list of values. In today’s post, I will walk through a few customizations that can be done to bar plots and histograms in order to achieve the desired results.

We start by binning the raw data into pre-selected bins. This can easily be done using the builtin histc (deprecated) or histcounts functions. We can then use the bar function to plot the results:

[binCounts, binEdges] = histcounts(data);
hBars = bar(hAxes, binEdges(1:end-1), binCounts);

Basic histogram bar plot

Basic histogram bar plot

Let’s improve the appearance: In my specific case, the data was financial return (percentage) values, so let’s modify the x-label format accordingly and display a title. To make the labels and title more legible, we decrease the axes FontSize to 8 and remove the axes box:

hAxes = hBar.Parent;
xtickformat(hAxes, '%g%%');
title(hAxes, 'Distribution of total returns (monthly %)');
set(hAxes, 'FontSize',8, 'Box','off')

Improved histogram bar plot

Improved histogram bar plot

So far nothing undocumented. Note that the xtickformat/ytickformat functions were only introduced in R2016b – for earlier Matlab releases see this post (which does rely on undocumented aspects).

Now, let’s use a couple of undocumented properties: to remove the excess white-space margin around the axes we’ll set the axes’ LooseInset property, and to remove the annoying white space between the tick labels and the X-axis we’ll set the XRuler‘s TickLabelGapOffset property to -2 (default: +2):

set(hAxes, 'LooseInset',[0,0,0,0]);    % default = [.13,.11,.095,.075]
hAxes.XRuler.TickLabelGapOffset = -2;  % default = +2

Even better histogram bar plot

Even better histogram bar plot

Note that I used the undocumented axes XRuler property instead of the axes’ documented XAxis property, because XAxis is only available since R2015b, whereas XRuler (which points to the exact same object as XAxis) exists ever since R2014b, and so is better from a backward-compatibility standpoint. In either case, the ruler’s TickLabelGapOffset property is undocumented. Note that the ruler also contains another associated and undocumented TickLabelGapMultiplier property (default: 0.2), which I have not modified in this case.

Now let’s take a look at the bin labels: The problem with the bar plot above is that it’s not intuitively clear whether the bin for “5%”, for example, includes data between 4.5-5.5 or between 5.0-6.0 (which is the correct answer). It would be nicer if the labels were matched to the actual bin edges. There are 3 basic ways to fix this:

  1. We could modify the bar plot axes tick values and labels, in essence “cheating” by moving the tick labels half a bin leftward of their tick values (don’t forget to add the extra tick label on the right):
    hAxes.XTick(end+1) = hAxes.XTick(end) + 1;  % extra tick label on the right
    labels = hAxes.XTickLabels;       % preserve tick labels for later use below
    hAxes.XTick = hAxes.XTick - 0.5;  % move tick labels 1/2 bin leftward
    hAxes.XTickLabel = labels;        % restore pre-saved tick labels
    hAxes.XLim = hAxes.XLim - 0.5;    % ...and align the XLim

    Improved labels

    Improved labels

  2. We could use the bar function’s optional 'histc' flag, in order to display the bars in histogram mode. The problem in histogram mode is that while the labels are now placed correctly, the bars touch each other – I personally find distinct bars that are separated by a small gap easier to understand.
    hBars = bar(..., 'histc');
    % [snip] - same customizations to hAxes as done above

    Basic histogram plot

    Basic histogram plot

    With the original bar chart we could use the built-in BarWidth to set the bar/gap width (default: 0.8 meaning a 10% gap on either side of the bar). Unfortunately, calling bar with 'hist' or 'histc' (i.e. histogram mode) results in a Patch (not Bar) object, and patches do not have a BarWidth property. However, we can modify the resulting patch vertices in order to achieve the same effect:

    % Modify the patch vertices (5 vertices per bar, row-based)
    hBars.Vertices(:,1) = hBars.Vertices(:,1) + 0.1;
    hBars.Vertices(4:5:end,1) = hBars.Vertices(4:5:end,1) - 0.2;
    hBars.Vertices(5:5:end,1) = hBars.Vertices(5:5:end,1) - 0.2;
     
    % Align the bars & labels at the center of the axes
    hAxes.XLim = hAxes.XLim + 0.5;

    This now looks the same as option #1 above, except that the top-level handle is a Patch (not Bar) object. For various additional customizations, either Patch or Bar might be preferable, so you have a choice.

    Improved histogram plot

    Improved histogram plot

  3. Lastly, we could have used the builtin histogram function instead of bar. This function also displays a plot with touching bars, as above, using Quadrilateral objects (a close relative of Patch). The solution here is very similar to option #2 above, but we need to dig a bit harder to modify the patch faces, since their vertices is not exposed as a public property of the Histogram object. To modify the vertices, we first get the private Face property (explanation), and then modify its vertices, keeping in mind that in this specific case the bars have 4 vertices per bar and a different vertices matrix orientation:
    hBars = histogram(data, 'FaceAlpha',1.0, 'EdgeColor','none');
    % [snip] - same customizations to hAxes as done above
     
    % Get access to *ALL* the object's properties
    oldWarn = warning('off','MATLAB:structOnObject');
    warning off MATLAB:hg:EraseModeIgnored
    hBarsStruct = struct(hBars);
    warning(oldWarn);
     
    % Modify the patch vertices (4 vertices per bar, column-based)
    drawnow;  % this is important, won't work without this!
    hFace = hBarsStruct.Face;  % a Quadrilateral object (matlab.graphics.primitive.world.Quadrilateral)
    hFace.VertexData(1,:) = hFace.VertexData(1,:) + 0.1;
    hFace.VertexData(1,3:4:end) = hFace.VertexData(1,3:4:end) - 0.2;
    hFace.VertexData(1,4:4:end) = hFace.VertexData(1,4:4:end) - 0.2;

In conclusion, there are many different ways to improve the appearance of charts in Matlab. Even if at first glance it may seem that some visualization function does not have the requested customization property or feature, a little digging will often find either a relevant undocumented property, or an internal object whose properties could be modified. If you need assistance with customizing your charts for improved functionality and appearance, then consider contacting me for a consulting session.

]]>
https://undocumentedmatlab.com/blog/customizing-histogram-plots/feed 0
Toolbar button labelshttps://undocumentedmatlab.com/blog/toolbar-button-labels https://undocumentedmatlab.com/blog/toolbar-button-labels#respond Mon, 08 Jan 2018 17:34:17 +0000 https://undocumentedmatlab.com/?p=7270
 
Related posts:
  1. Customizing figure toolbar background Setting the figure toolbar's background color can easily be done using just a tiny bit of Java magic powder. This article explains how. ...
  2. Builtin PopupPanel widget We can use a built-in Matlab popup-panel widget control to display lightweight popups that are attached to a figure window. ...
  3. FindJObj – find a Matlab component’s underlying Java object The FindJObj utility can be used to access and display the internal components of Matlab controls and containers. This article explains its uses and inner mechanism....
  4. FindJObj GUI – display container hierarchy The FindJObj utility can be used to present a GUI that displays a Matlab container's internal Java components, properties and callbacks....
 
]]>
I was recently asked by a client to add a few buttons labeled “1”-“4” to a GUI toolbar. I thought: How hard could that be? Simply get the toolbar’s handle from the figure, then use the builtin uipushtool function to add a new button, specifying the label in the String property, right?

Labeled toolbar buttons

Well, not so fast it seems:

hToolbar = findall(hFig, 'tag','FigureToolBar');  % get the figure's toolbar handle
uipushtool(hToolbar, 'String','1');               % add a pushbutton to the toolbar
Error using uipushtool
There is no String property on the PushTool class. 

Apparently, for some unknown reason, standard Matlab only enables us to set the icon (CData) of a toolbar control, but not a text label.

Once again, Java to the rescue:

We first get the Java toolbar reference handle, then add the button in standard Matlab (using uipushtool, uitoggletool and their kin). We can now access the Java toolbar’s last component, relying on the fact that Matlab always adds new buttons at the end of the toolbar. Note that we need to use a short drawnow to ensure that the toolbar is fully re-rendered, otherwise we’d get an invalid Java handle. Finally, once we have this reference handle to the underlying Java button component, we can set and customize its label text and appearance (font face, border, size, alignment etc.):

hToolbar = findall(hFig, 'tag','FigureToolBar');     % get the figure's toolbar handle
jToolbar = hToolbar.JavaContainer.getComponentPeer;  % get the toolbar's Java handle
for buttonIdx = 1 : 4
    % First create the toolbar button using standard Matlab code
    label = num2str(buttonIdx);  % create a string label
    uipushtool(hToolbar, 'ClickedCallback',{@myCallback,analysisIdx}, 'TooltipString',['Run analysis #' label]);
 
    % Get the Java reference handle to the newly-created button
    drawnow; pause(0.01);  % allow the GUI time to re-render the toolbar
    jButton = jToolbar.getComponent(jToolbar.getComponentCount-1);
 
    % Set the button's label
    jButton.setText(label)
end

The standard Matlab toolbar button size (23×23 pixels) is too small to display more than a few characters. To display a longer label, we need to widen the button:

% Make the button wider than the standard 23 pixels
newSize = java.awt.Dimension(50, jButton.getHeight);
jButton.setMaximumSize(newSize)
jButton.setPreferredSize(newSize)
jButton.setSize(newSize)

Using a text label does not prevent us from also displaying an icon: In addition to the text label, we can also display a standard icon (by setting the button’s CData property in standard Matlab). This icon will be displayed to the left of the text label. You can widen the button, as shown in the code snippet above, to make space for both the icon and the label. If you want to move the label to a different location relative to the icon, simply modify the Java component’s HorizontalTextPosition property:

jButton.setHorizontalTextPosition(jButton.RIGHT);   % label right of icon (=default)
jButton.setHorizontalTextPosition(jButton.CENTER);  % label on top of icon
jButton.setHorizontalTextPosition(jButton.LEFT);    % label left of icon

In summary, here’s the code snippet that generated the screenshot above:

% Get the Matlab & Java handles to the figure's toolbar
hToolbar = findall(hFig, 'tag','FigureToolBar');     % get the figure's toolbar handle
jToolbar = hToolbar.JavaContainer.getComponentPeer;  % get the toolbar's Java handle
 
% Button #1: label only, no icon, 23x23 pixels
h1 = uipushtool(hToolbar);
drawnow; pause(0.01);
jButton = jToolbar.getComponent(jToolbar.getComponentCount-1);
jButton.setText('1')
 
% Create the icon CData from an icon file
graphIcon = fullfile(matlabroot,'/toolbox/matlab/icons/plotpicker-plot.gif');
[graphImg,map] = imread(graphIcon);
map(map(:,1)+map(:,2)+map(:,3)==3) = NaN;  % Convert white pixels => transparent background
cdata = ind2rgb(graphImg,map);
 
% Button #2: label centered on top of icon, 23x23 pixels
h2 = uipushtool(hToolbar, 'CData',cdata);
drawnow; pause(0.01);
jButton = jToolbar.getComponent(jToolbar.getComponentCount-1);
jButton.setText('2')
jButton.setHorizontalTextPosition(jButton.CENTER)
 
% Button #3: label on right of icon, 50x23 pixels
h3 = uipushtool(hToolbar, 'CData',cdata);
drawnow; pause(0.01);
jButton = jToolbar.getComponent(jToolbar.getComponentCount-1);
jButton.setText('3...')
d = java.awt.Dimension(50, jButton.getHeight);
jButton.setMaximumSize(d); jButton.setPreferredSize(d); jButton.setSize(d)
 
% Button #4: label on left of icon, 70x23 pixels
h4 = uipushtool(hToolbar, 'CData',cdata);
drawnow; pause(0.01);
jButton = jToolbar.getComponent(jToolbar.getComponentCount-1);
jButton.setText('and 4:')
jButton.setHorizontalTextPosition(jButton.LEFT)
d = java.awt.Dimension(70, jButton.getHeight);
jButton.setMaximumSize(d); jButton.setPreferredSize(d); jButton.setSize(d)

Many additional toolbar customizations can be found here and in my book “Undocumented Secrets of MATLAB-Java Programming“. If you’d like me to design a professional-looking GUI for you, please contact me.

Caveat emptor: all this only works with the regular Java-based GUI figures, not web-based (“App-Designer”) uifigures.

]]>
https://undocumentedmatlab.com/blog/toolbar-button-labels/feed 0
Using SQLite in Matlabhttps://undocumentedmatlab.com/blog/using-sqlite-in-matlab https://undocumentedmatlab.com/blog/using-sqlite-in-matlab#respond Wed, 27 Dec 2017 21:53:54 +0000 https://undocumentedmatlab.com/?p=7255
 
Related posts:
  1. Speeding up Matlab-JDBC SQL queries Fetching SQL ResultSet data from JDBC into Matlab can be made significantly faster. ...
  2. Types of undocumented Matlab aspects This article lists the different types of undocumented/unsupported/hidden aspects in Matlab...
  3. ScreenCapture utility The ScreenCapture utility uses purely-documented Matlab for capturing a screen region as an image from within Matlab. ...
  4. Smart listbox & editbox scrollbars Matlab listbox and multi-line editbox scrollbars can easily be made smarter, for improved appearance. ...
 
]]>
MathWorks invests a huge amount of effort in recent years on supporting large distributed databases. The business case for this focus is entirely understandable, but many Matlab users have much simpler needs, which are often served by the light-weight open-source SQLite database (which claims to be the most widely-used database worldwide). Although SQLite is very widely used, and despite the fact that built-in support for SQLite is included in Matlab (for its internal use), MathWorks has chosen not to expose any functionality or wrapper function that would enable end-users to access it. In any case, I recently came across a need to do just that, when a consulting client asked me to create an interactive data-browser for their SQLite database that would integrate with their Matlab program:
SQLite data browser

In today’s post I will discuss several possible mechanisms to integrate SQLite in Matlab code, and you can take your pick among them. Except for the Database Toolbox, all the alternatives are free (open-source) libraries (even the commercial Database Toolbox relies on one of the open-source libraries, by the way).

sqlite4java

sqlite4java is a Java package by ALM Works that is bundled with Matlab for the past several years (in the %matlabroot%/java/jarext/sqlite4java/ folder). This is a lightweight open-source package that provides a minimalist and fast (although not very convenient) interface to SQLite. You can either use the package that comes with your Matlab installation, or download and use the latest version from the project repository, where you can also find documentation.

Mark Mikofski exposed this hidden nugget back in 2015, and you are welcome to view his post for additional details. Here’s a sample usage:

% Open the DB data file
db = com.almworks.sqlite4java.SQLiteConnection(java.io.File('C:\Yair\Data\IGdb 2017-11-13.sqlite'));
db.open;
 
% Prepare an SQL query statement
stmt = db.prepare(['select * from data_table where ' conditionStr]);
 
% Step through the result set rows
row = 1;
while stmt.step
   numericValues(row) = stmt.columnInt(0);    % column #0
   stringValues{row}  = stmt.columnString(1); % column #1
end
 
% Cleanup
stmt.dispose
db.dispose

Note that since sqlite4java uses a proprietary interface (similar, but not identical, to JDBC), it can take a bit of time to get used to it. I am generally a big fan of preferring built-in components over externally-installed ones, but in this particular case I prefer other alternatives.

JDBC

JDBC (Java Database Connectivity) is the industry standard for connectivity to databases. Practically all databases nowadays have at least one JDBC connector, and many DBs have multiple JDBC drivers created by different groups. As long as they all adhere to the JDBC interface standard, these drivers are all equivalent and you can choose between them based on availability, cost, support, license, performance and other similar factors. SQLite is no exception to this rule, and has several JDBC driver implementations, including xerial’s sqlite-jdbc (also discussed by Mark Mikofski) and sqlitejdbc. If you ask me,
sqlite-jdbc is better as it is being maintained with new versions released periodically.

The example above would look something like this with sqlite-jdbc:

% Add the downloaded JAR library file to the dynamic Java classpath
javaaddpath('C:\path\to\sqlite\sqlite-jdbc-3.21.0.jar')
 
% Open the DB file
jdbc = org.sqlite.JDBC;
props = java.util.Properties;
conn = jdbc.createConnection('jdbc:sqlite:C:\Yair\Data\IGdb 2017-11-13.sqlite',props);  % org.sqlite.SQLiteConnection object
 
% Prepare and run an SQL query statement
sqlStr = ['select * from data_table where ' conditionStr];
stmt = conn.createStatement;     % org.sqlite.jdbc4.JDBC4Statement object
rs = stmt.executeQuery(sqlStr);  % org.sqlite.jdbc4.JDBC4ResultSet object
 
% Step through the result set rows
rows = 1;
while rs.next
   numericValues(row) = rs.getLong('ID');
   stringValues{row}  = rs.getString('Name');
end
 
% Cleanup
rs.close
stmt.close
conn.close

Database toolbox

In addition to all the above, MathWorks sells the Database Toolbox which has an integral SQLite connector, in two flavors – native and JDBC (the JDBC connector is simply sqlite-jdbc that I mentioned above, see a short discussion here).

I assume that the availability of this feature in the DB toolbox is the reason why MathWorks has never created a documented wrapper function for the bundled sqlite4java. I could certainly understand this from a business perspective. Still, with so many free alternatives available as discussed in this post, I see not reason to purchase the toolbox merely for its SQLite connector. Then again, if you need to connect to several different database types, not just SQLite, then getting the toolbox might make sense.

mksqlite

My personal favorite is actually none of these Java-based connectors (surprise, surprise), but rather the open-source mksqlite connector by Martin Kortmann and Andreas Martin. This is a native (Mex-file) connector that acts as a direct Matlab function. The syntax is pretty straight-forward and supports SQL queries. IMHO, its usage is a much simpler than with any of the other alternatives:

% Open the DB file
mksqlite('open', 'C:\Yair\Data\IGdb 2017-11-13.sqlite');
 
% Query the database
results = mksqlite(['select * from data_table where ' conditionStr]);
numericValues = [results.ID];
stringValues  = {results.Name};
 
% Cleanup
mksqlite('close');

Can it be any simpler than this!?

However, the main benefit of mksqlite over the other connectors is not its simplicity but the connector’s speed. This speed is due to the fact that the query is vectorized and we do not need to loop over all the separate data rows and fields. With the other connectors, it is actually not the loop that takes so long in Matlab, but rather the overheads and inefficiencies of numerous library calls to fetch one single value at a time from the result-set – this is avoided in mksqlite where there is only a single call. This results in lightning speed: A couple of years ago I consulted to a client who used a JDBC connector to an SQLite database; by switching from a JDBC connector to mksqlite, I reduced the execution time from 7 secs to 70 msecs – a 100x speedup! In that specific case, this made the difference between an unusable program and a highly interactive/responsive one.

Other alternatives

In addition to all the above, we can also use a .NET-based connector or a Python one – I leave these as an exercise for the reader…

Have I forgotten some important alternative? Or perhaps you have have some related tip you’d like to share? If so, then please leave a comment below.

Happy New Year everybody!

]]>
https://undocumentedmatlab.com/blog/using-sqlite-in-matlab/feed 0
PlotEdit context-menu customizationhttps://undocumentedmatlab.com/blog/plotedit-context-menu-customization https://undocumentedmatlab.com/blog/plotedit-context-menu-customization#respond Wed, 13 Dec 2017 12:57:14 +0000 https://undocumentedmatlab.com/?p=7236
 
Related posts:
  1. FindJObj – find a Matlab component’s underlying Java object The FindJObj utility can be used to access and display the internal components of Matlab controls and containers. This article explains its uses and inner mechanism....
  2. FindJObj GUI – display container hierarchy The FindJObj utility can be used to present a GUI that displays a Matlab container's internal Java components, properties and callbacks....
  3. Tab panels – uitab and relatives This article describes several undocumented Matlab functions that support tab-panels...
  4. Graphic sizing in Matlab R2015b Matlab release R2015b's new "DPI-aware" nature broke some important functionality. Here's what can be done... ...
 
]]>
Last week, a Matlab user asked whether it is possible to customize the context (right-click) menu that is presented in plot-edit mode. This menu is displayed by clicking the plot-edit (arrow) icon on the standard Matlab figure toolbar, then right-clicking any graphic/GUI element in the figure. Unfortunately, it seems that this context menu is only created the first time that a user right-clicks in plot-edit mode – it is not accessible before then, and so it seems impossible to customize the menu before it is presented to the user the first time.

Customized plot-edit context-menu

Customized plot-edit context-menu

A few workarounds were suggested to the original poster and you are most welcome to review them. There is also some discussion about the technical reasons that none of the “standard” ways of finding and modifying menu items fail in this case.

In today’s post I wish to repost my solution, in the hope that it might help other users in similar cases.

My solution is basically this:

  1. First, enter plot-edit mode programmatically using the plotedit function
  2. Next, move the mouse to the screen location of the relevant figure component (e.g. axes). This can be done in several different ways (the root object’s PointerLocation property, the moveptr function, or java.awt.Robot.mouseMove() method).
  3. Next, automate a mouse right-click using the built in java.awt.Robot class (as discussed in this blog back in 2010)
  4. Next, locate the relevant context-menu item and modify its label, callback or any of its other properties
  5. Next, dismiss the context-menu by simulating a follow-on right-click using the same Robot object
  6. Finally, exit plot-edit mode and return the mouse pointer to its original location
% Create an initial figure / axes for demostration purpose
fig = figure('MenuBar','none','Toolbar','figure');
plot(1:5); drawnow; 
 
% Enter plot-edit mode temporarily
plotedit(fig,'on'); drawnow
 
% Preserve the current mouse pointer location
oldPos = get(0,'PointerLocation');
 
% Move the mouse pointer to within the axes boundary
% ref: https://undocumentedmatlab.com/blog/undocumented-mouse-pointer-functions
figPos = getpixelposition(fig);   % figure position
axPos  = getpixelposition(gca,1); % axes position
figure(fig);  % ensure that the figure is in focus
newPos = figPos(1:2) + axPos(1:2) + axPos(3:4)/4;  % new pointer position
set(0,'PointerLocation',newPos);  % alternatives: moveptr(), java.awt.Robot.mouseMove()
 
% Simulate a right-click using Java robot
% ref: https://undocumentedmatlab.com/blog/gui-automation-robot
robot = java.awt.Robot;
robot.mousePress  (java.awt.event.InputEvent.BUTTON3_MASK); pause(0.1)
robot.mouseRelease(java.awt.event.InputEvent.BUTTON3_MASK); pause(0.1)
 
% Modify the <clear-axes> menu item
hMenuItem = findall(fig,'Label','Clear Axes');
if ~isempty(hMenuItem)
   label = '<html><b><i><font color="blue">Undocumented Matlab';
   callback = 'web(''https://undocumentedmatlab.com'',''-browser'');';
   set(hMenuItem, 'Label',label, 'Callback',callback);
end
 
% Hide the context menu by simulating a left-click slightly offset
set(0,'PointerLocation',newPos+[-2,2]);  % 2 pixels up-and-left
pause(0.1)
robot.mousePress  (java.awt.event.InputEvent.BUTTON1_MASK); pause(0.1)
robot.mouseRelease(java.awt.event.InputEvent.BUTTON1_MASK); pause(0.1)
 
% Exit plot-edit mode
plotedit(fig,'off'); drawnow
 
% Restore the mouse pointer to its previous location
set(0,'PointerLocation',oldPos);

In this code, I sprinkled a few pauses at several locations, to ensure that everything has time to fully render. Different pause values, or perhaps no pause at all, may be needed on your specific system.

Modifying the default context-menu shown in plot-edit mode may perhaps be an uncommon use-case. But the technique that I demonstrated above – of using a combination of Matlab and Java Robot commands to automate a certain animation – can well be used in many other use-cases where we cannot easily access the underlying code. For example, when the internal code is encoded/encrypted, or when a certain functionality (such as the plot-edit context-menu) is created on-the-fly.

If you have encountered a similar use-case where such automated animations can be used effectively, please add a comment below.

]]>
https://undocumentedmatlab.com/blog/plotedit-context-menu-customization/feed 0
Builtin PopupPanel widgethttps://undocumentedmatlab.com/blog/builtin-popuppanel-widget https://undocumentedmatlab.com/blog/builtin-popuppanel-widget#comments Wed, 06 Dec 2017 16:00:34 +0000 https://undocumentedmatlab.com/?p=7188
 
Related posts:
  1. Toolbar button labels GUI toolbar button labels can easily be set and customized using underlying Java components. ...
  2. Uitable sorting Matlab's uitables can be sortable using simple undocumented features...
  3. Customizing figure toolbar background Setting the figure toolbar's background color can easily be done using just a tiny bit of Java magic powder. This article explains how. ...
  4. Frameless (undecorated) figure windows Matlab figure windows can be made undecorated (borderless, title-less). ...
 
]]>
8 years ago I blogged about Matlab’s builtin HelpPopup widget. This control is used by Matlab to display popup-windows with help documentation, but can also be used by users to display custom lightweight popups that contain HTML-capable text and even URLs of entire webpages. Today I’d like to highlight another builtin Matlab widget, ctrluis.PopupPanel, which can be used to display rich contents in a lightweight popup box attached to a specific Matlab figure:
Matlab's builtin PopupPanel widget

Matlab's builtin PopupPanel widget

As you can see, this popup-panel displays richly-formatted contents, having either an opaque or transparent background, with vertical scrollbars being applied automatically. The popup pane is not limited to displaying text messages – in fact, it can display any Java GUI container (e.g. a settings panel). This popup-panel is similar in concept to the HelpPopup widget, and yet much more powerful in several aspects.

Creating the popup panel

Creating a PopupPanel is very simple:

% Create the popup-panel in the specified figure
hPopupPanel = ctrluis.PopupPanel(gcf);  % use gcf or any figure handle
hPopupPanel.setPosition([.1,.1,.8,.8]);  % set panel position (normalized units)
 
% Alternative #1: set popup-panel's contents to some HTML-formatted message
% note: createMessageTextPane() has optional input args FontName (arg #2), FontSize (#3)
jPanel = ctrluis.PopupPanel.createMessageTextPane('testing <b><i>123</i></b> ...')
hPopupPanel.setPanel(jPanel);
 
% Alternative #2: set popup-panel's contents to a webpage URL
url = 'http://undocumentedmatlab.com/files/sample-webpage.html';
jPanel = javaObjectEDT(javax.swing.JEditorPane(url));
hPopupPanel.setPanel(jPanel);

The entire contents are embedded within a scroll-box (which is a com.mathworks.widgets.LightScrollPane object) whose scrollbars automatically appear as-needed, so we don’t need to worry about the contents fitting the allocated space.

To display custom GUI controls in the popup, we can simply contain those GUI controls in a Java container (e.g., a JPanel) and then do hPopupPanel.setPanel(jPanel). This functionality can be used to create unobtrusive settings panels, input dialogs etc.

The nice thing about the popup widget is that it is attached to the figure, and yet is not assigned a heavyweight window (so it does not appear in the OS task-bar). The popup moves along with the figure when the figure is moved, and is automatically disposed when the figure is closed.

A few caveats about the ctrluis.PopupPanel control:

  • The widget’s parent is expected to be a figure that has pixel units. If it doesn’t, the internal computations of ctrluis.PopupPanel croak.
  • The widget’s position is specified in normalized units (default: [0,0,1,1]). This normalized position is only used during widget creation: after creation, if you resize the figure the popup-panel’s position remains unchanged. To modify/update the position of the popup-panel programmatically, use hPopupPanel.setPosition(newPosition). Alternatively, update the control’s Position property and then call hPopupPanel.layout() (there is no need to call layout when you use setPosition).
  • This functionality is only available for Java-based figures, not the new web-based (AppDesigner) uifigures.

Popup panel customizations

We can open/close the popup panel by clicking on its icon, as shown in the screenshots above, or programmatically using the control’s methods:

% Programmatically open/close the popup-panel
hPopupPanel.showPanel;
hPopupPanel.hidePanel;
 
% Show/hide entire popup-panel widget (including its icon)
hPopupPanel.setVisible(true);   % or .setVisible(1) or .Visible=1
hPopupPanel.setVisible(false);  % or .setVisible(0) or .Visible=0

To set a transparent background to the popup-panel (as shown in the screenshots above), we need to unset the opacity of the displayed panel and several of its direct parents:

% Set a transparent popup-panel background
for idx = 1 : 6
   jPanel.setOpaque(false);  % true=opaque, false=transparent
   jPanel = jPanel.getParent;
end
jPanel.repaint

Note that in the screenshots above, the panel’s background is made transparent, but the contained text and image remain opaque. Your displayed images can of course contain transparency and animation, if this is supported by the image format (for example, GIF).

iptui.internal.utilities.addMessagePane

ctrluis.PopupPanel is used internally by iptui.internal.utilities.addMessagePane(hFig,message) in order to display a minimizable single-line message panel at the top of a specified figure:

hPopupPanel = iptui.internal.utilities.addMessagePane(gcf, 'testing <b>123</b> ...');  % note the HTML formatting

The function updates the message panel’s position whenever the figure’s size is modified (by trapping the figure’s SizeChangedFcn), to ensure that the panel is always attached to the top of the figure and spans the full figure width. This is a simple function so I encourage you to take a look at its code (%matlabroot%/toolbox/images/imuitools/+iptui/+internal/+utilities/addMessagePane.m) – note that this might require the Image Processing Toolbox (I’m not sure).

Matlab's builtin iptui.internal.utilities.addMessagePane

Matlab's builtin iptui.internal.utilities.addMessagePane

Professional assistance anyone?

As shown by this and many other posts on this site, a polished interface and functionality is often composed of small professional touches, many of which are not exposed in the official Matlab documentation for various reasons. So if you need top-quality professional appearance/functionality in your Matlab program, or maybe just a Matlab program that is dependable, robust and highly-performant, consider employing my consulting services.

]]>
https://undocumentedmatlab.com/blog/builtin-popuppanel-widget/feed 1
Customizing uifigures part 3https://undocumentedmatlab.com/blog/customizing-uifigures-part-3 https://undocumentedmatlab.com/blog/customizing-uifigures-part-3#comments Mon, 27 Nov 2017 15:00:24 +0000 https://undocumentedmatlab.com/?p=7169
 
Related posts:
  1. Customizing uifigures part 2 Matlab's new web-based uifigures can be customized using custom CSS and Javascript code. ...
  2. uiundo – Matlab’s undocumented undo/redo manager The built-in uiundo function provides easy yet undocumented access to Matlab's powerful undo/redo functionality. This article explains its usage....
  3. FindJObj – find a Matlab component’s underlying Java object The FindJObj utility can be used to access and display the internal components of Matlab controls and containers. This article explains its uses and inner mechanism....
  4. FindJObj GUI – display container hierarchy The FindJObj utility can be used to present a GUI that displays a Matlab container's internal Java components, properties and callbacks....
 
]]>
As I have repeatedly posted in recent years, Matlab is advancing towards web-based GUI. The basic underlying technology is more-or-less stable: an HTML/Javascript webpage that is created-on-the-fly and rendered in a stripped-down browser window (based on Chromium-based jxBrowser in recent years). However, the exact mechanism by which the controls (“widgets”) are actually converted into visible components (currently based on the Dojo toolkit and its Dijit UI library) and interact with Matlab (i.e., the internal Matlab class structures that interact with the browser and Dojo) is still undergoing changes and is not quite as stable.

Customization hacks reported on this blog last year (part 1, part 2) may fail in some cases due to the changing nature of the undocumented internals. Some examples are the way by which we can extract the uifigure’s URL (which changed in R2017a), the ability to display and debug uifigures in a standard webbrowser with associated dev tools (which seems to have stopped working in R2017b), and the way by which we can extract the Dijit reference of displayed uicontrols.

Greatly assisting in this respect is Iliya Romm, who was the guest blogger for part 2 of this series last year. Iliya co-authored the open-source (GitHub) mlapptools toolbox, which enables accessing and customizing uifigure components using standard CSS, without users having to bother about the ugly hacks discussed in the previous parts of the series. This toolbox is really just a single Matlab class (mlapptools), contained within a single m-file (mlapptools.m). In addition to this class, the toolbox includes a README.md mark-down usage documentation, and two demo functions, DOMdemoGUI.m and TableDemo.m.

Here is the effect of using TableDemo, that shows how we can customize individual uitable cells (each uitable cell is a separate Dijit widget that can be customized individually):

CSS customizations of uifigure components

CSS customizations of uifigure components


The mlapptools class contains several static methods that can be used individually:

  • textAlign(uielement, alignment) – Modify text horizontal alignment ('left', 'center', 'right', 'justify' or 'initial')
  • fontWeight(uielement, weight) – Modify font weight ('normal', 'bold', 'bolder', 'lighter' or 'initial'), depending on availability in the font-face used
  • fontColor(uielement, color) – Modify font color (e.g. 'red', '#ff0000', 'rgb(255,0,0)' or other variants)
  • setStyle(uielement, styleAttr, styleValue) – Modify a specified CSS style attribute
  • aboutDojo() – Return version information about the Dojo toolkit
  • getHTML(hFig) – Return the full HTML code of a uifigure
  • getWebWindow(hFig) – Return a webwindow handle from a uifigure handle
  • getWebElements (hControl) – Return a webwindow handle and a widget ID for the specified uicontrol handle
  • getWidgetList(hFig, verboseFlag) – Return a cell-array of structs containing information about all widgets in the uifigure
  • getWidgetInfo(hWebwindow, widgetId, verboseFlag) – Return information about a specific dijit widget
  • setTimeout(hFig, seconds) – Override the default timeout (=5 secs) for dojo commands, for a specific uifigure

A few simple usage examples:

mlapptools.fontColor(hButton,'red')  % set red text color
mlapptools.fontWeight(hButton,'bold')  % set bold text font
mlapptools.setStyle(hButton,'border','2px solid blue')  % add a 2-pixel solid blue border
mlapptools.setStyle(hButton,'background-image','url(https://www.mathworks.com/etc/designs/mathworks/img/pic-header-mathworks-logo.svg)')  % add background image

Once you download mlapptools and add its location to the Matlab path, you can use it in any web-based GUI that you create, either programmatically or with Add-Designer.

The mlapptools is quite well written and documented, so if you are interested in the inner workings I urge you to take a look at this class’s private methods. For example, to understand how a Matlab uicontrol handle is converted into a Dojo widget-id, which is then used with the built-in dojo.style() Javascript function to modify the CSS attributes of the HTML <div> or <span> that are the control’s visual representation on the webpage. An explanation of the underlying mechanism can be found in part 2 of this series of articles on uifigure customizations. Note that the mlapptools code is newer than the article and contains some new concepts that were not covered in that article, for example searching through Dijit’s registry of displayed widgets.

Note: web-based GUI is often referred to as “App-Designed” (AD) GUI, because using the Matlab App Designer is the typical way to create and customize such GUIs. However, just as great-looking GUIs could be created programmatically rather than with GUIDE, so too can web-based GUIS be created programmatically, using regular built-in Matlab commands such as uifigure, uibutton and uitable (an example of such programmatic GUI creation can be found in Iliya’s TableDemo.m, discussed above). For this reason, I believe that the new GUIs should be referred to as “uifigures” or “web GUIs”, and not as “AD GUIs”.

If you have any feature requests or bugs related to mlapptools, please report them on its GitHub issues page. For anything else, please add a comment below.

]]>
https://undocumentedmatlab.com/blog/customizing-uifigures-part-3/feed 2
Customizing contour plots part 2https://undocumentedmatlab.com/blog/customizing-contour-plots-part2 https://undocumentedmatlab.com/blog/customizing-contour-plots-part2#comments Sun, 12 Nov 2017 11:03:37 +0000 https://undocumentedmatlab.com/?p=7149
 
Related posts:
  1. Draggable plot data-tips Matlab's standard plot data-tips can be customized to enable dragging, without being limitted to be adjacent to their data-point. ...
  2. Customizing contour plots Contour labels, lines and fill patches can easily be customized in Matlab HG2. ...
  3. Matlab’s HG2 mechanism HG2 is presumably the next generation of Matlab graphics. This article tries to explore its features....
  4. getundoc – get undocumented object properties getundoc is a very simple utility that displays the hidden (undocumented) properties of a specified handle object....
 
]]>
A few weeks ago a user posted a question on Matlab’s Answers forum, asking whether it is possible to display contour labels in the same color as their corresponding contour lines. In today’s post I’ll provide some insight that may assist users with similar customizations in other plot types.

Matlab does not provide, for reasons that escape my limited understanding, documented access to the contour plot’s component primitives, namely its contour lines, labels and patch faces. Luckily however, these handles are accessible (in HG2, i.e. R2014b onward) via undocumented hidden properties aptly named EdgePrims, TextPrims and FacePrims, as I explained in a previous post about contour plots customization, two years ago.

Let’s start with a simple contour plot of the peaks function:

[X,Y,Z] = peaks;
[C,hContour] = contour(X,Y,Z, 'ShowText','on', 'LevelStep',1);

The result is the screenshot on the left:

Standard Matlab contour labels

Standard Matlab contour labels

 
Customized Matlab contour labels

Customized Matlab contour labels

In order to update the label colors (to get the screenshot on the right), we create a short updateContours function that updates the TextPrims color to their corresponding EdgePrims color:

The updateContours() function

function updateContours(hContour)
    % Update the text label colors
    drawnow  % very important!
    levels = hContour.LevelList;
    labels = hContour.TextPrims;  % undocumented/unsupported
    lines  = hContour.EdgePrims;  % undocumented/unsupported
    for idx = 1 : numel(labels)
        labelValue = str2double(labels(idx).String);
        lineIdx = find(abs(levels-labelValue)<10*eps, 1);  % avoid FP errors using eps
        labels(idx).ColorData = lines(lineIdx).ColorData;  % update the label color
        %labels(idx).Font.Size = 8;                        % update the label font size
    end
    drawnow  % optional
end

Note that in this function we don’t directly equate the numeric label values to the contour levels’ values: this would work well for integer values but would fail with floating-point ones. Instead I used a very small 10*eps tolerance in the numeric comparison.

Also note that I was careful to call drawnow at the top of the update function, in order to ensure that EdgePrims and TextPrims are updated when the function is called (this might not be the case before the call to drawnow). The final drawnow at the end of the function is optional: it is meant to reduce the flicker caused by the changing label colors, but it can be removed to improve the rendering performance in case of rapidly-changing contour plots.

Finally, note that I added a commented line that shows we can modify other label properties (in this case, the font size from 10 to 8). Feel free to experiment with other label properties.

Putting it all together

The final stage is to call our new updateContours function directly, immediately after creating the contour plot. We also want to call updateContours asynchronously whenever the contour is redrawn, for example, upon a zoom/pan event, or when one of the relevant contour properties (e.g., LevelStep or *Data) changes. To do this, we add a callback listener to the contour object’s [undocumented] MarkedClean event that reruns our updateContours function:

[X,Y,Z] = peaks;
[C,hContour] = contour(X,Y,Z, 'ShowText','on', 'LevelStep',1);
 
% Update the contours immediately, and also whenever the contour is redrawn
updateContours(hContour);
addlistener(hContour, 'MarkedClean', @(h,e)updateContours(hContour));

Contour level values

As noted in my comment reply below, the contour lines (hContour.EdgePrims) correspond to the contour levels (hContour.LevelList).

For example, to make all negative contour lines dotted, you can do the following:

[C,hContour] = contour(peaks, 'ShowText','on', 'LevelStep',1); drawnow
set(hContour.EdgePrims(hContour.LevelList<0), 'LineStyle', 'dotted');

Customized Matlab contour lines

Customized Matlab contour lines

Prediction about forward compatibility

As I noted on my previous post on contour plot customization, I am marking this article as “High risk of breaking in future Matlab versions“, not because of the basic functionality (being important enough I don’t presume it will go away anytime soon) but because of the property names: TextPrims, EdgePrims and FacePrims don’t seem to be very user-friendly property names. So far MathWorks has been very diligent in making its object properties have meaningful names, and so I assume that when the time comes to expose these properties, they will be renamed (perhaps to TextHandles, EdgeHandles and FaceHandles, or perhaps LabelHandles, LineHandles and FillHandles). For this reason, even if you find out in some future Matlab release that TextPrims, EdgePrims and FacePrims don’t exist, perhaps they still exist and simply have different names. Note that these properties have not changed their names or functionality in the past 3 years, so while it could well happen next year, it could also remain unchanged for many years to come. The exact same thing can be said for the MarkedClean event.

Professional assistance anyone?

As shown by this and many other posts on this site, a polished interface and functionality is often composed of small professional touches, many of which are not exposed in the official Matlab documentation for various reasons. So if you need top-quality professional appearance/functionality in your Matlab program, or maybe just a Matlab program that is dependable, robust and highly-performant, consider employing my consulting services.

]]>
https://undocumentedmatlab.com/blog/customizing-contour-plots-part2/feed 6
The HotLinks featurehttps://undocumentedmatlab.com/blog/the-hotlinks-feature https://undocumentedmatlab.com/blog/the-hotlinks-feature#comments Tue, 24 Oct 2017 12:43:13 +0000 https://undocumentedmatlab.com/?p=7128
 
Related posts:
  1. uiundo – Matlab’s undocumented undo/redo manager The built-in uiundo function provides easy yet undocumented access to Matlab's powerful undo/redo functionality. This article explains its usage....
  2. Matlab’s HG2 mechanism HG2 is presumably the next generation of Matlab graphics. This article tries to explore its features....
  3. New information on HG2 More information on Matlab's new HG2 object-oriented handle-graphics system...
  4. Pinning annotations to graphs Annotation object can be programmatically set at, and pinned-to, plot axes data points. ...
 
]]>
Back in 2010, I posted about Matlab’s undocumented feature function. One of the features that I mentioned was 'HotLinks'. A few days ago I had an occasion to remember this feature when a StackOverflow user complained that the headers of table outputs in the Matlab console appear with HTML tags (<strong>) in his diary output. He asked whether it was possible to turn off this automated headers markup.

There are several ways this problem can be solved, ranging from creating a custom table display function, to modifying the table’s internal disp method (%matlabroot%/toolbox/matlab/datatypes/@tabular/disp.m), to using this method’s second optional argument (disp(myTable,false)). Note that simply subclassing the table class to overload disp() will not work because the table class is Sealed, but we could instead subclass table‘s superclass (tabular) just like table does.

Inside the disp.m method mentioned above, the headers markup is controlled (around line 45, depending on your Matlab release) by matlab.internal.display.isHot. Unfortunately, there is no corresponding setHot() method, nor corresponding m- or p-code that can be inspected. But the term “Hot” rang a bell, and then I remembered my old post about the HotLinks feature, which is apparently reflected by matlab.internal.display.isHot.

feature('HotLinks',false);  % temporarily disable bold headers and hyperlinks (matlab.internal.display.isHot=false)
disp(myTable)
myTable        % this calls disp() implicitly
feature('HotLinks',true);   % restore the standard behavior (markup displayed, matlab.internal.display.isHot=true)

Searching for “isHot” or “HotLinks” under the Matlab installation folder, we find that this feature is used in hundreds of places (the exact number depends on your installed toolboxes). The general use appears to be to disable/enable output of hyperlinks to the Matlab console, such as when you display a Matlab class, when its class name is hyperlinked and so is the “Show all properties” message at the bottom. But in certain cases, such as for the table output above, the feature is also used to determine other types of markup (bold headers in this case).

>> feature('HotLinks',0)  % temporarily disable bold headers and hyperlinks
>> groot
ans = 
  Graphics Root with properties:
 
          CurrentFigure: [0×0 GraphicsPlaceholder]
    ScreenPixelsPerInch: 96
             ScreenSize: [1 1 1366 768]
       MonitorPositions: [2×4 double]
                  Units: 'pixels'
 
  Use GET to show all properties

>> feature('HotLinks',1)  % restore the standard behavior (markup displayed)
>> groot
ans = 
  Graphics Root with properties:
 
          CurrentFigure: [0×0 GraphicsPlaceholder]
    ScreenPixelsPerInch: 96
             ScreenSize: [1 1 1366 768]
       MonitorPositions: [2×4 double]
                  Units: 'pixels'
 
  Show all properties

There’s nothing really earth shuttering in all this, but the HotLinks feature could be useful when outputting console output into a diary file. Of course, if diary would have automatically stripped away markup tags we would not need to resort to such hackery. Then again, this is not the only problem with diary, which is long-overdue an overhaul.

]]>
https://undocumentedmatlab.com/blog/the-hotlinks-feature/feed 1