Java Swing’s Event Dispatch Thread (EDT)
or: why does my GUI foul up?

Matlab for the most part is a single threaded environment. That is, all commands are executed sequentially along a single execution thread. The main exception to this are the Handle Graphics (GUI) components whose operations execute on the Java Event Dispatch Thread (EDT). EDT effects are reflected even in mundane Matlab GUI operations.
If we execute the code below we will probably see nothing until the loop completes and the figure appears with the text label showing ‘10000’:

h = figure;
txt = uicontrol('Parent',h, 'Style','text', 'String','1');
for n = 1:10000
    set(txt,'String',int2str(n))
end %for

By adding a couple of drawnow commands we get the figure and text label to render and then we see the count progress to 10000.

h = figure;
txt = uicontrol('Parent',h, 'Style','text', 'String','1');
drawnow;
for n = 1:10000
    set(txt,'String',int2str(n));
    drawnow;
end %for

The drawnow function allows the EDT queue to be flushed and the pending graphics operations to be evaluated. This will also happen with pause and several other commands.
If we want to use Swing (or AWT) components in our user interfaces we need to take this multi-threaded environment into account. The Swing toolkit designers decided to make all the Swing components thread un-safe in order to decrease their complexity. As a consequence, all access to Swing components should be done from the event dispatch thread (EDT), to ensure the operations are executed sequentially, at the exact order in which they were dispatched. Any action on a Swing component done on another thread (Matlab’s main processing thread in our case) risks a race-condition or deadlock with the EDT, which could (and often does) result in weird, non-deterministic and non-repetitive behavior – all of which should be avoided in any application which should behave in a precisely deterministic manner.
In Java, the usual pattern to accomplish EDT dispatching is to create a Runnable object, encapsulate the GUI code in the run method of the Runnable object, then pass the Runnable object to the static EventQueue.invokeLater (or EventQueue.invokeAndWait if we need to block operations to get a return value) method.

Runnable runnable = new Runnable()
{
    public void run()
    {
        //GUI Code here
    }
}
EventQueue.invokeLater(runnable);

There are several functions in Matlab that implement this programming pattern for us: javaObjectEDT, javaMethodEDT, awtinvoke, awtcreate and javacomponent. JavaMethodEDT and javaObjectEDT were introduced in version R2008b (7.7) and are minimally and only partially documented although they have reasonably complete help comments. The other three are semi-documented (meaning they are unsupported but if you edit or type their m-file you’ll see a fairly detailed help section), and although there is some overlap in their functionality they are still available.

javaObjectEDT and javaMethodEDT

javaObjectEDT is the the preferred method since R2008b of creating swing components to be used on the EDT. An object created with javaObjectEDT will have all of its subsequent method calls run on the EDT. This is termed Auto Delegation. Auto-delegation greatly simplifies and increases the readability of code. Note that objects created as a result of method calls may not be implemented on the EDT.
If you have an existing Java object, you can pass it to javaObjectEDT at any time – all its subsequent calls will then onward run on the EDT. Note that this useful functionality is an under-documented javaObjectEDT feature: it is not mentioned in the main help section but only implied from the example.

% Create a button on the EDT
btn = javaObjectEDT('javax.swing.JButton');
% this will run on EDT since btn was javaObjectEDT-created
btn.setText('Button');
% Create a button NOT on the EDT
btn2 = javax.swing.JButton;
% Dangerous! call will run on main Matlab thread
btn2.setText('Button2');
% modify btn2 so its methods will start running on the EDT
javaObjectEDT(btn2);
btn2.setText('Button2');

The following example shows the use of javaObjectEDT and javaMethodEDT in a more complex situation using a JTable:

function tableExample
hFig = figure;
drawnow; %need to get figure rendered
%use Yair's createTable to add a javax.swing.JTable
%http://www.mathworks.com/matlabcentral/fileexchange/14225-java-based-data-table
%wrap ceateTable in javaObjectEDT to put the ensuing method calls on the EDT
f = java.awt.Font(java.lang.String('Dialog'),java.awt.Font.PLAIN,14);
headers = {'Selected','File','Analysis Routine','Task Status'};
tbl = javaObjectEDT(createTable(hFig,headers,[],false,'Font',f));
%set column 1 to use check boxes and set up a change callback
tbl.setCheckBoxEditor(1);
jtable = javaObjectEDT(tbl.getTable); %get the underlying Java Table. IMPORTANT: we need to put jtable on the EDT
columnModel = javaObjectEDT(jtable.getColumnModel); %now we can now do direct calls safely on jtable
selectColumn = javaObjectEDT(columnModel.getColumn(0));
selectColumnCellEditor = selectColumn.getCellEditor;
chk = javaMethodEDT('getComponent',selectColumnCellEditor);
set(chk,'ItemStateChangedCallback',@chkChange_Callback);
%make column three a combo drop down
analysisTable = {'Analysis1';'Analysis2';'Analysis3'};
cb = javaObjectEDT('com.mathworks.mwswing.MJComboBox',analysisTable);
cb.setEditable(false);
cb.setFont(f);
set(cb,'ItemStateChangedCallback',@cbChange_Callback);
editor = javaObjectEDT('javax.swing.DefaultCellEditor',cb);
analysisColumn = javaObjectEDT(columnModel.getColumn(2));
analysisColumn.setCellEditor(editor);
%set some column with restrictions
selectColumn.setMaxWidth(100);
analysisColumn.setPreferredWidth(300);
%set the data
SELECTED = java.awt.event.ItemEvent.SELECTED;
tbl.setData({false,'file1','Analysis2','Analysis2';...
             true,'file2','Analysis3','Analysis3'});
drawnow;
    function cbChange_Callback(src,ev) %#ok
        jRow = jtable.getSelectedRow;
        stateChange = javaMethodEDT('getStateChange',ev);
        if stateChange == SELECTED
            newData = javaMethodEDT('getItem',ev);
            model = jtable.getModel;
            javaMethodEDT('setValueAt',model,newData,jRow,3);
        end %if
    end %cbChange
    function chkChange_Callback(src,ev) %#ok
        chkBox = javaMethodEDT('getItem',ev);
        if logical(javaMethodEDT('isSelected',chkBox))
            beep; %put useful code here
        else
            beep;
            pause(0.1)
            beep; %put useful code here
        end %if
    end %chkChange_Callback
end %tableExample

If you are running Matlab R2008a or later, javacomponent uses the javaObjectEDT function to create the returned objects so you do not have to do anything further to these objects to have their calls dispatched on the EDT. Users need to take care that objects added directly to the components created by javacomponent are on the EDT as well as specialized sub-components (e.g. CellRenderers and CellEditors). The overhead of calling javaMethodEDT is fairly small so if in doubt, use it.
javaObjectEDT and its kin first appeared in R2008a, although they only became supported in R2008b. Unfortunately, using them on R2008a sometimes causes hangs and all sorts of other mis-behaviors. This problem was fixed in the R2008b release, when javaObjectEDT became a fully-supported function. The problem with using javaObjectEDT in our application is that if it ever runs on an R2008a platform it might hang! (on Matlab release R2007b and earlier we will get an informative message saying that the javaObjectEDT function does not exist)
For this reason, I am using the following method in my projects:

function result = javaObjEDT(varargin)
%Placeholder of Matlab's buggy javaObjectEDT function on R2008a
% Programmed by Yair M. Altman: altmany(at)gmail.com
% $Revision: 1.2 $  $Date: 2009/01/25 11:31:08 $
  try
      try
          result = varargin{1};
      catch
          result = [];
      end
      v = version;
      if str2double(v(1:3)) > 7.6
          result = builtin('javaObjectEDT',varargin{:});
      end
  catch
      % never mind
  end
end

Note that javaMethodEDT has the method name as its first input argument, and the object name or reference as its second arg. This is inconsistent with many other Matlab/Java functions, which normally accept the target object as the first argument (compare: invoke, awtinvoke, notify etc.). It also means that we cannot use the familiar obj.javaMethodEDT(methodName) format.
One final note: when javaObjectEDT and javaMethodEDT first appeared in R2008a, they were complemented by the javaObjectMT and javaMethodMT functions, which create and delegate Java objects on the main Matlab computational thread. Their internal documentation says that there are cases when execution must occur on the MT rather than EDT, although I am personally not aware of any such case.

awtcreate and awtinvoke

For users with versions prior to R2008b the user must use the awtcreate function to create objects on the EDT. One huge disadvantage of this older function is that if you have to pass java objects in the parameter list you must use the very cumbersome JNI style notation. For example, for the simple task of setting a button label, one has to use:

btn = awtcreate('javax.swing.JButton');
awtinvoke(btn,'setText(Ljava/lang/String;)','click me')

The other disadvantage is that creating the object using awtcreate does not ensure that its subsequent method calls will be executed on the EDT. The awtinvoke function must be used for each call.
Also, both awtcreate and awtinvoke have some limitations due to bugs in the private parseJavaSignature function (for example, invoking methods which accept a java.lang.Object) which forces one to use the direct call to the method, using the main Matlab thread. This can result in the undesired effects described above. In this situation the best workaround is to call pause(0.01) to allow the event queue to clear.
Versions of javacomponent earlier than R2008a use awtcreate and objects created by these versions must have their subsequent methods called by awtinvoke to be used on the EDT.
A very rare CSSM thread discusses the usage of awtcreate and awtinvoke with some very interesting remarks by MathWorks personnel.
There is an interesting option in awtinvoke that was not carried over into the newer javaMethodEDT. This option allows the user to pass a function handle in the argument list along with its parameters. This option creates an undocumented com.mathworks.jmi.Callback object that has a delayed callback. The delayed callback is dispatched on the EDT so that it will be called once the java method used in awtinvoke is finished. Note that the actual function will still execute on the main Matlab thread the delayed callback will just control when it is called. However this may be useful at times. It is possible to put this functionality into a separate function we can call to delay execution until the event queue is cleared.

%CALLBACKONEDTQUEUE will place a callback on the EDT to asynchronously
%run a function.
%CALLBBACKONEDTQUEUE(FCN) will run function handle FCN once all previous
%methods dispatched to the EDT have completed.
%CALLBBACKONEDTQUEUE(FCN,ARG1,ARG2,...) ill run function handle FCN with
%arguments ARG1,ARG2...once all previous methods dispatched to the EDT
%have completed.
%Note that the function is still executing on the main Matlab thread. This
%function just delays when it will be called.
function callbackOnEDTQueue(varargin)
    validateattributes(varargin{1},{'function_handle'},{});
    callbackObj = handle(com.mathworks.jmi.Callback,'callbackProperties');
    set(callbackObj,'delayedCallback',{@cbEval,varargin(:)});
    callbackObj.postCallback;
    function cbEval(src,evt,args) %#ok
        feval(args{:});
    end %cbEval
end %callbackOnEDTQueue

The post Matlab and the Event Dispatch Thread (EDT) appeared first on Undocumented Matlab.

function inactiveBtnToolTip
  %Illustrates how to make a tooltip appear on an inactive control
  h = figure('WindowButtonMotionFcn',@windowMotion,'Pos',[400,400,200,200]);
  col = get(h,'color');
  lbl = uicontrol('Style','text', 'Pos',[10,160,120,20], ...
                  'Background',col, 'HorizontalAlignment','left');
  btn = uicontrol('Parent',h, 'String','Button', ...
                  'Enable','inactive', 'Pos',[10,40,60,20]);
  uicontrol('Style','check', 'Parent',h, 'String','Enable button tooltip', ...
            'Callback',@chkTooltipEnable, 'Value',1, ...
            'Pos',[10,80,180,20], 'Background',col);
  drawnow;
  %create the tooltip and postTip action
  jBtn = findjobj(btn);
  import java.awt.event.ActionEvent;
  javaMethodEDT('setToolTipText',jBtn,'This button is inactive');
  actionMap = javaMethodEDT('getActionMap',jBtn);
  action = javaMethodEDT('get',actionMap,'postTip');
  actionEvent = ActionEvent(jBtn, ActionEvent.ACTION_PERFORMED, 'postTip');
  %get the extents plus 2 pixels of the control to compare to the mouse position
  btnPos = getpixelposition(btn)+[-2,-2,4,4]; %give a little band around the control
  left = btnPos(1);
  right = sum(btnPos([1,3]));
  btm = btnPos(2);
  top =  sum(btnPos([2,4]));
  % add a listener on mouse movement events
  tm = javax.swing.ToolTipManager.sharedInstance; %tooltip manager
  pointListener = handle.listener(h,'WindowButtonMotionEvent',@figMouseMove);
  %inControl is a flag to prevent multiple triggers of the postTip action
  %while mouse remains in the button
  inControl = false;
  function figMouseMove(src,evtData) %#ok
    %get the current point
    cPoint = evtData.CurrentPoint;
    if cPoint(1) >= left && cPoint(1) <= right &&...
       cPoint(2) >= btm  && cPoint(2) <= top
      if ~inControl %we just entered
        inControl = true;
        action.actionPerformed(actionEvent); %show the tooltip
      end %if
    else
      if inControl %we just existed
        inControl = false;
        %toggle to make it disappear when leaving button
        javaMethodEDT('setEnabled',tm,false);
        javaMethodEDT('setEnabled',tm,true);
      end %if
    end %if
  end %gpMouseMove
  function windowMotion(varargin)
    %illustrate that we can still do a regular window button motion callback
    set(lbl,'String',sprintf('Mouse position: %d, %d',get(h,'CurrentPoint')));
    drawnow;
  end %windowMotion
  function chkTooltipEnable(src,varargin)
    if get(src,'Value')
      set(pointListener,'Enable','on');
    else
      set(pointListener,'Enable','off');
    end %if
  end %chkTooltipEnable
end %inactiveBtnToolTip

1. The code illustrates that we can successfully add an additional listener to listen for mouse motion events while still carrying out the original WindowButtonMotionFcn callback. This makes chaining callbacks much easier.
2. The handle.listener object has an Enable property that we can use to temporarily turn the listener on and off. This can be seen in the chkTooltipEnable() callback for the check box in the code above. If we wanted to permanently remove the listener we would simply use delete(pointListener). Note that addlistener adds a hidden property to the object being listened to, so that the listener is tied to the object's lifecycle. If you create a listener directly using handle.listener you are responsible for it's disposition. Unfortunately, addlistener fails for HG handles on pre-R2009 Matlab releases, so we use handle.listener directly.
3. The code illustrates a good practice when tracking rapidly firing events like mouse movement of handling reentry into the callback while it is still processing a previous callback. Here we use a flag called inControl to prevent the postTip action being continuously fired while the mouse remains in the control.
4. I was unable to determine if there is any corresponding action for the postTip to dismiss tips so I resorted to using the ToolTipManager to toggle its own Enable property to cleanly hide the tooltip as the mouse leaves the control.

Each Matlab callback has an associated event with it. Some of the ones that might be immediately useful at the figure-level are WindowButtonDown, WindowButtonUp, WindowKeyPress, and WindowKeyRelease. They can all be accessed through handle.listener or addlistener as in the code above.
Unfortunately, events do not always have names that directly correspond to the callback names. In order to see the list of available events for a particular Matlab object, use the following code, which relies on another undocumented function - classhandle. Here we list the events for gcf:

>> get(get(classhandle(handle(gcf)),'Events'),'Name')
ans =
    'SerializeEvent'
    'FigureUpdateEvent'
    'ResizeEvent'
    'WindowKeyReleaseEvent'
    'WindowKeyPressEvent'
    'WindowButtonUpEvent'
    'WindowButtonDownEvent'
    'WindowButtonMotionEvent'
    'WindowPostChangeEvent'
    'WindowPreChangeEvent'

Note that I have made extensive use of the javaMethodEDT function to execute Java methods that affect swing components on Swing's Event Dispatch Thread. I plan to write about this and related functions in my next article.

The post Inactive Control Tooltips & Event Chaining appeared first on Undocumented Matlab.

Situation 1: Displaying a tooltip on disabled controls

One issue with the stock Matlab uicontrol tooltips is that if you turn the uicontrol’s Enable property to ‘inactive’ or ‘off’, its tooltip no longer displays. This is the behavior that we normally want, but occasionally we wish to display a tooltip on a disabled control, for example, to explain why the control is disabled.
You can use the findjobj utility to find the Java handle for the uicontrol. This handle can then be used to set the tooltip text. The tooltip will display if you disable the control using its Java handle’s Enabled property rather than the Matlab handle’s Enable property:

hButton = uicontrol('String','Button');
drawnow;
jButton= findjobj(hButton);
set(jButton,'Enabled',false);
set(jButton,'ToolTipText','This is disabled for a reason');

As customary for Java objects, its properties can also be set using their corresponding accessor methods:

javaMethodEDT('setEnabled',jButton,false);
javaMethodEDT('setToolTipText',jButton,'Button is disabled for a reason');

hEditbox = uicontrol('String','Edit Text','Style','edit');
drawnow;
jEditbox = findjobj(hEditbox);
set(jEditbox,'Editable',false);
set(jEditbox,'ToolTipText','Text is inactive for a reason');

Situation 2: Displaying a tooltip on truncated text

If we want to conditionally display a tooltip for an editbox uicontrol when the text exceeds the control’s width, we can use the TipWhenTruncatedEnabled property (or its corresponding setTipWhenTruncatedEnabled() method). This will display a tooltip with the editbox contents if the string is shown truncated. This saves the user having to scroll through the control to see its contents. I often use this for edit controls that may contain long path names:

hEditbox(1) = uicontrol('Style','edit','Units','norm','Pos',[0.1,0.8,0.4,0.05], 'String','Too Short');
hEditbox(2) = uicontrol('Style','edit','Units','norm','Pos',[0.1,0.7,0.2,0.05], 'String','Long Enough to Display a Tool Tip');
drawnow;
jEditbox1 = findjobj(hEditbox(1));
jEditbox2 = findjobj(hEditbox(2));
set(jEditbox1,'TipWhenTruncatedEnabled',true);  % property-based alternative
javaMethod('setTipWhenTruncatedEnabled',jEditbox2,true);  % method-based alternative

Situation 3: Controlling tooltip timing

As you have probably noticed, there is a slight delay between the time your mouse enters the control and when the tooltip actually appears. If you display a tooltip over a control for sufficiently long the tooltip will then disappear. Sometimes the default delays are too slow or fast for your application. These times can be controlled through the javax.swing.ToolTipManager. The ToolTipManager sets these parameters globally (including for your Matlab desktop components), but they are not persistent between sessions.
Some examples using the ToolTipManager:

btn = uicontrol('String','Button','Tooltip','This is a button.','Pos',[100,100,75,25]);
txt = uicontrol('Style','edit','String','Edit Text','Tooltip','This is editable text','Pos',[100,50,75,25]);
tm = javax.swing.ToolTipManager.sharedInstance; %static method to get ToolTipManager object
initialDelay = javaMethodEDT('getInitialDelay',tm); %get the delay before display in milliseconds (=750 on my system)
javaMethodEDT('setInitialDelay',tm,0); %set tooltips to appear immediately
dismissDelay = javaMethodEDT('getDismissDelay',tm); %get the delay before the tooltip disappears (=10000 (10 sec) on my system)
javaMethodEDT('setDismissDelay',tm,2000); %set the dismiss delay to 2 seconds
javaMethodEDT('setEnabled',tm,false); %turns off all tooltips in system (including the Matlab desktop)
javaMethodEDT('setEnabled',tm,true);
javaMethodEDT('setInitialDelay',tm,initialDelay);
javaMethodEDT('setDismissDelay',tm,dismissDelay);

Note that I have made extensive use of the undocumented built-in javaMethodEDT function to execute Java methods that affect Swing components on the Swing Event Dispatch Thread (EDT). I plan to write about EDT following my next post on event callback chaining.

The post Additional uicontrol tooltip hacks appeared first on Undocumented Matlab.

The problem

The problem arose as I was refactoring some code that was written in R2006b to R2009b. The code in question uses the impoint tool on an image along with an associated text object that moves with the point to display information as it is dragged around the image. At the time of the R2006b release the impoint tool was written as an API. In R2006b the call to impoint returns a handle to an hggroup containing a structure of function handles in its application data under the tag ‘API’. This programming pattern was common before the advent of the new class syntax in MATLAB version 7.6 (R2008a).
Here is an example of how impoint would be used in R2006b:

function impointBehavior_R2006b
%IMPOINTBEHAVIOR_R2006B shows how impoint would be used in R2006b
%Note: RUN UNDER R2006B (will run under R2009b but actually uses
%classdef impoint so it will show the same issue)
  % Display the image in a figure window
  figure;  imshow('rice.png');
  % In R2006b calling impoint returns the hggroup handle
  h = impoint(gca,100,200);
  % In 2006b iptgetapi returns a structure of function handles
  api = iptgetapi(h);
  % Add a new position callback to set the text string
  api.addNewPositionCallback(@newPos_Callback);
  % Construct boundary constraint function so we can't go outside the axes
  fcn = makeConstrainToRectFcn('impoint',get(gca,'XLim'),get(gca,'YLim'));
  api.setDragConstraintFcn(fcn);
  % Fire callback so we get initial text
  newPos_Callback(api.getPosition());
  function newPos_Callback(newPos)
    % Display the current point position in a text label
    api.setString(sprintf('(%1.0f,%1.0f)',newPos(1),newPos(2)));
  end %newPos_Callback
end %impointBehavior_R2006b

The code above, when run in R2006b, produces the desired behavior of displaying a text object containing the point coordinates that moves around with the point as it is dragged around the axes.
In R2009b, impoint is now a true MATLAB class using the new classdef syntax, so I wanted to update the existing code. Initially this appeared to be a straightforward translation of the code to make use of the new impoint class syntax. The first attempt to rewrite the code was:

function impointBehavior_R2009b
%IMPOINTBEHAVIOR_R2009B shows the undesirable behavior when
%using the setString method in R2009b.
  % Display the image in a figure window
  figure;  imshow('rice.png');
  h = impoint(gca,100,200);
  % Add a new position callback to set the text string
  h.addNewPositionCallback(@newPos_Callback);
  % Construct boundary constraint function so we can't go outside the axes
  fcn = makeConstrainToRectFcn('impoint',get(gca,'XLim'),get(gca,'YLim'));
  % Enforce boundary constraint function
  h.setPositionConstraintFcn(fcn);
  % Fire callback so we get initial text
  newPos_Callback(h.getPosition());
  function newPos_Callback(newPos)
    % Display the current point position in a text label
    h.setString(sprintf('(%1.0f,%1.0f)',newPos(1),newPos(2)))
  end %newPos_Callback
end %impointBehavior_R2009b

Unfortunately, when this code is run, dragging the mouse around the axes produces a trail of labels as shown below:

The solution

So how do we work around this bug to get to the behavior we want? One solution would be to rewrite the offending MATLAB code but this is somewhat risky in terms of maintainability and compatibility.
A more elegant solution is to subclass the impoint class and substitute the setString behavior we want. Looking at the impoint code we find that impoint is a subclass of imroi. In the imroi property declarations we see a number of undocumented properties that are protected. We can access these properties in a subclass but not outside the class. One of these undocumented properties is h_group which is an hggroup that contains the handle graphic objects that make up the impoint on the screen. The label, when created, becomes part of this hggroup with its Tag property set to ‘label’. When performing the setString method the behavior we want to see is that if the text object exists we want to update its String property. If it does not exist we want it to perform its existing functionality:

classdef impointtextupdate < impoint
%IMPOINTTEXTUPDATE subclasses impoint to override the setString
%method of impoint so that it does not create a new text object
%each time it is called.
  methods
    function obj = impointtextupdate(varargin)
      obj = obj@impoint(varargin{:});
    end %impointtextupdate
    function setString(obj,str)
      %override impoint setString
      %check to see if there is already a text label
      label = findobj(obj.h_group,'Type','text','Tag','label');
      if isempty(label)
        %do the default behavior
        setString@impoint(obj,str);
      else
        %update the existing tag
        set(label(1),'String',str);
      end %if
    end %setString
  end %methods
end %impointtextupdate

Substituting calls to impoint with the new impointupdatetext subclass now produces the desired effect as shown below:

Conclusions

This case illustrates a couple of points:

• Much of the existing code in the MATLAB toolboxes is being updated to the new object oriented syntax. This presents many opportunities to easily and elegantly modify the default behavior without modifying provided toolbox code In the example above we retain all the desirable behavior of impoint while overriding the undesirable behavior.
• Many of the properties and methods in the provided toolbox objects are hidden or protected and are undocumented. It takes some simple detective work to find these out through examining the code. MATLAB is very generous in providing much of the existing code openly. Open the functions and classes you use in the editor to really find out how they work. Over the years I've learned and adopted a lot of useful MATLAB programming patterns by examining the code in the various toolboxes (there are a few coding horrors as well!).

I hope to explore some other under-documented features of the IPT and other toolboxes in future posts.

The post Solving a MATLAB bug by subclassing appeared first on Undocumented Matlab.