Archive for the ‘Guest bloggers’ Category

JMI wrapper – remote MatlabControl

Wednesday, May 26th, 2010

Perhaps the most difficult aspect of Matlab-Java integration is JMI (Java-to-Matlab Integration) over a remote connection, meaning a Java program that communicates with a separate Matlab process. Once again I welcome guest blogger Joshua Kaplan, who concludes his series of JMI-related articles with the awaited holy grail on this topic.

Remote control of Matlab

Last week I demonstrated using matlabcontrol to call Matlab from Java from within the Matlab application. Today I will explain how to control Matlab from a remote Java session. We will create a small Java program that allows us to launch and connect to Matlab, then send it eval commands and receive the results. While this example will involve creating a dedicated user interface, matlabcontrol can be integrated into any existing Java program without requiring any user interface.

matlabcontrol was originally created for controlling Matlab, not for performing computations. If your exclusive concern is to perform Matlab computations and use the results in Java, then check the Matlab Builder JA toolbox, which is made by MathWorks and is officially supported. Unfortunately this toolbox is quite expensive and does not enable interaction with a running Matlab session (it uses the non-GUI Matlab engine, much as the compiler does). It is for this purpose that the open-source (free) matlabcontrol package was created.

Note that matlabcontrol opens a new running Matlab session and does not connect to an already-running session. Matlab commands can then be invoked either interactively (in Matlab’s Command Window) or remotely (from Java). Debugging an already-open Matlab session can be done with jdb over a dedicated port, using an altogether different mechanism than matlabcontrol – this will be discussed in a future post.

Matlab has documented support for a COM interface (Windows) and process pipes (Unix/Mac) that allow remote communication from external applications. Unfortunately Java does not natively support COM, which is where matlabcontrol helps using its RMI approach. Interested readers can also try using a Java/COM bridge (JACOB or JCOM) as an alternative that would have the added benefits of enabling communication with an existing Matlab session and of MathWorks’ documented support.

A simple RemoteExample

Today’s RemoteExample demo is too long to paste into this post; instead you can download the source code, or a jar file that contains both the pre-compiled classes and matlabcontrol. To run this jar on Windows or Mac OS X you only need to double click on it (if you are running on Linux I’m sure you know what to do…). If you wish to download and compile the source file, remember that you will need the matlabcontrol jar referenced in your Java classpath.

Let’s dive in: The file begins with the mainline followed by default sizes and status messages. The user interface is then built using standard Swing components – there’s nothing special here, just some panels, panes, text fields, buttons, etc.

The interesting part begins when the RemoteMatlabProxyFactory object is created:

RemoteMatlabProxyFactory factory = new RemoteMatlabProxyFactory();

This Matlab-proxy factory object is used when the user clicks the “Connect” button:

factory.requestProxy();

This creates a RemoteMatlabProxy object. RemoteMatlabProxys must be created by a RemoteMatlabProxyFactory and cannot be directly constructed. When requestProxy() is called, matlabcontrol launches Matlab and connects to it using RMI. When the connection is established, a MatlabConnectionListener added to the factory will be notified using its connectionEstablished(RemoteMatlabProxy proxy) callback method. The RemoteMatlabProxy object passed into this method is now connected to Matlab.

Connecting Java to Matlab using RMI

Connecting Java to Matlab using RMI

While this example only deals with communicating with a single Matlab session, matlabcontrol can handle multiple remote sessions. Whenever a new session is established, connectionEstablished(RemoteMatlabProxy proxy) is invoked on each connection listener. When a connection is lost due to Matlab closing, or in very rare cases Matlab encountering extremely severe errors, connectionLost(RemoteMatlabProxy proxy) is called. Calling methods on this proxy will lead to exceptions being thrown, as it can no longer communicate with Matlab. The proxy is passed because this information is useful when controlling multiple sessions of Matlab simultaneously.

When the “Invoke” button is pressed, the command and number of return arguments is sent to Matlab. If the number of return arguments is 0, the command will still execute but nothing will be returned. If the number is positive but less than the total number of return arguments, then only up to that number of arguments will be returned. If the number of return arguments specified exceeds the actual amount of arguments returned, a Java exception will be thrown. By default the fields are populated to return the result of “sqrt(5)“. Press “Invoke” to see what happens. Change the number of return arguments to 0, and click “Invoke” again. Now change the number to 2 and try once more.

Invoking Matlab commands from Java using JMI

Invoking Matlab commands from Java using JMI

When the Java program closes, it also exits Matlab. This is accomplished by adding a WindowListener to the program, which detects the Java closure event. It is important to call Matlab’s exit command as opposed to eval(“exit”), because all other proxy methods block (pause) until completion but in the case of exiting Matlab no signal will ever be sent by Matlab to indicate it has closed.

Parsing Matlab’s return values

Our eval commands are being sent using RemoteMatlabProxy’s returningEval(String command, int returnCount) method, whose return type is Object, because Matlab can return multiple return types. For example, the expression “sqrt(5)” will return an array of doubles, “pwd” will return a java.lang.String, and “whos” will return a complicated array of arrays with a variety of base types and Objects.

This is what occurs in Matlab R2009b, but not necessarily in past or future versions. You will have to experiment to find out what is being returned on your particular platform. The demo can help as it lists everything returned, including array contents. The demo contains the formatResult(Object result, int level) method which recursively goes through the object returned from Matlab and builds a description of what it contains. As discussed in my introductory post on matlabcontrol, Matlab functions will either return a base type, an array of base types, or a String. However, if we call a Java function inside the Matlab environment then any Java object might be returned. This isn’t actually that absurd of a situation; it might arise if you are trying to control the Matlab UI.

When returning Java objects from Matlab certain restrictions and limitations apply. First, the object must be Serializable because of the underlying use of RMI. In practice this isn’t a huge issue as a very large number of built-in Java classes are Serializable, and making your own classes Serializable is usually trivial.

The second limitation is that whatever class you send from the Matlab environment to your Java program must be defined in your Java program. For any standard built-in Java class this won’t cause issues. However, if you attempt to send over classes custom to Matlab then your Java program must have those classes in its classpath (in practice this means reference the jar file containing that class). For HG (or rather, UDD) classes, you can use the following Matlab function to create a Java class interface that can be used in your Java code to access the Matlab object, or access the jar file that contains that class as explained above:

% This will create a figure.java file in the current folder:
myClassHandle = classhandle(handle(gcf));
myClassHandle.createJavaInterface(myClassHandle.name, pwd);
 
% alternately, you can use myClassHandle.JavaInterfaces{1}
% = 'com.mathworks.hg.Figure' in this particular case
% i.e., in your java code import com.mathworks.hg.Figure

(UDD will be described in much more detail in a future article that is currently being prepared)

The third caveat is that you will be returned a copy of the Java object. This means that if you have a Java array in Matlab, send it to your Java program and then insert an object into the array, the array in Matlab will not be affected. However, you can then send that array back to Matlab, so in practice this does not cause significant issues. None of these restrictions are applicable to matlabcontrol for local sessions.

Conclusion

Today we have shown how a dedicated Matlab session can be started from a Java application, which then communicates with that Matlab session using the matlabcontrol package. This concludes my series on matlabcontrol. Please check out the matlabcontrol website for more information, examples and updates. If you use matlabcontrol, filling out this survey so that I can improve it, would be greatly appreciated.

Since the past few articles were heavy on Matlab-Java topics, the next few articles will be devoted to undocumented pure-Matlab tips and tricks, starting next week with the topic of how to customize the default menubar and toolbar actions.

JMI wrapper – local MatlabControl part 2

Wednesday, May 19th, 2010

Once again I welcome guest blogger Joshua Kaplan for his continued series on MatlabControl – the Java-to-Matlab Interface (JMI) wrapper

Last week I introduced a Java package called matlabcontrol which can be used to access Matlab from Java.

Let’s now put matlabcontrol to work from Java. Below is a very simple Java class called LocalExample which uses Swing to create a window (JFrame), a text field (JTextField), and a button (JButton). When the button is pressed, the text in the field will be evaluated in Matlab using LocalMatlabProxy.eval(…), which was explained in the previous post.

import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import java.awt.event.WindowAdapter;
import java.awt.event.WindowEvent;
 
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
 
import javax.swing.JButton;
import javax.swing.JFrame;
import javax.swing.JPanel;
import javax.swing.JTextField;
 
import matlabcontrol.LocalMatlabProxy;
import matlabcontrol.MatlabInvocationException;
 
/**
 * A simple demo of some of matlabcontrol's functionality.
 * 
 * @author Joshua Kaplan
 */
public class LocalExample extends JFrame
{
  /**
   * Constructs this example and displays it.
   */
  public LocalExample()
  {
    //Window title
    super("Local Session Example");
 
    //Panel to hold field and button
    JPanel panel = new JPanel();
    this.add(panel);
 
    //Input field
    final JTextField inputField = new JTextField();
    inputField.setColumns(15);
    panel.add(inputField);
 
    //Eval button
    JButton evalButton = new JButton("eval");
    panel.add(evalButton);
 
    //Eval runnable, to execute in separate (non-EDT) thread
    final ExecutorService executor = Executors.newSingleThreadExecutor();
    final Runnable evalRunnable = new Runnable()
    {
      public void run()
      {
        try
        {
          LocalMatlabProxy.eval(inputField.getText());
        }
        catch (MatlabInvocationException exc) { }
      }
    };
 
    //Eval action event for button and field
    ActionListener evalAction = new ActionListener()
    {
      public void actionPerformed(ActionEvent e)
      {
        //This uses the EDT thread, which may hang Matlab...
        //LocalMatlabProxy.eval(inputField.getText());
 
        //Execute runnable on a separate (non-EDT) thread
        executor.execute(evalRunnable);
      }
    };
    evalButton.addActionListener(evalAction);
    inputField.addActionListener(evalAction);
 
    //On closing, release resources of this frame
    this.addWindowListener(new WindowAdapter()
    {
      public void windowClosing(WindowEvent e)
      {
        LocalExample.this.dispose();
      }
    });
 
    //Display
    this.pack();
    this.setResizable(false);
    this.setVisible(true);
  }
}

We can either copy and compile the above code (remember to import the matlabcontrol JAR file in your compiler; four class files will be created), or download the already-compiled code here. Note that the already-compiled classes were compiled using JDK 1.6, so if you have Matlab R2007a (7.4) or earlier you will need to compile using an earlier Java compiler.

We now need to tell Matlab where to find both the matlabcontrol JAR file, and our LocalExample class files. We can either add the files to the static java classpath (by editing the classpath.txt file), or add them only to the current Matlab session’s dynamic classpath:

% Add Java files to current Matlab session's dynamic classpath
javaaddpath LocalExample.zip
javaaddpath matlabcontrol-3.01.jar

To run the Java program, simply type LocalExample in the Matlab Command Window – A small Java window will appear and JMI will evaluate any expression typed into it:

>> LocalExample
ans =
LocalExample[frame0,0,0,202x67,title=Local Session Example,...]

Java window calling Matlab via JMI

Java window calling Matlab via JMI

a =
          1.77245385090552

When we called LocalMatlabControl.eval(…) from the Command Window last week, what occurred behind the scenes was actually quite different from what happens when we press the “eval” button in the Java program. This is because in the Command Window everything executes in Matlab’s single main thread. When we pressed “eval”, the code executed from the Event Dispatch Thread (EDT), which is a separate thread.

EDT is used extensively by Matlab when accessing graphical components such as a figure window, uicontrols or plots. When calling a function from JMI, the calling thread always blocks (pauses) until Matlab is done completing whatever has been asked of it. So, if we called JMI/matlabcontrol from the EDT and Matlab needed to use the EDT, then everything will lock up. To fix this potential problem, when you press the “eval” button the command is sent off to a separate thread which can block without preventing Matlab from doing its work. Future versions of matlabcontrol will try to simplify this process further. matlabcontrol over a remote connection, the topic of my next post, does not suffer from this complication because Matlab and your program are not sharing the same EDT or even the same Java runtime process (JVM).

While the above Java example is quite simple, using a combination of all of the methods described throughout my previous post, a much more sophisticated program can be created. To explore the methods in more detail you can use the demo available here. The demo uses a remote connection to Matlab, but the available methods are the same. In my next and final article on this subject, I will explore controlling Matlab using matlabcontrol over a remote connection.

JMI wrapper – local MatlabControl part 1

Wednesday, May 12th, 2010

Once again I would like to welcome guest blogger Joshua Kaplan, who continues his series of JMI-related articles

Local and remote MatlabControl

Several weeks ago, I discussed the undocumented world of JMI (Java-to-Matlab Interface), that enables calling Matlab from Java. Today I will discuss matlabcontrol, an open-source wrapper I have written for JMI, that is both documented and user-friendly.

matlabcontrol supports calling Matlab in two different ways: local control where the Java code is launched from Matlab, and remote control where the Java code launches Matlab. Today we shall explore matlabcontrol’s local control methods; in my next post we’ll create a simple Java program that uses matlabcontrol for local control; a later post will discuss the remote control path. These posts assume a medium to high level of Java experience.

matlabcontrol is a collection of Java classes, bundled together in a downloadable jar file, which is essentially just a zip file with a different file extension. As of this post the most recent version is matlabcontrol-3.01.jar. Note down where you’ve downloaded the jar file – you’ll need to use this information shortly.

For local control we’ll interact with the classes LocalMatlabProxy and MatlabInvocationException. LocalMatlabProxy contains all the methods required for calling Matlab; instances of MatlabInvocationException will be thrown when a problem occurs while attempting to control Matlab.

To tell Matlab where matlabcontrol-3.01.jar is, add the jar file path to Matlab’s dynamic (via the built-in javaaddpath function) or static (via edit(‘classpath.txt’)) Java classpath. You will need to restart Matlab if you have modified the static classpath, but it has the benefit of working better than the dynamic classpath in some situations.

Matlab now knows where to find the Java class files in matlabcontrol. To save some typing later on, type the following in the Matlab Command Window (or in your JMI-empowered Matlab application):

import matlabcontrol.*

LocalMatlabProxy methods

LocalMatlabProxy is easy to use. All of its methods are static meaning they can be called without needing to assign LocalMatlabProxy to a variable. The methods are:

void exit()
java.lang.Object getVariable(java.lang.String variableName)
void setVariable(java.lang.String variableName, java.lang.Object value)
void eval(java.lang.String command)
java.lang.Object returningEval(java.lang.String command, int returnCount)
void feval(java.lang.String functionName, java.lang.Object[] args)
java.lang.Object returningFeval(java.lang.String functionName, java.lang.Object[] args)
java.lang.Object returningFeval(java.lang.String functionName, java.lang.Object[] args, int returnCount)
void setEchoEval(boolean echoFlag)

Detailed javadocs exist for all these methods. Here is an overview:

  • exit() is as straightforward as it sounds: it will exit Matlab. While it is possible to programmatically exit Matlab by other means, they may be unreliable. So, to exit Matlab from Java:
     LocalMatlabProxy.exit();
  • Setting and getting variables can be done using the getVariable(…) and setVariable(…) methods. These methods will auto-convert between Java and Matlab types where applicable.

       Using getVariable(…):

    • Java types in the Matlab environment retrieved will be returned as Java types.
    • Matlab types will be converted into Java types.

       Using setVariable(…):

    • Java types will be converted into Matlab types if they can. The rules are outlined here. Java Strings are converted to Matlab char arrays. Additionally, arrays of one of those Java types are converted to arrays of the corresponding Matlab type.

       Using these methods is fairly intuitive:

     >> LocalMatlabProxy.setVariable('x',5)
     
    >> LocalMatlabProxy.getVariable('x') 
    ans = 
         5

       Getting and setting basic types (numbers, strings and Java objects) is quite reliable and consistent. It gets complicated when passing in an array (particularly multidimensional) from Java using setVariable(…), or getting a Matlab struct or cell array using getVariable(…). The type conversion in such cases is unpredictable, and may be inconsistent across Matlab versions. In such cases you are best off building a Java object with Matlab code and then getting the Java object you created.

  • The eval() and feval() methods were described in detail in my previous post. The functions will return the result, if any, as a Java object. Due to the way the underlying JMI operates, it is necessary to know in advance the number of expected return arguments. Matlab built-in nargout function reveals this number. Some functions (e.g., feval) return a variable number of arguments, in which case nargout returns -1. For instance:
     >> nargout sqrt 
    ans = 
         1
     
    >> nargout feval
    ans = 
         -1
  • LocalMatlabProxy’s returningFeval(functionName, args) method uses the nargout information to determine the number of returned arguments and provide it to JMI. It will likely not function as expected if the function specified by functionName returns a variable number of arguments. In such a case, call returningFeval(…) with a third input argument that specifies the expected number of returned arguments. Since an eval() function can evaluate anything, just as if it were typed in the Command Window, there is no reliable way to determine what will be returned. All of this said, in most situations returningEval(…) can be used with a return count of 1, and the returningFeval(…) that automatically determines the return count will operate as expected.

Some simple usage examples

Let’s perform some of the same simple square root operations we did in the pure-JMI article, this time using matlabcontrol. First we’ll take the square root of 5, assigning the result to the Matlab variable y (note that we are calling Matlab from Java, that is called from within Matlab):

>> LocalMatlabProxy.eval('sqrt(5)') 
ans = 
    2.2361 
 
>> y = LocalMatlabProxy.returningEval('sqrt(5)',1) 
y = 
    2.2361 
 
>> LocalMatlabProxy.feval('sqrt',5)
 
>> y = LocalMatlabProxy.returningFeval('sqrt',5) 
y = 
    2.2361 
 
>> y = LocalMatlabProxy.returningFeval('sqrt',5,1) 
y = 
    2.2361

In this situation there is no major difference between using eval() or feval() in the above situation. However, if instead of taking the square root of 5 we want to take the square root of a variable, then eval() is our only option.

>> a = 5 
a = 
     5 
 
>> LocalMatlabProxy.eval('sqrt(a)') 
ans = 
    2.2361 
 
>> y = LocalMatlabProxy.returningEval('sqrt(a)',1) 
y = 
    2.2361 
 
>> LocalMatlabProxy.feval('sqrt','a')
??? Undefined function or method 'sqrt' for input arguments of type 'char'.
??? Java exception occurred:
matlabcontrol.MatlabInvocationException: Method could not return a value because of an internal Matlab exception
      at matlabcontrol.JMIWrapper.returningFeval(JMIWrapper.java:256)
      at matlabcontrol.JMIWrapper.feval(JMIWrapper.java:210)
      at matlabcontrol.LocalMatlabProxy.feval(LocalMatlabProxy.java:132)
Caused by: com.mathworks.jmi.MatlabException: Undefined function or method 'sqrt' for input arguments of type 'char'.
      at com.mathworks.jmi.NativeMatlab.SendMatlabMessage(Native Method)
      at com.mathworks.jmi.NativeMatlab.sendMatlabMessage(NativeMatlab.java:212)
      at com.mathworks.jmi.MatlabLooper.sendMatlabMessage(MatlabLooper.java:121)
      at com.mathworks.jmi.Matlab.mtFevalConsoleOutput(Matlab.java:1511)
      at matlabcontrol.JMIWrapper.returningFeval(JMIWrapper.java:252)
      ... 2 more

The automatic Matlab/Java type conversions discussed above are equally applicable to eval() and feval(). feval() automatically converted the argument ‘a’ into a Matlab char, instead of considering it as a Matlab variable. As seen above, the feval() invocation fails with a Java MatlabInvocationException. So, the only way to interact with Matlab variables is via eval() methods; feval() will not work.

Lastly there is the setEchoEval(echoFlag) method: If this method is called with a true argument, then all Java to Matlab calls will be logged in a dedicated window. This can be very helpful for debugging.

In my next post we shall put together this knowledge to create a small Java program that uses matlabcontrol to interact with Matlab.

Advanced JIDE Property Grids

Wednesday, April 21st, 2010

Once again, I would like to welcome guest blogger Levente Hunyadi.

Non-standard property renderers and editors

Last week, I discussed JIDE’s property table and showed how we can add custom properties and present this table in our Matlab GUI. Today, I will extend the previous week’s example to include more sophisticated renderers and editors.

Cell renderers and editors are the backbone of JTable implementations, JIDE’s property grid included. Each property is associated with a type, and a renderer and an editor may be registered for a type. The cell renderer controls how the property value is displayed, while the editor determines how it is edited. For example, flags (Java Booleans) are often both rendered and edited using a checkbox, but can also use a text renderer with a combo-box editor. PropertyTable automatically assigns a default renderer and editor to each property, based on its type: Flags are assigned a combo-box editor of true/false values, and similarly for other types.

Let us now modify the preassigned editor. First, let’s set a checkbox editor (BooleanCheckBoxCellEditor) for flags and a spinner for numbers:

% Initialize JIDE's usage within Matlab
com.mathworks.mwswing.MJUtilities.initJIDE;
 
% Prepare the properties list:
% First two logical values (flags)
list = java.util.ArrayList();
prop1 = com.jidesoft.grid.DefaultProperty();
prop1.setName('mylogical');
prop1.setType(javaclass('logical'));
prop1.setValue(true);
list.add(prop1);
 
prop2 = com.jidesoft.grid.DefaultProperty();
prop2.setName('mycheckbox');
prop2.setType(javaclass('logical'));
prop2.setValue(true);
prop2.setEditorContext(com.jidesoft.grid.BooleanCheckBoxCellEditor.CONTEXT);
list.add(prop2);
 
% Now integers (note the different way to set property values):
prop3 = com.jidesoft.grid.DefaultProperty();
javatype = javaclass('int32');
set(prop3,'Name','myinteger','Type',javatype,'Value',int32(1));
list.add(prop3);
 
prop4 = com.jidesoft.grid.DefaultProperty();
set(prop4,'Name','myspinner','Type',javatype,'Value',int32(1));
set(prop4,'EditorContext',com.jidesoft.grid.SpinnerCellEditor.CONTEXT);
list.add(prop4);
 
% Prepare a properties table containing the list
model = com.jidesoft.grid.PropertyTableModel(list);
model.expandAll();
grid = com.jidesoft.grid.PropertyTable(model);
pane = com.jidesoft.grid.PropertyPane(grid);
 
% Display the properties pane onscreen
hFig = figure;
panel = uipanel(hFig);
javacomponent(pane, [0 0 200 200], panel);

A property grid with checkbox and spinner controls

A property grid with checkbox and spinner controls

Notice how the EditorContext is used to specify a non-standard renderer/editor for myspinner and mycheckbox: The mylogical flag displays as a string label, while mycheckbox displays as a checkbox; myinteger uses a regular integer editor that accepts whole numbers, while myspinner uses a spinner control to modify the value.

Note that instead of creating an entirely new properties list and table, we could have run last week’s example, modified list and then simply called model.refresh() to update the display.

Also note that Matlab types are automatically converted to Java types, but we must be careful that the results of the conversion should match our setType declaration. The logical value true converts to java.lang.Boolean, but 1 by default would be a double, which is the standard numeric type in Matlab. The int32 wrapper is needed to force a conversion to a java.lang.Integer.

Spinners with indefinite value bounds are seldom useful. The following shows how to register a new editor to restrict values to a fixed range. Remember to unregister the editor when it is no longer used:

javatype = javaclass('int32');
value  = int32(0);
minVal = int32(-2);
maxVal = int32(5);
step   = int32(1);
spinner = javax.swing.SpinnerNumberModel(value, minVal, maxVal, step);
editor = com.jidesoft.grid.SpinnerCellEditor(spinner);
context = com.jidesoft.grid.EditorContext('spinnereditor');
com.jidesoft.grid.CellEditorManager.registerEditor(javatype, editor, context);
 
prop = com.jidesoft.grid.DefaultProperty();
set(prop, 'Name','myspinner', 'Type',javatype, ...
          'Value',int32(1), 'EditorContext',context);
 
% [do something useful here...]
 
com.jidesoft.grid.CellEditorManager.unregisterEditor(javatype, context);

The principle is the same for combo-boxes:

javatype = javaclass('char', 1);
options = {'spring', 'summer', 'fall', 'winter'};
editor = com.jidesoft.grid.ListComboBoxCellEditor(options);
context = com.jidesoft.grid.EditorContext('comboboxeditor');
com.jidesoft.grid.CellEditorManager.registerEditor(javatype, editor, context);
 
prop = com.jidesoft.grid.DefaultProperty();
set(prop, 'Name','season', 'Type',javatype, ...
          'Value','spring', 'EditorContext',context);
 
% [do something useful here...]
 
com.jidesoft.grid.CellEditorManager.unregisterEditor(javatype, context);

A property grid with a combobox control

A property grid with a combobox control

Nested properties

Properties can act as a parent node for other properties. A typical example is an object’s dimensions: a parent node value may be edited as a 2-by-1 matrix, but width and height may also be exposed individually. Nested properties are created as regular properties. However, rather than adding them directly to a PropertyTableModel, they are added under a Property instance using its addChild method:

propdimensions = com.jidesoft.grid.DefaultProperty();
propdimensions.setName('dimensions');
propdimensions.setEditable(false);
 
propwidth = com.jidesoft.grid.DefaultProperty();
propwidth.setName('width');
propwidth.setType(javaclass('int32'));
propwidth.setValue(int32(100));
propdimensions.addChild(propwidth);
 
propheight = com.jidesoft.grid.DefaultProperty();
propheight.setName('height');
propheight.setType(javaclass('int32'));
propheight.setValue(int32(100));
propdimensions.addChild(propheight);

A property grid with nested property

A property grid with nested property

PropertyTableModel accesses properties in a hierarchical naming scheme. This means that the parts of nested properties are separated with a dot (.). In the above example, these two fully-qualified names are dimensions.width and dimensions.height.

Trapping property change events

Sometimes it is desirable to subscribe to the PropertyChange event. This event is fired by PropertyTableModel whenever any property value is updated. To expose Java events to Matlab, we use the two-parameter form of the handle function with the optional CallbackProperties parameter.

hModel = handle(model, 'CallbackProperties');
set(hModel, 'PropertyChangeCallback', @callback_onPropertyChange);

The callback function receives two input arguments: The first is the PropertyTableModel object that fired the event, the second is a PropertyChangeEvent object with properties PropertyName, OldValue and NewValue. The PropertyTableModel’s getProperty(PropertyName) method may be used to fetch the Property instance that has changed.

Callbacks enable property value validation: OldValue can be used to restore the original property value, if NewValue fails to meet some criteria that cannot be programmed into the cell editor. We may, for instance, set the property type to a string; then, in our callback function, use str2num as a validator to try to convert NewValue to a numeric matrix. If the conversion fails, we restore the OldValue:

function callback_onPropertyChange(model, event)
   string = event.getNewValue();
   [value, isvalid] = str2num(string); %#ok
   prop = model.getProperty(event.getPropertyName());
   if isvalid  % standardize value entered
      string = mat2str(value);
   else  % restore previous value
      string = event.getOldValue();
   end
   prop.setValue(string);
   model.refresh();  % refresh value onscreen

The JIDE packages that are pre-bundled in Matlab contain numerous other useful classes. Some of these will be described in future articles.

JIDE Property Grids

Wednesday, April 21st, 2010

I would like to welcome guest blogger Levente Hunyadi.

Matlab’s property inspector

We often wish to edit properties of heterogeneous objects using a common interface. Matlab’s property inspector, invoked with the built-in inspect function, answers this need. The inspector is based on a two-column table of property names and values. Properties and their values are populated automatically, and the user can edit values in-place. The inspector enables property categorization, sub-categorization and sorting, which help users find and modify properties easily. For each property, the inspector displays a matching edit control: editbox/combobox/checkbox etc. This simplifies property value editing and prevents illegal value entry. Matlab’s GUI builder, GUIDE, uses the inspector to let users edit GUI properties such as position, color etc. It is also used by other tools such as the Plot Editor.

Matlab's built-in property inspector

Matlab's built-in property inspector

The Matlab inspector can be embedded, with not-too-much effort, within Matlab GUI applications. Examples of this can be found in the FindJObj and UIInspect utilities.

FindJObj - embedded property inspector

FindJObj - embedded property inspector

Unfortunately, Matlab’s property inspector is limited to Handle Graphics, Java and COM objects, and cannot be used for structures or user-defined Matlab classes. We shall see below how to set up our own property grid, populate it with data, and subscribe to property change events. This is a rather procedural approach. It is usually more convenient to use a declarative approach in which a structure or Matlab class is passed to a function that automatically discovers its properties and their meta-information. The Property Grid utility at Matlab File Exchange provides these services.

A simple property grid

Matlab’s property inspector is based on a property grid control by JIDE Software. JIDE Grids is a collection of components that extend the standard Java Swing JTable component, and is included in each Matlab installation (/java/jarext/jide/jide-grids.jar under the Matlab root). In particular, JIDE Grids includes the PropertyTable class, which is a fully customizable property grid component. You can find further details on JIDE Grids in the Developer Guide and the Javadoc documentation.

There are several related classes associated with the PropertyTable class. First, a PropertyTableModel encapsulates all properties that are visualized in the property grid. Each property derives from the Property abstract class, which features some common actions to properties, most notably to get and set property value. DefaultProperty is a default concrete subclass of Property. Finally, PropertyPane decorates a property grid with icons for changing category view to alphabetically sorted view as well as expanding and collapsing categories, and a description text box at the bottom that can be shown or hidden.

Here are the DefaultProperty fields and their respective roles:

Field Role
Name Interal property name, not necessarily displayed, used as a key to identify the property.
DisplayName A short property name shown in the left column of the property grid.
Description A concise description of the property, shown at the bottom of the property pane, below the grid.
Type The Java type associated with the property, used to invoke the appropriate renderer or editor.
EditorContext An editor context object. If set, both the type and the context are used to look up the renderer or editor to use. This lets, for instance, one flag value to display as a true/false label, while another as a checkbox.
Category A string specifying the property’s category, for grouping purposes.
Editable Specifies whether the property value is modifiable or read-only.
Value The current property value, as a Java object.

Just as with any Java object, these fields may either be accessed with the Java get/set semantics (e.g. getName() or setName(name)), or the Matlab get/set semantics (e.g. get(prop,’Name’) or set(prop,’Name’,name)). When using the Matlab syntax, remember to wrap the Java object in a handle() call, to prevent a memory leak.

To use a property grid in Matlab, first construct a set of DefaultProperty objects. For each object, set at least the name, type and initial value. Next, add the properties to a table model. Finally, construct a property grid with the given table model and encapsulate in a property pane:

% Initialize JIDE's usage within Matlab
com.mathworks.mwswing.MJUtilities.initJIDE;
 
% Prepare the properties list
list = java.util.ArrayList();
prop1 = com.jidesoft.grid.DefaultProperty();
prop1.setName('stringProp');
prop1.setType(javaclass('char',1));
prop1.setValue('initial value');
prop1.setCategory('My Category');
prop1.setDisplayName('Editable string property');
prop1.setDescription('A concise description for my property.');
prop1.setEditable(true);
list.add(prop1);
 
prop2 = com.jidesoft.grid.DefaultProperty();
prop2.setName('flagProp');
prop2.setType(javaclass('logical'));
prop2.setValue(true);
prop2.setCategory('My Category');
prop2.setDisplayName('Read-only flag property');
prop2.setEditable(false);
list.add(prop2);
 
% Prepare a properties table containing the list
model = com.jidesoft.grid.PropertyTableModel(list);
model.expandAll();
grid = com.jidesoft.grid.PropertyTable(model);
pane = com.jidesoft.grid.PropertyPane(grid);
 
% Display the properties pane onscreen
hFig = figure;
panel = uipanel(hFig);
javacomponent(pane, [0 0 200 200], panel);
 
% Wait for figure window to close & display the prop value
uiwait(hFig);
disp(prop1.getValue());

Here, com.mathworks.mwswing.MJUtilities.initJIDE is called to initialize JIDE’s usage within Matlab. Without this call, we may see a JIDE warning message in some Matlab releases. We only need to initJIDE once per Matlab session, although there is no harm in repeated calls.

javaclass is a function (included in the Property Grid utility, or directly downloadable from here) that returns a Java class for the corresponding Matlab type with the given dimension: javaclass(‘logical’) or javaclass(‘logical’,0) (a single logical flag value) returns a java.lang.Boolean class; javaclass(‘char’,1) (a character array) returns a java.lang.String class; javaclass(‘double’,2) (a matrix of double-precision floating point values) returns double[][].

javacomponent is the undocumented built-in Matlab function that adds Java Swing components to a Matlab figure, using the given dimensions and parent handle. When the user closes the figure, prop.getValue() fetches and displays the new property value.

A simple user-defined property grid

A simple user-defined property grid

Next week’s article will show how to add more complex renderers and editors (display the flag value as a checkbox for example), define nested properties, and subscribe to property value change events. So stay tuned…

JMI – Java-to-Matlab Interface

Wednesday, April 14th, 2010

I would like to welcome guest writer Joshua Kaplan, an expert in one of the darkest undocumented corners of Matlab – that of the Java-to-Matlab interface (JMI). Today, Joshua will introduce JMI and its core functionality. Later articles will explore additional aspects of this complex technology.

As you’ve seen many times on this blog before, Matlab can easily call Java. This article explores the dark undocumented territory known as JMI (Java Matlab Interface) that enables calling Matlab from Java. This interface takes the form of a file called jmi.jar that comes with every copy of Matlab released in the past decade. JAR files are essentially a zip file containing Java class files. In this case, several Java classes in jmi.jar enable interaction with Matlab. This post will discuss the most important class: com.mathworks.jmi.Matlab.

Matlab’s dynamic evaluation functions

JMI easily allows calling two built-in Matlab functions: eval and feval. Essentially, eval evaluates any string typed into Matlab’s Command Window, and feval allows calling any function by name and passing in arguments. For example:

>> sqrt(5)
ans =
    2.2361
>> eval('sqrt(5)')
ans =
    2.2361
>> feval('sqrt',5)
ans =
    2.2361

The first approach takes the square root of 5 normally by directly calling Matlab’s square root function sqrt. JMI does not enable this direct-invocation approach. Instead, JMI uses the second approach, where eval mimics the first call by evaluating the entire expression inside single quotes. The third option, also used by JMI, is to use feval where the function and arguments are specified separately. While in the above example calling eval and feval is very similar, there are differences. For instance, assignment can be done as x=5 or eval(‘x=5’) but there is no feval equivalent. There are also other eval relatives, such as hgfeval, evalc and evalin, which will not be explained today.

Before we explore com.mathworks.jmi.Matlab, it is important to note a few things: Everything that follows is based on many hours of experimentation and online research and so while I am more or less certain of what I am about to explain, it could be deficient or incorrect in many respects. In fact, this area is so undocumented that an article written in 2002 by one of The MathWorks employees and published in the official newsletter, has been removed from their website a year or two ago.

Secondly, this post is written to satisfy people’s curiosities regarding JMI. If you actually wish to call Matlab from Java, I strongly suggest you use MatlabControl, a user-friendly Java library I have written that wraps JMI. There are many complications that arise with threading, method completion blocking, virtual machine restrictions, and other domains that prevent using JMI directly to be practical and reliable.

com.mathworks.jmi.Matlab class and its mtEval() method

With all of that said, let’s dive in! In the Matlab Command Window type:

methodsview('com.mathworks.jmi.Matlab')

You will see all of the Matlab class’s numerous methods. Many of the methods have extremely similar names; many others have the same names and just different parameters. In order to call eval and feval we are going to use two of Matlab’s methods:

public static Object mtEval(String command, int returnCount)
public static Object mtFeval(String functionName, Object[] args, int returnCount)

Since Matlab can call Java, we can experiment with these methods from Matlab. That’s right, we’re about to use Matlab to call Java to call Matlab!

First, let’s import the Java package that contains the Matlab class, to reduce typing:

import com.mathworks.jmi.*

Now let’s take the square root of 5 like we did above, but this time from Java. Using JMI’s eval-equivalent:

Matlab.mtEval('sqrt(5)',1)

Here, ’sqrt(5)’ is what will be passed to eval, and 1 signifies that Matlab should expect one value to be returned. It is important that the second argument be accurate. If instead the call had been:

Matlab.mtEval('sqrt(5)',0)

Then an empty string (”) will be returned. If instead, 2 or more were passed in:

Matlab.mtEval('sqrt(5)',2)

Then a Java exception like the one below will occur:

??? Java exception occurred:
com.mathworks.jmi.MatlabException: Error using ==> sqrt
Too many output arguments.
    at com.mathworks.jmi.NativeMatlab.SendMatlabMessage(Native Method)
    at com.mathworks.jmi.NativeMatlab.sendMatlabMessage(NativeMatlab.java:212)
    at com.mathworks.jmi.MatlabLooper.sendMatlabMessage(MatlabLooper.java:121)
    at com.mathworks.jmi.Matlab.mtFeval(Matlab.java:1478)
    at com.mathworks.jmi.Matlab.mtEval(Matlab.java:1439)

Looking at this exception’s stack trace we notice that mtEval() is actually internally calling mtFeval().

com.mathworks.jmi.Matlab class’s mtFeval() method

Now to perform the square root using JMI’s feval-equivalent:

Matlab.mtFeval('sqrt',5,1)

Here, ’sqrt’ is the name of the Matlab function to be called, 5 is the argument to the function, and 1 is the expected number of return values. Again, the number of return values. If this number is specified as 0 instead of 1, the function call will still succeed, although not all of the results will necessarily be returned. The second mtFeval() argument, which specifies the arguments to the invoked Matlab function, can take any number of arguments as an array. Therefore the following is acceptable:

Matlab.mtFeval('sqrt',[5 3],1)

This will return an array containing both of their square roots. Note that although two vales are returned, they are considered as only 1 since it is a single array that is returned.

Multiple Matlab arguments can be specified in mtFeval() using a cell array. For example, consider the following equivalent formats (note the different returned array orientation):

>> min(1:4,2)
ans =
     1     2     2     2
>> Matlab.mtFeval('min',{1:4,2},1)
ans =
     1
     2
     2
     2

As we observed above, mtEval() is really just calling mtFeval(). This works because eval is a function, so feval can call it. An illustration:

Matlab.mtFeval('eval','sqrt(5)',1)

com.mathworks.jmi.Matlab class’s mtFevalConsoleOutput() method

Both mtFeval() and mtEval() have allowed us to interact with Matlab, but the effects are not shown in the Command Window. There is a method that will allow us to do this:

public static Object mtFevalConsoleOutput(String functionName, Object[] args, int returnCount)

mtFevalConsoleOutput() is just liked the mtFeval() command except that its effects will be shown. For instance:

>> Matlab.mtFeval('disp','hi',0);  % no visible output
>> Matlab.mtFevalConsoleOutput('disp','hi',0);
hi

There is no equivalent mtEvalConsoleOutput() method, but that’s not a problem because we have seen that eval can be accomplished using feval:

>> Matlab.mtFevalConsoleOutput('eval','x=5',0);
x =
     5

Other methods in com.mathworks.jmi.Matlab

There are many more eval and feval methods in the Matlab class. Most of these methods’ names begin with eval or feval instead of mtEval and mtFeval. Many of these methods are asynchronous, which means their effect on Matlab can occur after the method call returns. This is often problematic because if one method call creates a variable which is then used by the next call, there is no guarantee that the first call has completed (or even begun) by the time the second call tries to use the new variable. Unlike mtEval() and mtFeval(), these methods are not static, meaning we must have an instance of the Java class Matlab:

>> proxy = Matlab
proxy =
com.mathworks.jmi.Matlab@1faf67f0

Using this instance we will attempt to assign a variable and then retrieve it into a different variable. The result will be a Java exception indicating that ‘a’ does not currently exist:

>> proxy.evalConsoleOutput('a=5'); b = proxy.mtEval('a',1)
??? Java exception occurred:
com.mathworks.jmi.MatlabException: Error using ==> eval
Undefined function or variable 'a'.
    at com.mathworks.jmi.NativeMatlab.SendMatlabMessage(Native Method)
    at com.mathworks.jmi.NativeMatlab.sendMatlabMessage(NativeMatlab.java:212)
    at com.mathworks.jmi.MatlabLooper.sendMatlabMessage(MatlabLooper.java:121)
    at com.mathworks.jmi.Matlab.mtFeval(Matlab.java:1478)
    at com.mathworks.jmi.Matlab.mtEval(Matlab.java:1439)
a =
     5

If you run the above code you are not guaranteed to get that exception because of the nature of asynchronous method calls. However, this inherent unpredictability makes it difficult to perform almost any sequential action. Therefore, it is best to stick to mtEval, mtFeval, and mtFevalConsoleOutput, where this type of exception will be very remote. (They can still occur, about 1 in 100 times, as to why – I’d love to know as well.)

Two particular methods that may come in handy are mtSet() and mtGet(), which are the Java proxies for the Matlab set and get functions – they accept a Matlab handle (a double value) and a property name (a string) and either set the value or return it. Like Matlab, they also accept an array of property names. This can be used to update Matlab HG handles from within Java code, without needing to pass through an intermediary Matlab eval function:

>> Matlab.mtSet(gcf,'Color','b')
>> Matlab.mtGet(gcf,'Color')
ans =
     0
     0
     1
>> Matlab.mtGet(gcf,{'Color','Name'})
ans =
java.lang.Object[]:
    [3x1 double]
    'My figure'

Summary

With just eval and feval, an enormous amount of Matlab’s functionality can be accessed from Java. For instance, this allows for creating sophisticated Java GUIs with Swing and then being able to call Matlab code when the user clicks a button or moves a slider.

My next post will be on using MatlabControl to control Matlab from Java from within Matlab. The following post will discuss doing the same, but from a Java program launched outside of Matlab.

Matlab and the Event Dispatch Thread (EDT)

Wednesday, March 10th, 2010

Once again I welcome guest blogger Matt Whitaker, with the long awaited EDT article.

Java Swing’s Event Dispatch Thread (EDT)
or: why does my GUI foul up?

Matlab for the most part is a single threaded environment. That is, all commands are executed sequentially along a single execution thread. The main exception to this are the Handle Graphics (GUI) components whose operations execute on the Java Event Dispatch Thread (EDT). EDT effects are reflected even in mundane Matlab GUI operations.

If we execute the code below we will probably see nothing until the loop completes and the figure appears with the text label showing ‘10000′:

h = figure;
txt = uicontrol('Parent',h, 'Style','text', 'String','1');
for n = 1:10000
    set(txt,'String',int2str(n))
end %for

By adding a couple of drawnow commands we get the figure and text label to render and then we see the count progress to 10000.

h = figure;
txt = uicontrol('Parent',h, 'Style','text', 'String','1');
drawnow;
for n = 1:10000
    set(txt,'String',int2str(n));
    drawnow;
end %for

The drawnow function allows the EDT queue to be flushed and the pending graphics operations to be evaluated. This will also happen with pause and several other commands.

If we want to use Swing (or AWT) components in our user interfaces we need to take this multi-threaded environment into account. The Swing toolkit designers decided to make all the Swing components thread un-safe in order to decrease their complexity. As a consequence, all access to Swing components should be done from the event dispatch thread (EDT), to ensure the operations are executed sequentially, at the exact order in which they were dispatched. Any action on a Swing component done on another thread (Matlab’s main processing thread in our case) risks a race-condition or deadlock with the EDT, which could (and often does) result in weird, non-deterministic and non-repetitive behavior – all of which should be avoided in any application which should behave in a precisely deterministic manner.

In Java, the usual pattern to accomplish EDT dispatching is to create a Runnable object, encapsulate the GUI code in the run method of the Runnable object, then pass the Runnable object to the static EventQueue.invokeLater (or EventQueue.invokeAndWait if we need to block operations to get a return value) method.

Runnable runnable = new Runnable()
{
    public void run()
    {
        //GUI Code here
    }
}
 
EventQueue.invokeLater(runnable);

There are several functions in Matlab that implement this programming pattern for us: javaObjectEDT, javaMethodEDT, awtinvoke, awtcreate and javacomponent. JavaMethodEDT and javaObjectEDT were introduced in version R2008b (7.7) and are minimally and only partially documented although they have reasonably complete help comments. The other three are semi-documented (meaning they are unsupported but if you edit or type their m-file you’ll see a fairly detailed help section), and although there is some overlap in their functionality they are still available.

javaObjectEDT and javaMethodEDT

javaObjectEDT is the the preferred method since R2008b of creating swing components to be used on the EDT. An object created with javaObjectEDT will have all of its subsequent method calls run on the EDT. This is termed Auto Delegation. Auto-delegation greatly simplifies and increases the readability of code. Note that objects created as a result of method calls may not be implemented on the EDT.

If you have an existing Java object, you can pass it to javaObjectEDT at any time – all its subsequent calls will then onward run on the EDT. Note that this useful functionality is an under-documented javaObjectEDT feature: it is not mentioned in the main help section but only implied from the example.

% Create a button on the EDT
btn = javaObjectEDT('javax.swing.JButton');
% this will run on EDT since btn was javaObjectEDT-created
btn.setText('Button');
 
% Create a button NOT on the EDT
btn2 = javax.swing.JButton;
% Dangerous! call will run on main Matlab thread
btn2.setText('Button2');
% modify btn2 so its methods will start running on the EDT
javaObjectEDT(btn2);
btn2.setText('Button2');

The following example shows the use of javaObjectEDT and javaMethodEDT in a more complex situation using a JTable:

function tableExample
hFig = figure;
drawnow; %need to get figure rendered
 
%use Yair's createTable to add a javax.swing.JTable
%http://www.mathworks.com/matlabcentral/fileexchange/14225-java-based-data-table
%wrap ceateTable in javaObjectEDT to put the ensuing method calls on the EDT
f = java.awt.Font(java.lang.String('Dialog'),java.awt.Font.PLAIN,14);
headers = {'Selected','File','Analysis Routine','Task Status'};
tbl = javaObjectEDT(createTable(hFig,headers,[],false,'Font',f));
 
%set column 1 to use check boxes and set up a change callback
tbl.setCheckBoxEditor(1);
jtable = javaObjectEDT(tbl.getTable); %get the underlying Java Table. IMPORTANT: we need to put jtable on the EDT
columnModel = javaObjectEDT(jtable.getColumnModel); %now we can now do direct calls safely on jtable
selectColumn = javaObjectEDT(columnModel.getColumn(0));
selectColumnCellEditor = selectColumn.getCellEditor;
chk = javaMethodEDT('getComponent',selectColumnCellEditor);
set(chk,'ItemStateChangedCallback',@chkChange_Callback);
 
%make column three a combo drop down
analysisTable = {'Analysis1';'Analysis2';'Analysis3'};
cb = javaObjectEDT('com.mathworks.mwswing.MJComboBox',analysisTable);
cb.setEditable(false);
cb.setFont(f);
set(cb,'ItemStateChangedCallback',@cbChange_Callback);
editor = javaObjectEDT('javax.swing.DefaultCellEditor',cb);
analysisColumn = javaObjectEDT(columnModel.getColumn(2));
analysisColumn.setCellEditor(editor);
 
%set some column with restrictions
selectColumn.setMaxWidth(100);
analysisColumn.setPreferredWidth(300);
 
%set the data
SELECTED = java.awt.event.ItemEvent.SELECTED;
tbl.setData({false,'file1','Analysis2','Analysis2';...
             true,'file2','Analysis3','Analysis3'});
drawnow;
 
    function cbChange_Callback(src,ev) %#ok
        jRow = jtable.getSelectedRow;
        stateChange = javaMethodEDT('getStateChange',ev);
        if stateChange == SELECTED
            newData = javaMethodEDT('getItem',ev);
            model = jtable.getModel;
            javaMethodEDT('setValueAt',model,newData,jRow,3);
        end %if
    end %cbChange
 
    function chkChange_Callback(src,ev) %#ok
        chkBox = javaMethodEDT('getItem',ev);
        if logical(javaMethodEDT('isSelected',chkBox))
            beep; %put useful code here
        else
            beep;
            pause(0.1)
            beep; %put useful code here
        end %if
    end %chkChange_Callback
 
end %tableExample

If you are running Matlab R2008a or later, javacomponent uses the javaObjectEDT function to create the returned objects so you do not have to do anything further to these objects to have their calls dispatched on the EDT. Users need to take care that objects added directly to the components created by javacomponent are on the EDT as well as specialized sub-components (e.g. CellRenderers and CellEditors). The overhead of calling javaMethodEDT is fairly small so if in doubt, use it.

javaObjectEDT and its kin first appeared in R2008a, although they only became supported in R2008b. Unfortunately, using them on R2008a sometimes causes hangs and all sorts of other mis-behaviors. This problem was fixed in the R2008b release, when javaObjectEDT became a fully-supported function. The problem with using javaObjectEDT in our application is that if it ever runs on an R2008a platform it might hang! (on Matlab release R2007b and earlier we will get an informative message saying that the javaObjectEDT function does not exist)

For this reason, I am using the following method in my projects:

function result = javaObjEDT(varargin)
%Placeholder of Matlab's buggy javaObjectEDT function on R2008a
 
% Programmed by Yair M. Altman: altmany(at)gmail.com
% $Revision: 1.2 $  $Date: 2009/01/25 11:31:08 $
 
  try
      try
          result = varargin{1};
      catch
          result = [];
      end
      v = version;
      if str2double(v(1:3)) > 7.6
          result = builtin('javaObjectEDT',varargin{:});
      end
  catch
      % never mind
  end
end

Note that javaMethodEDT has the method name as its first input argument, and the object name or reference as its second arg. This is inconsistent with many other Matlab/Java functions, which normally accept the target object as the first argument (compare: invoke, awtinvoke, notify etc.). It also means that we cannot use the familiar obj.javaMethodEDT(methodName) format.

One final note: when javaObjectEDT and javaMethodEDT first appeared in R2008a, they were complemented by the javaObjectMT and javaMethodMT functions, which create and delegate Java objects on the main Matlab computational thread. Their internal documentation says that there are cases when execution must occur on the MT rather than EDT, although I am personally not aware of any such case.

awtcreate and awtinvoke

For users with versions prior to R2008b the user must use the awtcreate function to create objects on the EDT. One huge disadvantage of this older function is that if you have to pass java objects in the parameter list you must use the very cumbersome JNI style notation. For example, for the simple task of setting a button label, one has to use:

btn = awtcreate('javax.swing.JButton');
awtinvoke(btn,'setText(Ljava/lang/String;)','click me')

The other disadvantage is that creating the object using awtcreate does not ensure that its subsequent method calls will be executed on the EDT. The awtinvoke function must be used for each call.

Also, both awtcreate and awtinvoke have some limitations due to bugs in the private parseJavaSignature function (for example, invoking methods which accept a java.lang.Object) which forces one to use the direct call to the method, using the main Matlab thread. This can result in the undesired effects described above. In this situation the best workaround is to call pause(0.01) to allow the event queue to clear.

Versions of javacomponent earlier than R2008a use awtcreate and objects created by these versions must have their subsequent methods called by awtinvoke to be used on the EDT.

A very rare CSSM thread discusses the usage of awtcreate and awtinvoke with some very interesting remarks by MathWorks personnel.

There is an interesting option in awtinvoke that was not carried over into the newer javaMethodEDT. This option allows the user to pass a function handle in the argument list along with its parameters. This option creates an undocumented com.mathworks.jmi.Callback object that has a delayed callback. The delayed callback is dispatched on the EDT so that it will be called once the java method used in awtinvoke is finished. Note that the actual function will still execute on the main Matlab thread the delayed callback will just control when it is called. However this may be useful at times. It is possible to put this functionality into a separate function we can call to delay execution until the event queue is cleared.

%CALLBACKONEDTQUEUE will place a callback on the EDT to asynchronously
%run a function.
%CALLBBACKONEDTQUEUE(FCN) will run function handle FCN once all previous
%methods dispatched to the EDT have completed.
%CALLBBACKONEDTQUEUE(FCN,ARG1,ARG2,...) ill run function handle FCN with
%arguments ARG1,ARG2...once all previous methods dispatched to the EDT
%have completed.
%Note that the function is still executing on the main Matlab thread. This
%function just delays when it will be called.
function callbackOnEDTQueue(varargin)
    validateattributes(varargin{1},{'function_handle'},{});
    callbackObj = handle(com.mathworks.jmi.Callback,'callbackProperties');
    set(callbackObj,'delayedCallback',{@cbEval,varargin(:)});
    callbackObj.postCallback;
 
    function cbEval(src,evt,args) %#ok
        feval(args{:});
    end %cbEval
end %callbackOnEDTQueue

Inactive Control Tooltips & Event Chaining

Wednesday, February 24th, 2010

Once again, I welcome guest blogger Matt Whitaker, who continues his series of articles.

In my last post I explored some tips on tooltips. One of these tips involved displaying tooltips on disabled uicontrols. I explained that displaying tooltips on inactive controls is problematic since Matlab appears to intercept mouse events to these inactive controls, so even setting the tooltip on the underlying Java object will not work: The java object appears not to receive the mouse-hover event and therefore does not “know” that it’s time to display the tooltip.

When Yair and I deliberated this issue, he pointed me to his comment on a previous article showing an undocumented Java technique (Java also has some…) for forcing a tooltip to appear using the ActionMap of the uicontrol’s underlying Java object to get at a postTip action. We discussed using a WindowButtonMotionFcn callback to see if the mouse was above the inactive control, then triggering the forced tooltip display. Yair then went on to remind me and I quote: “you’ll need to chain existing WindowButtonMotionFcn callbacks and take into account ModeManagers that override them.”

Frankly, having written code previously that handles callback chaining, I would rather poke myself in the eye with a fork!

The Image Processing Toolbox has the nice pair of iptaddcallback and iptremovecallback functions that largely handle these issues. But for general Matlab, there seemed to be no alternative until I remembered that events trigger callbacks. I decided to use a listener for the WindowButtonMotion event to detect the mouse motion. Event listeners were briefly explained two weeks ago and deserve a dedicated future article. The advantage of using an event listener is that we don’t disturb any existing WindowButtonMotionFcn callback. We still need to be somewhat careful that our listeners don’t do conflicting things, but it’s a lot easier than trying to manage everything through the single WindowButtonMotionFcn.

A demonstration of this appears below with some comments following (note that this code uses the FindJObj utility):

function inactiveBtnToolTip
  %Illustrates how to make a tooltip appear on an inactive control
  h = figure('WindowButtonMotionFcn',@windowMotion,'Pos',[400,400,200,200]);
  col = get(h,'color');
  lbl = uicontrol('Style','text', 'Pos',[10,160,120,20], ...
                  'Background',col, 'HorizontalAlignment','left');
  btn = uicontrol('Parent',h, 'String','Button', ...
                  'Enable','inactive', 'Pos',[10,40,60,20]);
  uicontrol('Style','check', 'Parent',h, 'String','Enable button tooltip', ...
            'Callback',@chkTooltipEnable, 'Value',1, ...
            'Pos',[10,80,180,20], 'Background',col);
  drawnow;
 
  %create the tooltip and postTip action
  jBtn = findjobj(btn);
  import java.awt.event.ActionEvent;
  javaMethodEDT('setToolTipText',jBtn,'This button is inactive');
  actionMap = javaMethodEDT('getActionMap',jBtn);
  action = javaMethodEDT('get',actionMap,'postTip');
  actionEvent = ActionEvent(jBtn, ActionEvent.ACTION_PERFORMED, 'postTip');
 
  %get the extents plus 2 pixels of the control to compare to the mouse position
  btnPos = getpixelposition(btn)+[-2,-2,4,4]; %give a little band around the control
  left = btnPos(1);
  right = sum(btnPos([1,3]));
  btm = btnPos(2);
  top =  sum(btnPos([2,4]));
 
  % add a listener on mouse movement events
  tm = javax.swing.ToolTipManager.sharedInstance; %tooltip manager
  pointListener = handle.listener(h,'WindowButtonMotionEvent',@figMouseMove);
 
  %inControl is a flag to prevent multiple triggers of the postTip action
  %while mouse remains in the button
  inControl = false;
 
  function figMouseMove(src,evtData) %#ok
    %get the current point
    cPoint = evtData.CurrentPoint;
 
    if cPoint(1) >= left && cPoint(1) <= right &&...
       cPoint(2) >= btm  && cPoint(2) <= top
 
      if ~inControl %we just entered
        inControl = true;
        action.actionPerformed(actionEvent); %show the tooltip
      end %if
    else
      if inControl %we just existed
        inControl = false;
        %toggle to make it disappear when leaving button
        javaMethodEDT('setEnabled',tm,false);
        javaMethodEDT('setEnabled',tm,true);
      end %if
    end %if
  end %gpMouseMove
 
  function windowMotion(varargin)
    %illustrate that we can still do a regular window button motion callback
    set(lbl,'String',sprintf('Mouse position: %d, %d',get(h,'CurrentPoint')));
    drawnow;
  end %windowMotion
 
  function chkTooltipEnable(src,varargin)
    if get(src,'Value')
      set(pointListener,'Enable','on');
    else
      set(pointListener,'Enable','off');
    end %if
  end %chkTooltipEnable
end %inactiveBtnToolTip

Tooltip on an inactive button

Tooltip on an inactive button

Comments on the code:

  1. The code illustrates that we can successfully add an additional listener to listen for mouse motion events while still carrying out the original WindowButtonMotionFcn callback. This makes chaining callbacks much easier.
  2. The handle.listener object has an Enable property that we can use to temporarily turn the listener on and off. This can be seen in the chkTooltipEnable() callback for the check box in the code above. If we wanted to permanently remove the listener we would simply use delete(pointListener). Note that addlistener adds a hidden property to the object being listened to, so that the listener is tied to the object’s lifecycle. If you create a listener directly using handle.listener you are responsible for it’s disposition. Unfortunately, addlistener fails for HG handles on pre-R2009 Matlab releases, so we use handle.listener directly.
  3. The code illustrates a good practice when tracking rapidly firing events like mouse movement of handling reentry into the callback while it is still processing a previous callback. Here we use a flag called inControl to prevent the postTip action being continuously fired while the mouse remains in the control.
  4. I was unable to determine if there is any corresponding action for the postTip to dismiss tips so I resorted to using the ToolTipManager to toggle its own Enable property to cleanly hide the tooltip as the mouse leaves the control.

Each Matlab callback has an associated event with it. Some of the ones that might be immediately useful at the figure-level are WindowButtonDown, WindowButtonUp, WindowKeyPress, and WindowKeyRelease. They can all be accessed through handle.listener or addlistener as in the code above.

Unfortunately, events do not always have names that directly correspond to the callback names. In order to see the list of available events for a particular Matlab object, use the following code, which relies on another undocumented function – classhandle. Here we list the events for gcf:

>> get(get(classhandle(handle(gcf)),'Events'),'Name')
ans = 
    'SerializeEvent'
    'FigureUpdateEvent'
    'ResizeEvent'
    'WindowKeyReleaseEvent'
    'WindowKeyPressEvent'
    'WindowButtonUpEvent'
    'WindowButtonDownEvent'
    'WindowButtonMotionEvent'
    'WindowPostChangeEvent'
    'WindowPreChangeEvent'

Note that I have made extensive use of the javaMethodEDT function to execute Java methods that affect swing components on Swing’s Event Dispatch Thread. I plan to write about this and related functions in my next article.

Additional uicontrol tooltip hacks

Wednesday, February 17th, 2010

Once again I’d like to welcome guest blogger Matthew Whitaker. Today Matt will discuss uicontrol tooltips hacks as the first of several posts that follow a logical thread culminating in the long promised article on the EDT.

A while back Yair wrote a cool article, Spicing up Matlab uicontrol tooltips, describing use of HTML formatting and images in uicontrol tooltips. I want to share some limitations I’ve seen with tooltips and their solution using the Matlab control’s underlying Java object.

Situation 1: Displaying a tooltip on disabled controls

One issue with the stock Matlab uicontrol tooltips is that if you turn the uicontrol’s Enable property to ‘inactive’ or ‘off’, its tooltip no longer displays. This is the behavior that we normally want, but occasionally we wish to display a tooltip on a disabled control, for example, to explain why the control is disabled.

You can use the findjobj utility to find the Java handle for the uicontrol. This handle can then be used to set the tooltip text. The tooltip will display if you disable the control using its Java handle’s Enabled property rather than the Matlab handle’s Enable property:

hButton = uicontrol('String','Button');
drawnow;
jButton= findjobj(hButton);
set(jButton,'Enabled',false);
set(jButton,'ToolTipText','This is disabled for a reason');

As customary for Java objects, its properties can also be set using their corresponding accessor methods:

javaMethodEDT('setEnabled',jButton,false);
javaMethodEDT('setToolTipText',jButton,'Button is disabled for a reason');

tooltip on a disabled uicontrol

tooltip on a disabled uicontrol

Unfortunately, this hack does not work for ‘inactive’ controls. There is no direct Java analogy for inactive controls – it’s a Matlab extension. It appears that Matlab somehow intercepts the mouse events associated with inactive controls. My next post will show how event callback can be used to display tooltips for inactive controls.

As an alternative for inactive edit-box controls, we can simulate the inactive behavior by setting the Java object’s Editable property (or by using its setEditable() accessor method), then setting the tooltip. Note that the extremely-useful Java Editable property is unavailable in the Matlab handle, for some inexplicable reason:

hEditbox = uicontrol('String','Edit Text','Style','edit');
drawnow;
jEditbox = findjobj(hEditbox);
set(jEditbox,'Editable',false);
set(jEditbox,'ToolTipText','Text is inactive for a reason');

tooltip on a non-editable editbox

tooltip on a non-editable editbox

Situation 2: Displaying a tooltip on truncated text

If we want to conditionally display a tooltip for an editbox uicontrol when the text exceeds the control’s width, we can use the TipWhenTruncatedEnabled property (or its corresponding setTipWhenTruncatedEnabled() method). This will display a tooltip with the editbox contents if the string is shown truncated. This saves the user having to scroll through the control to see its contents. I often use this for edit controls that may contain long path names:

hEditbox(1) = uicontrol('Style','edit','Units','norm','Pos',[0.1,0.8,0.4,0.05], 'String','Too Short');
hEditbox(2) = uicontrol('Style','edit','Units','norm','Pos',[0.1,0.7,0.2,0.05], 'String','Long Enough to Display a Tool Tip');
drawnow;
jEditbox1 = findjobj(hEditbox(1));
jEditbox2 = findjobj(hEditbox(2));
set(jEditbox1,'TipWhenTruncatedEnabled',true);  % property-based alternative
javaMethod('setTipWhenTruncatedEnabled',jEditbox2,true);  % method-based alternative

TipWhenTruncatedEnabled tooltip

TipWhenTruncatedEnabled tooltip

The TipWhenTruncatedEnabled property property is also available for multi-line editboxes, but has (obviously) no effect when scrollbars are present. Also note that setting the TipWhenTruncatedEnabled property to true overrides any previous tooltip that might have been set for the editbox.

Finally, note that the TipWhenTruncatedEnabled property can also be set for the editbox component of popup-menu (aka drop-down) controls, after they have been set to be editable using their Java Editable property (note that both properties are false by default for Matlab uicontrols). In the following screenshot, the drop-down’s editbox component contained an HTML snippet, that is shown unformatted within the edit-box and HTML-formatted in the de-truncated tooltip:

de-truncated HTML-format tooltip

de-truncated HTML-format tooltip

Situation 3: Controlling tooltip timing

As you have probably noticed, there is a slight delay between the time your mouse enters the control and when the tooltip actually appears. If you display a tooltip over a control for sufficiently long the tooltip will then disappear. Sometimes the default delays are too slow or fast for your application. These times can be controlled through the javax.swing.ToolTipManager. The ToolTipManager sets these parameters globally (including for your Matlab desktop components), but they are not persistent between sessions.

Some examples using the ToolTipManager:

btn = uicontrol('String','Button','Tooltip','This is a button.','Pos',[100,100,75,25]);
txt = uicontrol('Style','edit','String','Edit Text','Tooltip','This is editable text','Pos',[100,50,75,25]);
 
tm = javax.swing.ToolTipManager.sharedInstance; %static method to get ToolTipManager object
initialDelay = javaMethodEDT('getInitialDelay',tm); %get the delay before display in milliseconds (=750 on my system)
javaMethodEDT('setInitialDelay',tm,0); %set tooltips to appear immediately
 
dismissDelay = javaMethodEDT('getDismissDelay',tm); %get the delay before the tooltip disappears (=10000 (10 sec) on my system)
javaMethodEDT('setDismissDelay',tm,2000); %set the dismiss delay to 2 seconds
 
javaMethodEDT('setEnabled',tm,false); %turns off all tooltips in system (including the Matlab desktop)
javaMethodEDT('setEnabled',tm,true);
 
javaMethodEDT('setInitialDelay',tm,initialDelay);
javaMethodEDT('setDismissDelay',tm,dismissDelay);

Note that I have made extensive use of the undocumented built-in javaMethodEDT function to execute Java methods that affect Swing components on the Swing Event Dispatch Thread (EDT). I plan to write about EDT following my next post on event callback chaining.

Solving a MATLAB bug by subclassing

Sunday, February 7th, 2010

I would like to welcome guest blogger Matthew Whitaker. Many of Matt’s CSSM submissions offer important insight of internal Matlab functionality. As shall be seen by today’s article and some future submissions, Matt has plenty to share vis-a-vis Matlab’s undocumented functionality.

In my day-to-day work I make extensive use of MATLAB’s Image Processing Toolbox (IPT). One area of the toolbox that has seen considerable change over the last few releases has been the development of a set of modular tools to aid in GUI design for image processing applications. In this article, I examine a bug in one of those tools to illustrate how we can use the power of subclassing these objects (using an undocumented property) to design a simple and effective workaround.

The problem

The problem arose as I was refactoring some code that was written in R2006b to R2009b. The code in question uses the impoint tool on an image along with an associated text object that moves with the point to display information as it is dragged around the image. At the time of the R2006b release the impoint tool was written as an API. In R2006b the call to impoint returns a handle to an hggroup containing a structure of function handles in its application data under the tag ‘API’. This programming pattern was common before the advent of the new class syntax in MATLAB version 7.6 (R2008a).

Here is an example of how impoint would be used in R2006b:

function impointBehavior_R2006b
%IMPOINTBEHAVIOR_R2006B shows how impoint would be used in R2006b
%Note: RUN UNDER R2006B (will run under R2009b but actually uses
%classdef impoint so it will show the same issue)
 
  % Display the image in a figure window
  figure;  imshow('rice.png');
 
  % In R2006b calling impoint returns the hggroup handle
  h = impoint(gca,100,200);
 
  % In 2006b iptgetapi returns a structure of function handles
  api = iptgetapi(h);
 
  % Add a new position callback to set the text string
  api.addNewPositionCallback(@newPos_Callback);
 
  % Construct boundary constraint function so we can't go outside the axes
  fcn = makeConstrainToRectFcn('impoint',get(gca,'XLim'),get(gca,'YLim'));
  api.setDragConstraintFcn(fcn);
 
  % Fire callback so we get initial text
  newPos_Callback(api.getPosition());
 
  function newPos_Callback(newPos)
    % Display the current point position in a text label
    api.setString(sprintf('(%1.0f,%1.0f)',newPos(1),newPos(2)));
  end %newPos_Callback
 
end %impointBehavior_R2006b

The code above, when run in R2006b, produces the desired behavior of displaying a text object containing the point coordinates that moves around with the point as it is dragged around the axes.

In R2009b, impoint is now a true MATLAB class using the new classdef syntax, so I wanted to update the existing code. Initially this appeared to be a straightforward translation of the code to make use of the new impoint class syntax. The first attempt to rewrite the code was:

function impointBehavior_R2009b
%IMPOINTBEHAVIOR_R2009B shows the undesirable behavior when
%using the setString method in R2009b. 
 
  % Display the image in a figure window
  figure;  imshow('rice.png');
  h = impoint(gca,100,200);
 
  % Add a new position callback to set the text string
  h.addNewPositionCallback(@newPos_Callback);
 
  % Construct boundary constraint function so we can't go outside the axes
  fcn = makeConstrainToRectFcn('impoint',get(gca,'XLim'),get(gca,'YLim'));
 
  % Enforce boundary constraint function
  h.setPositionConstraintFcn(fcn);
 
  % Fire callback so we get initial text
  newPos_Callback(h.getPosition());
 
  function newPos_Callback(newPos)
    % Display the current point position in a text label
    h.setString(sprintf('(%1.0f,%1.0f)',newPos(1),newPos(2)))
  end %newPos_Callback
 
end %impointBehavior_R2009b

Unfortunately, when this code is run, dragging the mouse around the axes produces a trail of labels as shown below:

Before - buggy behavior

Before - buggy behavior

Opening up impoint.m in the editor and tracing the code revealed that impoint’s setString method creates a new text object each time it is used. I reported this to MATLAB and the bug is now documented on the MathWorks Support site (574846).

The solution

So how do we work around this bug to get to the behavior we want? One solution would be to rewrite the offending MATLAB code but this is somewhat risky in terms of maintainability and compatibility.

A more elegant solution is to subclass the impoint class and substitute the setString behavior we want. Looking at the impoint code we find that impoint is a subclass of imroi. In the imroi property declarations we see a number of undocumented properties that are protected. We can access these properties in a subclass but not outside the class. One of these undocumented properties is h_group which is an hggroup that contains the handle graphic objects that make up the impoint on the screen. The label, when created, becomes part of this hggroup with its Tag property set to ‘label’. When performing the setString method the behavior we want to see is that if the text object exists we want to update its String property. If it does not exist we want it to perform its existing functionality:

classdef impointtextupdate < impoint
%IMPOINTTEXTUPDATE subclasses impoint to override the setString
%method of impoint so that it does not create a new text object
%each time it is called.   
 
  methods
    function obj = impointtextupdate(varargin)
      obj = obj@impoint(varargin{:});
    end %impointtextupdate
 
    function setString(obj,str)
      %override impoint setString
 
      %check to see if there is already a text label
      label = findobj(obj.h_group,'Type','text','Tag','label');
 
      if isempty(label)
        %do the default behavior
        setString@impoint(obj,str);
      else
        %update the existing tag
        set(label(1),'String',str);
      end %if
    end %setString
 
  end %methods
end %impointtextupdate

Substituting calls to impoint with the new impointupdatetext subclass now produces the desired effect as shown below:

After - expected behavior

After - expected behavior

Conclusions

This case illustrates a couple of points:

  • Much of the existing code in the MATLAB toolboxes is being updated to the new object oriented syntax. This presents many opportunities to easily and elegantly modify the default behavior without modifying provided toolbox code In the example above we retain all the desirable behavior of impoint while overriding the undesirable behavior.
  • Many of the properties and methods in the provided toolbox objects are hidden or protected and are undocumented. It takes some simple detective work to find these out through examining the code. MATLAB is very generous in providing much of the existing code openly. Open the functions and classes you use in the editor to really find out how they work. Over the years I’ve learned and adopted a lot of useful MATLAB programming patterns by examining the code in the various toolboxes (there are a few coding horrors as well!).

I hope to explore some other under-documented features of the IPT and other toolboxes in future posts.