Once again I welcome guest blogger Matt Whitaker, with the long awaited EDT article.
Java Swing’s Event Dispatch Thread (EDT)
or: why does my GUI foul up?
Matlab for the most part is a single threaded environment. That is, all commands are executed sequentially along a single execution thread. The main exception to this are the Handle Graphics (GUI) components whose operations execute on the Java Event Dispatch Thread (EDT). EDT effects are reflected even in mundane Matlab GUI operations.
If we execute the code below we will probably see nothing until the loop completes and the figure appears with the text label showing ‘10000’:
h = figure; txt = uicontrol('Parent',h, 'Style','text', 'String','1'); for n = 1:10000 set(txt,'String',int2str(n)) end %for |
By adding a couple of drawnow commands we get the figure and text label to render and then we see the count progress to 10000.
h = figure; txt = uicontrol('Parent',h, 'Style','text', 'String','1'); drawnow; for n = 1:10000 set(txt,'String',int2str(n)); drawnow; end %for |
The drawnow function allows the EDT queue to be flushed and the pending graphics operations to be evaluated. This will also happen with pause and several other commands.
If we want to use Swing (or AWT) components in our user interfaces we need to take this multi-threaded environment into account. The Swing toolkit designers decided to make all the Swing components thread un-safe in order to decrease their complexity. As a consequence, all access to Swing components should be done from the event dispatch thread (EDT), to ensure the operations are executed sequentially, at the exact order in which they were dispatched. Any action on a Swing component done on another thread (Matlab’s main processing thread in our case) risks a race-condition or deadlock with the EDT, which could (and often does) result in weird, non-deterministic and non-repetitive behavior – all of which should be avoided in any application which should behave in a precisely deterministic manner.
In Java, the usual pattern to accomplish EDT dispatching is to create a Runnable object, encapsulate the GUI code in the run method of the Runnable object, then pass the Runnable object to the static EventQueue.invokeLater (or EventQueue.invokeAndWait if we need to block operations to get a return value) method.
Runnable runnable = new Runnable() { public void run() { //GUI Code here } } EventQueue.invokeLater(runnable); |
There are several functions in Matlab that implement this programming pattern for us: javaObjectEDT, javaMethodEDT, awtinvoke, awtcreate and javacomponent. JavaMethodEDT and javaObjectEDT were introduced in version R2008b (7.7) and are minimally and only partially documented although they have reasonably complete help comments. The other three are semi-documented (meaning they are unsupported but if you edit or type their m-file you’ll see a fairly detailed help section), and although there is some overlap in their functionality they are still available.
javaObjectEDT and javaMethodEDT
javaObjectEDT is the the preferred method since R2008b of creating swing components to be used on the EDT. An object created with javaObjectEDT will have all of its subsequent method calls run on the EDT. This is termed Auto Delegation. Auto-delegation greatly simplifies and increases the readability of code. Note that objects created as a result of method calls may not be implemented on the EDT.
If you have an existing Java object, you can pass it to javaObjectEDT at any time – all its subsequent calls will then onward run on the EDT. Note that this useful functionality is an under-documented javaObjectEDT feature: it is not mentioned in the main help section but only implied from the example.
% Create a button on the EDT btn = javaObjectEDT('javax.swing.JButton'); % this will run on EDT since btn was javaObjectEDT-created btn.setText('Button'); % Create a button NOT on the EDT btn2 = javax.swing.JButton; % Dangerous! call will run on main Matlab thread btn2.setText('Button2'); % modify btn2 so its methods will start running on the EDT javaObjectEDT(btn2); btn2.setText('Button2'); |
The following example shows the use of javaObjectEDT and javaMethodEDT in a more complex situation using a JTable:
function tableExample hFig = figure; drawnow; %need to get figure rendered %use Yair's createTable to add a javax.swing.JTable %http://www.mathworks.com/matlabcentral/fileexchange/14225-java-based-data-table %wrap ceateTable in javaObjectEDT to put the ensuing method calls on the EDT f = java.awt.Font(java.lang.String('Dialog'),java.awt.Font.PLAIN,14); headers = {'Selected','File','Analysis Routine','Task Status'}; tbl = javaObjectEDT(createTable(hFig,headers,[],false,'Font',f)); %set column 1 to use check boxes and set up a change callback tbl.setCheckBoxEditor(1); jtable = javaObjectEDT(tbl.getTable); %get the underlying Java Table. IMPORTANT: we need to put jtable on the EDT columnModel = javaObjectEDT(jtable.getColumnModel); %now we can now do direct calls safely on jtable selectColumn = javaObjectEDT(columnModel.getColumn(0)); selectColumnCellEditor = selectColumn.getCellEditor; chk = javaMethodEDT('getComponent',selectColumnCellEditor); set(chk,'ItemStateChangedCallback',@chkChange_Callback); %make column three a combo drop down analysisTable = {'Analysis1';'Analysis2';'Analysis3'}; cb = javaObjectEDT('com.mathworks.mwswing.MJComboBox',analysisTable); cb.setEditable(false); cb.setFont(f); set(cb,'ItemStateChangedCallback',@cbChange_Callback); editor = javaObjectEDT('javax.swing.DefaultCellEditor',cb); analysisColumn = javaObjectEDT(columnModel.getColumn(2)); analysisColumn.setCellEditor(editor); %set some column with restrictions selectColumn.setMaxWidth(100); analysisColumn.setPreferredWidth(300); %set the data SELECTED = java.awt.event.ItemEvent.SELECTED; tbl.setData({false,'file1','Analysis2','Analysis2';... true,'file2','Analysis3','Analysis3'}); drawnow; function cbChange_Callback(src,ev) %#ok jRow = jtable.getSelectedRow; stateChange = javaMethodEDT('getStateChange',ev); if stateChange == SELECTED newData = javaMethodEDT('getItem',ev); model = jtable.getModel; javaMethodEDT('setValueAt',model,newData,jRow,3); end %if end %cbChange function chkChange_Callback(src,ev) %#ok chkBox = javaMethodEDT('getItem',ev); if logical(javaMethodEDT('isSelected',chkBox)) beep; %put useful code here else beep; pause(0.1) beep; %put useful code here end %if end %chkChange_Callback end %tableExample |
If you are running Matlab R2008a or later, javacomponent uses the javaObjectEDT function to create the returned objects so you do not have to do anything further to these objects to have their calls dispatched on the EDT. Users need to take care that objects added directly to the components created by javacomponent are on the EDT as well as specialized sub-components (e.g. CellRenderers and CellEditors). The overhead of calling javaMethodEDT is fairly small so if in doubt, use it.
javaObjectEDT and its kin first appeared in R2008a, although they only became supported in R2008b. Unfortunately, using them on R2008a sometimes causes hangs and all sorts of other mis-behaviors. This problem was fixed in the R2008b release, when javaObjectEDT became a fully-supported function. The problem with using javaObjectEDT in our application is that if it ever runs on an R2008a platform it might hang! (on Matlab release R2007b and earlier we will get an informative message saying that the javaObjectEDT function does not exist)
For this reason, I am using the following method in my projects:
function result = javaObjEDT(varargin) %Placeholder of Matlab's buggy javaObjectEDT function on R2008a % Programmed by Yair M. Altman: altmany(at)gmail.com % $Revision: 1.2 $ $Date: 2009/01/25 11:31:08 $ try try result = varargin{1}; catch result = []; end v = version; if str2double(v(1:3)) > 7.6 result = builtin('javaObjectEDT',varargin{:}); end catch % never mind end end |
Note that javaMethodEDT has the method name as its first input argument, and the object name or reference as its second arg. This is inconsistent with many other Matlab/Java functions, which normally accept the target object as the first argument (compare: invoke, awtinvoke, notify etc.). It also means that we cannot use the familiar obj.javaMethodEDT(methodName) format.
One final note: when javaObjectEDT and javaMethodEDT first appeared in R2008a, they were complemented by the javaObjectMT and javaMethodMT functions, which create and delegate Java objects on the main Matlab computational thread. Their internal documentation says that there are cases when execution must occur on the MT rather than EDT, although I am personally not aware of any such case.
awtcreate and awtinvoke
For users with versions prior to R2008b the user must use the awtcreate function to create objects on the EDT. One huge disadvantage of this older function is that if you have to pass java objects in the parameter list you must use the very cumbersome JNI style notation. For example, for the simple task of setting a button label, one has to use:
btn = awtcreate('javax.swing.JButton'); awtinvoke(btn,'setText(Ljava/lang/String;)','click me') |
The other disadvantage is that creating the object using awtcreate does not ensure that its subsequent method calls will be executed on the EDT. The awtinvoke function must be used for each call.
Also, both awtcreate and awtinvoke have some limitations due to bugs in the private parseJavaSignature function (for example, invoking methods which accept a java.lang.Object) which forces one to use the direct call to the method, using the main Matlab thread. This can result in the undesired effects described above. In this situation the best workaround is to call pause(0.01) to allow the event queue to clear.
Versions of javacomponent earlier than R2008a use awtcreate and objects created by these versions must have their subsequent methods called by awtinvoke to be used on the EDT.
A very rare CSSM thread discusses the usage of awtcreate and awtinvoke with some very interesting remarks by MathWorks personnel.
There is an interesting option in awtinvoke that was not carried over into the newer javaMethodEDT. This option allows the user to pass a function handle in the argument list along with its parameters. This option creates an undocumented com.mathworks.jmi.Callback object that has a delayed callback. The delayed callback is dispatched on the EDT so that it will be called once the java method used in awtinvoke is finished. Note that the actual function will still execute on the main Matlab thread the delayed callback will just control when it is called. However this may be useful at times. It is possible to put this functionality into a separate function we can call to delay execution until the event queue is cleared.
%CALLBACKONEDTQUEUE will place a callback on the EDT to asynchronously %run a function. %CALLBBACKONEDTQUEUE(FCN) will run function handle FCN once all previous %methods dispatched to the EDT have completed. %CALLBBACKONEDTQUEUE(FCN,ARG1,ARG2,...) ill run function handle FCN with %arguments ARG1,ARG2...once all previous methods dispatched to the EDT %have completed. %Note that the function is still executing on the main Matlab thread. This %function just delays when it will be called. function callbackOnEDTQueue(varargin) validateattributes(varargin{1},{'function_handle'},{}); callbackObj = handle(com.mathworks.jmi.Callback,'callbackProperties'); set(callbackObj,'delayedCallback',{@cbEval,varargin(:)}); callbackObj.postCallback; function cbEval(src,evt,args) %#ok feval(args{:}); end %cbEval end %callbackOnEDTQueue |
Isn’t the code
problematic on the upcoming version 7.10?
@Petter – you are correct of course. The relevant section should be changed to:
Hi Yair,
first thanks for all the code and insights you are sharing with us. I have to admit that you inspired me to use a lot of Java functionality in my GUIs. They are looking much better now;-) I was never much concerned with the threading problems in smaller GUIs.
But now I’ve rewrote a complete Matlab GUI with Java Swing and Netbeans and running into some serious threading trouble. With findjobj() I look for the java handles in Matlab and assign the existing Matlab functions as callbacks and change the elements content from Matlab.
I’m not sure if I understood the usage of javaObjectEDT correctly. If I could use javaObjectEDT to create the GUI from MATLAB, there would be no problems, occuring in the subsequent calls to the objects methods. But each time the return value is a new object I have to use javaMethodEDT?
Another question: Does the Matlab set() function dispatch all its calls automatically to the EDT? Should it be used if possible to change properties?
Thanks a lot for your time and keep up the great work. I don’t know what I would do without all your available examples and scripts.
Thanks a lot
Mark
thanks Mark,
For any newly-created object, simply pass this object through javaObjectEDT once to save you the trouble of using javaMethodEDT on each of its subsequent method invocations. As Matt noted in the article, javaObjectEDT‘s overhead is slight and its benefits far outweigh the negligible performance penalty (on R2008b+, that is…).
AFAIK, set does not automatically use EDT if the object was not auto-delegated using javaObjectEDT. Note to set the callback on handle(jObject,’CallbackProperties’) rather than directly on the Java object jObject, in order to prevent a memory leak.
Hello Yair,
You might already be aware of this and probably have a reason for not using it. 🙂
To check for version of MATLAB have you tried the verLessThan function?
Regards .. Varun
@Varun – the reason I do not use verLessThan was explained here.
Correction to a previously-posted comment above: to account for Matlab 7.10 and higher versions, modify the condition as follows:
There’s probably an internal Java class somewhere that returns the major and minor version numbers directly. If I’ll run into such a class I’ll post an addendum.
The check of the Matlab major and minor version with less than 2 STR2DOUBLE, 2 REGEXPREP, 2 VERSION commands:
This is a fast Mex, which can consider all 4 version numbers from VERSION: http://www.mathworks.com/matlabcentral/fileexchange/27231-ismatlabver
Kind regards, Jan
Thanks Jan 🙂
Welcome onboard!
Hi Yair
If I’m visualizing the state of some iterative algorithm, the drawnow call can often take up a significant proportion of processing time. Is there a way to make it asynchronous, so that it doesn’t block the main thread unless drawnow is called again, or even have the behaviour where successive calls are ignored if the current one hasn’t finished?
Thanks.
Oliver
@Oliver – I don’t know how to make drawnow asynchronous. In my GUI-intensive programs I normally add code that only calls drawnow every N iterations (or M milliseconds), for this exact reason.
I find MathWorks’ decision to program the GUI & graph rendering on the main Matlab processing thread a very unfortunate design decision. It is indeed strange that drawnow actually flushes the EDT queue but instead of using the full power of a different processing thread, it simply blocks execution until the EDT thread ends its work! Unfortunately, I don’t know of a way to circumvent this.
Note that while you could run a separate Java thread that periodically calls jComponent.repaint, this solution will only work for uicontrols and similar GUI components. Unfortunately, the graphic elements (anything drawn within Matlab axes) cannot be accessed by Java (AFAIK).
@Yair, while it might be a few years late. I was rereading the post and comments again when I saw your comment: “It is indeed strange that drawnow actually flushes the EDT queue … I don’t know of a way to circumvent this.” I think you are looking for “drawnow expose” or maybe “drawnow update”
[…] in the Command Window everything executes in Matlab’s single main thread. When we pressed “eval”, the code executed from the Event Dispatch Thread (EDT), which is a separate thread. […]
So I guess this works in all cases?
I mean: it seems that all the underlaying Java methods are still available on the obj handle. Are there any draw-backs to always use this construction for creating Java objects?
Also, should the javaObjectEDT function be called on reference handles? e.g.
@Martin (MJJ) – javaObjectEDT only needs to be called once, when you initially create the object. There is no need to call it again for its handle wrapper.
Answering your earlier questions, I do advise to use handle(javaObject,’CallbackProperties’) whenever possible. There are no significant drawbacks, just benefits.
EDT Issue
From http://java.sun.com/products/jfc/tsc/articles/painting/
“JComponent.repaint() registers an asynchronous repaint request to the component’s RepaintManager, which uses invokeLater() to queue a Runnable to later process the request on the event dispatching thread.”
As paints are always done on the EDT, I am not sure that queuing them from a separate thread could achieve much. As Swing uses Java graphics, the two are not entirely separable.
For JComponents there is always paintImmediately.
To skip successive calls, my isMultipleCall on the FEX may help (http://www.mathworks.cn/matlabcentral/fileexchange/26027-ismultiplecall)
Yair hi,
Isn’t this supposed to work for modal dialogs? When a java Dialog class is created without javaObjectEDT, it runs on main thread automatically as your examples suggest (by the way what is dangerous about it, i couldnt catch?). So that when type is modal it stops in main thread and waits. However it shouldnt do that if methods are dispatched from EDT. Well, it does stop.
Simple example;
@Arda – I believe that your code snippet can indeed be explained: You created a Java modal dialog (as explained here) on the EDT. The Matlab Command Window is itself a Java editor component on the EDT. Therefore, when the Dialog window halts the EDT, then the Matlab Command Window, Editor, Profiler, and any other Matlab GUI that runs on the EDT, also halt.
Hi Yair,
Thank you for your many helpful posts everywhere.
I am still a bit confused as to how this is supposed to work. In the following small example, I would have expected that all methods invoked on button1 get run on the EDT. This to me means that the setText() call would be queued up and run after button1’s callback has finished. However, this is not the case.
In other words, I would have expected the text ‘Button 1 text: New Button 1’ (or perhaps, ‘Button 1 text: Button 1’) to be the last thing displayed.
Can you provide me with more insight? Thanks!!
Sorry for the ill-formatted code. I cannot seem to get the code fragment tags to work.
@Ken – I believe, without being 100% certain, that since the callbacks run the svd(rand(1000, 1000)) function, which require Matlab’s Main Thread (MT) to compute, then at that point CPU control on the EDT passes onward to the next available task (button1.setText), which luckily does not require MT and therefore fully executes immediately.
An interesting side-effect can be seen in the Command Window, if immediately after submitting edtTest() you issue any command that requires MT (e.g., another computation or even a simple dir) – this will be blocked until button1Callback ends, and before button2Callback starts. As far as I understand, this is because button2Callback is scheduled on the EDT (by button1Callback’s doClick() request) and is therefore executed after the Command Window command.
Or maybe I’m just mixing things up…
Thanks Yair.
This example is still confusing to me, and I will have to study it more. My main concern is that in this type of situation, if the button1.setText() call was replaced by some other call that may depend on the state of an object that may have changed by button1.doClick(). Then, I believe I would need a drawnow() after button1.doClick() for things to occur in the correct order. With my limited understanding of how matlab interacts with the EDT thread, I almost feel as if I need many drawnow() calls in my matlab/swing gui.
The side-effect with dir is interesting. Also interesting is the case where we put a dir() command between the button1.doClick() and the button1.setText() calls. To me, it seems as if the button1.setText() call is always occurring on the main thread.
[…] make MATLAB UI responsive we have to call function drawnow which flushes Swing EDT queue, see also here and here. This is a known fact, so far so […]
After reading this thread: http://mathforum.org/kb/message.jspa?messageID=5863891
In the tableExample there are a couple of instances where callbacks are set on java objects like this
As the the above thread suggests, should these calls actually be implemented like this?
Just trying to understand all of this Java/Matlab unsafe threading junk.
By the way I just bought your book and am eagerly awaiting its arrival 🙂
@Corbin – you are correct. In practice, unless the code is called numerous times in a loop, the memory leak is unnoticeable. Still, good programming is to wrap the naked Java reference in a Matlab handle. I discuss this issue in this article, as well as in Section 3.4 of my book.
Regarding the following snippet I have question regarding the relationship between the main thread and EDT in matlab.
Once this callback is fired, because it is a matlab function assigned to a callback on a java reference, it is being executed on the main thread (unlike a Swing callback which would be on the EDT). Because of this you are using javaMethodEDT to ensure that both the getItem and isSelected methods are run on the EDT. So my question is this. This if statement
is waiting for isSelected to return with a boolean value. But since this method call has just been queued onto the EDT thread it has to wait in line to be executed before it can be executed. So at this point is the main thread blocked?? As if the javaMethodEDT call is using the SwingUtilities.invokeAndWait method behind the scenes?
By the way, your book arrived and I LOVE IT! 🙂 I never realized how messed up my matlab/java code really was.
@Corbin – this is indeed how I understand it to work. It’s educated guesswork, I really have no inside information on this. But if you take a look at the m-code of javaMethodEDT‘s predecessor, namely awtinvoke, you will see that this is indeed what happened there. It stands to reason that a similar mechanism is used in javaMethodEDT.
Thank you for the kind words and feedback on my book 🙂
Consider the following code which is the java equivalent of your above example :
Unlike your pure-matlab example, text value is refreshed onscreen at each iteration, even if drawnow in not invoked.
Do you know a way to prevent this behavior on JComponents and to wait for drawnow call before graphics is updated ?
@Julien – I do not know a way to control the refresh timing
Dear Yair Sir,
I met this problem and solved successfully according to the solution of your post.
Thanks for your kindly and selfless help very very much.
Hello Yair,
Is there a chance to create a table with one image icon type (or button like, if possible) column using javaObjectEDT and some default table/cell renderers and editors?
I implemented my own Image Renderer derived from DefaultTableCellRenderer java class (with one label and one imageIcon attributes), compiled & tested it using simple java program, zipped the .java and .class files (+ added one .png icon) but javaObjectEDT function ignores it completely and throws an exception : “No class ImageRenderer can be located on the Java class path”.
What would you recommend me?
Many thanks
@Bambi – take a look here: http://undocumentedmatlab.com/blog/java-class-access-pitfalls
Hi Yair,
I am building a pure Java Swing GUI for our backend Matlab application. Matlab starts the GUI using the javaObjectEDT() method. Unfortunately, running the GUI in this thread makes Matlab’s tab-completion unresponsive. Do you have an idea where this might come from and how to circumvent it?
Thank you very much for your help
Regards,
Peter
I have no immediate answer on this
Hi Yair,
thank you for your answer. Please let me know when something comes to your mind.
Best regards and thanks again,
Peter
Hi Peter
May I ask if you found a solution to the tab-completion issue ?
Hi Yair,
I’m interested in the last little bit on the callbackOnEDTQueue() function at the end. In testing this seems to solve my problem of trying to delay running a computation until some later point in time, allowing the rest of the axes to render following a xlim change.
https://www.mathworks.com/matlabcentral/answers/368964-queue-addlistener-events-or-place-event-on-edt
This seems like an alternative to the following post: http://undocumentedmatlab.com/blog/matlab-callbacks-for-java-events#comment-418911
Where my general use case would be to call the notify event (in the linked blog post) directly from Matlab: i.e.
evt.notifyMyTestrather than from Java as the linked post is really about.Am I understanding the callbackOnEDTQueue() function correctly, that it appears to serve the same functionality as the evt class in the link, without the need for writing a Java class and adding it to the Matlab path?
I guess I am confused by this statement:
What are the practical implications of that statement?
Thanks as always,
Jim
Hi Yair,
Thanks a lot for all the examples and information. I have a short question – is there a way to create a matlab GUI off the EDT, i.e. make a matlab GUI which runs on a different thread? I’m using matlab as an interface to a 3rd party application (MicroManager), which uses EDT according to its developers. It seems like matlab’s GUI is interfering with their GUI. Is there a way to bypass this problem somehow? I would greatly appreciate your reply
Thank you very much in advance,
Yiftach
@Yiftach – you can use App Designer and related uifigures/controls – this displays the GUI in a webpage which does not use Java Swing or EDT.
Thanks Yair! I will give it a try
Yiftach