Matlab callbacks for uifigure JavaScript events

I would like to welcome back guest blogger Iliya Romm of Israel’s Technion Turbomachinery and Heat Transfer Laboratory. Today Iliya will discuss how to assign Matlab callbacks to JavaScript events in the new web-based uifigures. Other posts on customizations of web-based Matlab GUI can be found here.

On several occasions (including the previous post by Khris Griffis) I came across people who were really missing the ability to have Matlab respond to various JavaScript (JS) events. While MathWorks are working on their plans to incorporate something similar to this in future releases, we’ll explore the internal tools already available, in the hopes of finding a suitable intermediate solution.

Today I’d like to share a technique I’ve been experimenting with, allowing Matlab to respond to pretty much any JS event to which we can attach a listener. This is an overview of how it works:

  1. create a UIFigure with the desired contents, and add to it (at least) one more dummy control, which has an associated Matlab callback.
  2. execute a JS snippet that programmatically interacts with the dummy control, whenever some event-of-interest happens, causing the Matlab callback to fire.
  3. query the webWindow, from within the Matlab callback, to retrieve any additional information (“payload”) that the JS passed.

This approach allows, for example, to easily respond to mouse events:

Attaching Matlab callback to a uifigure JavaScript event

Consider the code below, which demonstrates different ways of responding to JS events. To run it, save the .m file function below (direct download link) and the four accompanying .js files in the same folder, then run jsEventDemo(demoNum), where demoNum is 1..4. Note: this code was developed on R2018a, unfortunately I cannot guarantee it works on other releases.

function varargout = jsEventDemo(demoNum)
   % Handle inputs and outputs
   if ~nargin
      demoNum = 4;
   end
   if ~nargout
      varargout = {};
   end
 
   % Create a simple figure:
   hFig = uifigure('Position',[680,680,330,240],'Resize','off');
   hTA = uitextarea(hFig, 'Value', 'Psst... Come here...!','Editable','off');
   [hWin,idTA] = mlapptools.getWebElements(hTA);
 
   % Open in browser (DEBUG):
   % mlapptools.waitForFigureReady(hFig); mlapptools.unlockUIFig(hFig); pause(1);
   % web(hWin.URL,'-browser')
 
   % Prepare the JS command corresponding to the requested demo (1-4)
   switch demoNum
      % Demo #1: Respond to mouse events, inside JS, using "onSomething" bindings:
      case 1
         % Example from: https://dojotoolkit.org/documentation/tutorials/1.10/events/#dom-events
         jsCommand = sprintf(fileread('jsDemo1.js'), idTA.ID_val);
 
      % Demo #2: Respond to mouse click events, inside JS, using pub/sub:
      case 2
         % Example from: https://dojotoolkit.org/documentation/tutorials/1.10/events/#publish-subscribe
         hTA.Value = 'Click here and see what happens';
         jsCommand = sprintf(fileread('jsDemo2.js'), idTA.ID_val);
 
      % Demo #3: Trigger Matlab callbacks programmatically from JS by "pressing" a fake button:
      case 3
         hB = uibutton(hFig, 'Visible', 'off', 'Position', [0 0 0 0], ...
                       'ButtonPushedFcn', @fakeButtonCallback);
         [~,idB] = mlapptools.getWebElements(hB);
         jsCommand = sprintf(fileread('jsDemo3.js'), idTA.ID_val, idB.ID_val);
 
      % Demo 4: Trigger Matlab callbacks and include a "payload" (i.e. eventData) JSON:
      case 4
         hB = uibutton(hFig, 'Visible', 'off', 'Position', [0 0 0 0],...
                      'ButtonPushedFcn', @(varargin)smartFakeCallback(varargin{:}, hWin));
         [~,idB] = mlapptools.getWebElements(hB);
         jsCommand = sprintf(fileread('jsDemo4.js'), idTA.ID_val, idB.ID_val);
   end % switch
 
   % Execute the JS command
   hWin.executeJS(jsCommand);
end
 
% Matlab callback function used by Demo #3
function fakeButtonCallback(obj, eventData) %#ok<INUSD>
   disp('Callback activated!');
   pause(2);
end
 
% Matlab callback function used by Demo #4
function smartFakeCallback(obj, eventData, hWin)
   % Retrieve and decode payload JSON:
   payload = jsondecode(hWin.executeJS('payload'));
 
   % Print payload summary to the command window:
   disp(['Responding to the fake ' eventData.EventName ...
         ' event with the payload: ' jsonencode(payload) '.']);
 
   % Update the TextArea
   switch char(payload.action)
      case 'enter',  act_txt = 'entered';
      case 'leave',  act_txt = 'left';
   end
   str = ["Mouse " + act_txt + ' from: '; ...
          "(" + payload.coord(1) + ',' + payload.coord(2) + ')'];  
   obj.Parent.Children(2).Value = str;
end

Several thoughts:

  • The attached .js files will not work by themselves, rather, they require sprintf to replace the %s with valid widget IDs. Of course, these could be made into proper JS functions.
  • Instead of loading the JS files using fileread, we could place the JS code directly in the jsCommand variable, as a Matlab string (char array)
  • I tried getting it to work with a textarea control, so that we would get the payload right in the callback’s eventData object in Matlab, Unfortunately, I couldn’t get it to fire programmatically (solutions like this didn’t work). So instead, I store the payload as JSON, and retrieve it with jsondecode(hWin.executeJS('payload')) in the smartFakeCallback function.

JavaScript files

  1. jsDemo1.js (direct download link):
    require(["dojo/on", "dojo/dom", "dojo/dom-style", "dojo/mouse"],
    	function(on, dom, domStyle, mouse) {
    		var myDiv = dom.byId("%s");
    		on(myDiv, mouse.enter, function(evt){
    			domStyle.set(myDiv, "backgroundColor", "red");
    		});
    		on(myDiv, mouse.leave, function(evt){
    			domStyle.set(myDiv, "backgroundColor", "");
    		});
    	});
  2. jsDemo2.js (direct download link):
    require(["dojo/on", "dojo/topic", "dojo/dom"],
    	function(on, topic, dom) {
    		var myDiv = dom.byId("%s");
    		on(myDiv, "click", function() {
    			topic.publish("alertUser", "Your click was converted into an alert!");
    		});
    		topic.subscribe("alertUser", function(text){
    			alert(text);
    		});
    	});
  3. jsDemo3.js (direct download link):
    require(["dojo/on", "dojo/dom", "dojo/dom-style", "dojo/mouse"],
    	function(on, dom, domStyle, mouse) {
    		var myDiv = dom.byId("%s");
    		var fakeButton = dom.byId("%s");
    		on(myDiv, mouse.enter, function(evt){
    			fakeButton.click();
    		});
    	});
  4. jsDemo4.js (direct download link):
    var payload = [];
    require(["dojo/on", "dojo/dom", "dojo/topic", "dojo/mouse"],
    	function(on, dom, topic, mouse) {
    		var myDiv = dom.byId("%s");
    		var fakeButton = dom.byId("%s");
    		topic.subscribe("sendToMatlab", function(data){
    			payload = data;
    			fakeButton.click();
    		});
    		on(myDiv, mouse.enter, function(evt){
    			data = {action: "enter",
    				coord: [evt.clientX, evt.clientY]};
    			topic.publish("sendToMatlab", data);
    		});
    		on(myDiv, mouse.leave, function(evt){
    			data = {action: "leave",
    				coord: [evt.clientX, evt.clientY]};
    			topic.publish("sendToMatlab", data);
    		});
    	});

Conclusions

As you can see, this opens some interesting possibilities, and I encourage you to experiment with them yourself! This feature will likely be added to the mlapptools toolbox as soon as an intuitive API is conceived.

If you have any comments or questions about the code above, or just want to tell me how you harnessed this mechanism to upgrade your uifigure (I would love to hear about it!), feel free to leave a message below the gist on which this post is based (this way I get notifications!).

Categories: Figure window, Guest bloggers, GUI, High risk of breaking in future versions, Undocumented feature

Tags: , , , ,

Bookmark and SharePrint Print

4 Responses to Matlab callbacks for uifigure JavaScript events

  1. Great workaround Iliya!

    I was playing with something similar to this, though my approach was not as nice. I found that we can add listeners to the DOM for key presses and then use javascript (document.querySelector([id="uniqName_##_#"]').click()) to initiate a button press on an invisible button, which then fires the matlab callback for the .ButtonPushedFcn. Essentially doing what you’ve done here for mouse clicks but for keyboard presses. I haven’t extensively tested it, but the following function works to spit out keypresses into the command window (only tested on 2018a).

    I’m sure there are better ways to capture the keypress information (dojo/keys perhaps) and I would assume it’s possible to capture keypresses in defined contexts, like when a text area is active or when the mouse is hovering over an element, but I haven’t really tested this extensively.

    Cheers,
    Khris

    function [f,b,w] = testkey()
     
      f = uifigure;
     
      b = uibutton(f,'Visible',  'off');
     
      [w,bid] = mlapptools.getWebElements(b);
     
      function clickCallback(s,e)
     
        %callback for button press gets keypress text and displays in console.
     
        tx = w.executeJS('tK');
     
        disp(tx)
     
      end
     
      b.ButtonPushedFcn = @(s,e)clickCallback(s,e);
     
      bqry = sprintf('document.querySelector(''[%s="%s"]'')', ...
     
        bid.ID_attr, ...
     
        bid.ID_val ...
     
        );
     
      jstext = [ ...
     
        'var tK = "", ', ...
     
        'but = %s;', ...
     
        'document.addEventListener(''keypress'', function(e){', ...
     
        '  tK = "";',...
     
        '  if (e.altKey) tK += ''ALT+'';',...
     
        '  if (e.ctrlKey) tK += ''CTRL+'';',...
     
        '  if (e.metaKey) tK += ''META+'';',...
     
        '  if (e.shiftKey) tK += ''SHIFT+'';',...
     
        '  tK += e.key;',...
     
        '  but.click(); ' ...
     
        '});' ...
     
      ];
     
      w.executeJS(sprintf(jstext,bqry));
     
    end
    • Iliya says:

      Hi Khris,

      Thanks for sharing your attempt (and thoughts) regarding the key press listeners! I’m sure it would be a helpful starting point for those who would like to explore this direction further. If you have a suggestion regarding how to add this functionality to mlapptools in a way that would be convenient for the user – please don’t hesitate to open an issue in the repo and/or drop us a line in chat. We’d be happy to hear feedback regarding any existing (or missing) functionality, and ideas about alternative implementations.

      All the best,
      Iliya.

  2. Perttu says:

    Hi,

    I don’t know if this is “general knowledge” but you can turn on debugging by running:

    matlab.internal.webwindow('http://example.com', 'DebugPort', 4040)

    before any other App, or kill all MatlabWindow applications (from task manager) before the command. Then go with Chrome to http://localhost:4040/

  3. Manu says:

    Hi,

    thanks very much for this code. Working perfectly for me on a uiimage object which I use to display a tiled map (like OpenStreetMap).
    Since I needed more possible interactions than just the default MATLAB “ImageClicked” callback, this came as the perfect solution!

    Thanks again, great work.

Leave a Reply

Your email address will not be published. Required fields are marked *