Customizing uifigures part 2
Posted By Yair Altman On September 7, 2016 | 5 Comments
I would like to introduce guest blogger Iliya Romm of Israel’s Technion Turbomachinery and Heat Transfer Laboratory [1]. Today Iliya will discuss how Matlab’s new web-based figures can be customized with user-controlled CSS and JavaScript code.
When we compare the documented properties of a “classic” uicontrol [2] with an App Designer control such as uicheckbox [3], we see lists of 42 and 15 properties, respectively. At first glance, this implies that our ability to customize App Designer elements is relatively very limited. This is surely a disquieting conclusion, especially for those used to being able to change most aspect of their Matlab figures via Java. Fortunately, such a conclusion is quite far from reality, as we will shortly see.
To understand this claim, we need to consider a previous post on this blog [4], where Yair discussed how uifigures are actually HTML webpages rendered by Matlab. As such, they have a DOM that can be accessed and manipulated through JavaScript commands to achieve various visual customizations. Today we’ll explore the structure of the uifigure webpage; take a look at some possibilities provided by the Dojo Toolkit; and see how to use Dojo to customize uifigure controls visually using CSS styles and/or HTML attributes.
[5]User customizations of Matlab uifigures (click to zoom-in)
A brief introduction to CSS
CSS stands for Cascading Style Sheets. As described on the official webpage [6] of W3C (which governs web standards):
CSS is the language for describing the presentation of Web pages, including colors, layout, and fonts. CSS is independent of HTML. This is referred to as the separation of structure (or: content) from presentation.
CSS rules (or “styles”) can be defined in one of three places:
- A separate file, such as the main.css that Matlab uses for uifigures (this file is found minified in %matlabroot%\toolbox\matlab\uitools\uifigureappjs\release\gbtclient\css)
- An inline block inside the HTML’s
<head>section - Directly within a DOM node
Deciding which of the above to use, is largely a choice of the right tool for the job. Usually, the first two choices should be preferred, as they adhere to the “separation of structure and presentation” idea better. However, in the scope of this demonstration, we’ll be using mostly the 3rd option, because it allows us not to worry about possible CSS precedence issues (suggested read [7]).
The syntax of CSS is generally: selector { property: value }, but it can have other forms [8] as well.
Getting down to business
Let us consider a very basic uifigure that only contains a uitextarea [9] and its label:
The auto-generated code for it is:
classdef DOMdemo < matlab.apps.AppBase
% Properties that correspond to app components
properties (Access = public)
UIFigure matlab.ui.Figure % UI Figure
LabelTextArea matlab.ui.control.Label % Text Area
TextArea matlab.ui.control.TextArea % This is some text.
end
methods (Access = private)
% Code that executes after component creation
function startupFcn(app)
end
end
% App initialization and construction
methods (Access = private)
% Create UIFigure and components
function createComponents(app)
% Create UIFigure
app.UIFigure = uifigure;
app.UIFigure.Position = [100 100 280 102];
app.UIFigure.Name = 'UI Figure';
setAutoResize(app, app.UIFigure, true)
% Create LabelTextArea
app.LabelTextArea = uilabel(app.UIFigure);
app.LabelTextArea.HorizontalAlignment = 'right';
app.LabelTextArea.Position = [16 73 62 15];
app.LabelTextArea.Text = 'Text Area';
% Create TextArea
app.TextArea = uitextarea(app.UIFigure);
app.TextArea.Position = [116 14 151 60];
app.TextArea.Value = {'This is some text.'};
end
end
methods (Access = public)
% Construct app
function app = DOMdemo()
% Create and configure components
createComponents(app)
% Register the app with App Designer
registerApp(app, app.UIFigure)
% Execute the startup function
runStartupFcn(app, @startupFcn)
if nargout == 0
clear app
end
end
% Code that executes before app deletion
function delete(app)
% Delete UIFigure when app is deleted
delete(app.UIFigure)
end
end
end
Let's say we want to modify certain aspects of the TextArea widget, such as the text color, background, and/or horizontal alignment. The workflow for styling elements involves:
- Find the handle to the webfigure
- Find the DOM node we want to modify
- Find the property name that corresponds to the change we want
- Find a way to manipulate the desired node from Matlab
Step 1: Find the handle to the webfigure
The first thing we need to do is to strategically place a bit of code that would allow us to get the URL of the figure so we can inspect it in our browser:
function startupFcn(app)
% Customizations (aka "MAGIC GOES HERE"):
warning off Matlab:HandleGraphics:ObsoletedProperty:JavaFrame
warning off Matlab:structOnObject
while true
try
win = struct(struct(struct(app).UIFigure).Controller).Container.CEF;
disp(win.URL);
break
catch
disp('Not ready yet!');
pause(0.5); % Give the figure (webpage) some more time to load
end
end
end
This code waits until the page is sufficiently loaded, and then retrieve its local address (URL). The result will be something like this, which can be directly opened in any browser (outside Matlab):
http://localhost:31415/toolbox/matlab/uitools/uifigureappjs/componentContainer.html?channel=/uicontainer/861ef484-534e-4a50-993e-6d00bdba73a5&snc=88E96E
Step 2: Find the DOM node that corresponds to the component that we want to modify
Loading this URL in an external browser (e.g., Chrome, Firefox or IE/Edge) enables us to use web-development addins (e.g., FireBug) to inspect the page contents (source-code).
Note: in recent Matlab releases, the URL cannot be directly opened in a browser unless we first set Matlab to enable this (thanks to Perttu [10]). We can turn on such browser debugging by running:
matlab.internal.webwindow(' ', 'DebugPort',4040) %note: the space is important!
before any other App, or kill all MatlabWindow processes (from the operating system's task manager) before running the command. Then browse http://localhost:4040/ in your browser - this will display a list of links, one link for each uifigure, based on the uifigure's title (name); click the requested uifigure's link to see and debug the figure's HTML/CSS contents.
Opening the URL inside a browser and inspecting the page contents, we can see its DOM:
[11]Inspecting the DOM in Firefox (click to zoom-in)
Notice the three
data-tag entries marked by red frames. Any idea why there are exactly three nonempty tags like that? This is because our App Designer object, app, contains 3 declared children, as defined in:
createComponents(app):
app.UIFigure = uifigure;
app.LabelTextArea = uilabel(app.UIFigure);
app.TextArea = uitextarea(app.UIFigure);
... and each of them is assigned a random hexadecimal id whenever the app is opened.
Finding the relevant node involved some trial-and-error, but after doing it several times I seem to have found a consistent pattern that can be used to our advantage. Apparently, the nodes with data-tag are always above the element we want to style, sometimes as a direct parent and sometimes farther away. So why do we even need to bother with choosing more accurate nodes than these "tagged" ones? Shouldn't styles applied to the tagged nodes cascade down to the element we care about? Sure, sometimes it works like that, but we want to do better than "sometimes". To that end, we would like to select as relevant a node as possible.
Anyway, the next step in the program is to find the data-tag that corresponds to the selected component. Luckily, there is a direct (undocumented) way to get it:
% Determine the data-tag of the DOM component that we want to modify:
hComponent = app.TextArea; % handle to the component that we want to modify
data_tag = char(struct(hComponent).Controller.ProxyView.PeerNode.getId); % this part is generic: can be used with any web-based GUI component
Let's take a look at the elements marked with blue and green borders (in that order) in the DOM screenshot. We see that the data-tag property is exactly one level above these elements, in other words, the first child of the tagged node is an element that contains a widgetid property. This property is very important, as it contains the id of the node that we actually want to change. Think pointers. To summarize this part:
We shall use this transformation in Step 4 below.
I wanted to start with the blue-outlined element as it demonstrates this structure using distinct elements. The green-outlined element is slightly strange, as it contains a
widgetid that points back to itself. Since this obeys the same algorithm, it's not a problem.
Step 3: Find the CSS property name that corresponds to the change we want
There is no trick here: it's just a matter of going through a list of CSS properties [12] and choosing one that "sounds about right" (there are often several ways to achieve the same visual result with CSS). After we choose the relevant properties, we need to convert them to camelCase as per documentation of dojo.style() [13]:
If the CSS style property is hyphenated, the JavaScript property is camelCased. For example: "font-size" becomes "fontSize", and so on.
Note that Matlab R2016a comes bundled with Dojo v1.10.4, rev. f4fef70 [14] (January 11 2015). Other Matlab releases will probably come with other Dojo versions. They will never be the latest version of Dojo, but rather a version that is 1-2 years old. We should keep this in mind when searching the Dojo documentation. We can get the current Dojo version as follows:
>> f=uifigure; drawnow; dojoVersion = matlab.internal.webwindowmanager.instance.windowList(1).executeJS('dojo.version'), delete(f)
dojoVersion =
{"major":1,"minor":10,"patch":4,"flag":"","revision":"f4fef70"}
This tells us that Dojo 1.10.4.f4fef70 is the currently-used version. We can use this information to browse the relevant documentation branch, as well as possibly use different Dojo functions/features.
Step 4: Manipulate the desired element from Matlab
In this demo, we'll use a combination of several commands:
- {matlab.internal.webwindow.}executeJS() – For sending JS commands to the uifigure.
- dojo.query() [15] – for finding nodes inside the DOM.
- dojo.style() [16] (deprecated since v1.8) – for applying styles to the required nodes of the DOM.
Syntax: dojo.style(node, style, value); - dojo.setAttr [17] (deprecated since v1.8) – for setting some non-style attributes.
Syntax: dojo.setAttr(node, name, value);
Consider the following JS commands:
- search the DOM for nodes having a data-tag attribute having the specified value, take their first child of type
<div>, and return the value of this child'swidgetidattribute:['dojo.getAttr(dojo.query("[data-tag^=''' data_tag '''] > div")[0],"widgetid")'] - search the DOM for nodes with id of
widgetid, then take the first element of the result and set its text alignment:['dojo.style(dojo.query("#' widgetId(2:end-1) '")[0],"textAlign","center")'] - append the CSS style defined by
{SOME CSS STYLE}to the page (this style can later be used by nodes):['document.head.innerHTML += ''''']);
Putting it all together
It should finally be possible to understand the code that appears in the animated screenshot at the top of this post:
%% 1. Get a handle to the webwindow:
win = struct(struct(struct(app).UIFigure).Controller).Container.CEF;
%% 2. Find which element of the DOM we want to edit (as before):
data_tag = char(struct(app.TextArea).Controller.ProxyView.PeerNode.getId);
%% 3. Manipulate the DOM via a JS command
% ^ always references a class="vc-widget" element.
widgetId = win.executeJS(['dojo.getAttr(dojo.query("[data-tag^=''' data_tag '''] > div")[0],"widgetid")']);
% Change font weight:
dojo_style_prefix = ['dojo.style(dojo.query("#' widgetId(2:end-1) '")[0],'];
win.executeJS([dojo_style_prefix '"fontWeight","900")']);
% Change font color:
win.executeJS([dojo_style_prefix '"color","yellow")']);
% Add an inline css to the HTML :
win.executeJS(['document.head.innerHTML += ''''']);
% Add animation to control:
win.executeJS([dojo_style_prefix '"-webkit-animation","mymove 5s infinite")']);
% Change Dojo theme:
win.executeJS('dojo.setAttr(document.body,''class'',''nihilo'')[0]');
% Center text:
win.executeJS([dojo_style_prefix '"textAlign","center")']);
A similar method for center-aligning the items in a uilistbox [18] is described here [19] (using a CSS text-align directive).
The only thing we need to ensure before running code that manipulates the DOM, is that the page is fully loaded. The easiest way is to include a pause() of several seconds right after the createComponents(app) function (this will not interfere with the creation of the uifigure, as it happens on a different thread). I have been experimenting with another method [20] involving webwindow's PageLoadFinishedCallback callback, but haven’t found anything elegant yet.
A few words of caution
In this demonstration, we invoked Dojo functions via the webwindow's JS interface. For something like this to be possible, there has to exist some form of “bridge” that translates Matlab commands to JS commands issued to the browser and control the DOM. We also know that this bridge has to be bi-directional, because binding Matlab callbacks to uifigure actions (e.g. ButtonPushFcn for uibuttons) is a documented feature.
The extent to which the bridge might allow malicious code to control the Matlab process needs to be investigated. Until then, the ability of webwindows to execute arbitrary JS code should be considered a known vulnerability. For more information, see XSS [21] and related [22] vulnerabilities.
Final remarks
It should be clear now that there are actually lots of possibilities afforded by the new uifigures for user customizations. One would hope that future Matlab releases will expose easier and more robust hooks for CSS/JS customizations of uifigure contents. But until that time arrives (if ever), we can make do with the mechanism shown above.
Readers are welcome to visit the GitHub project [23] dedicated to manipulating uifigures using the methods discussed in this post. Feel free to comment, suggest improvements and ideas, and of course submit some pull requests 🙂
p.s. – it turns out that uifigures can also display MathML [24]. But this is a topic for another post...
Categories: Figure window, Guest bloggers, GUI, High risk of breaking in future versions, Undocumented feature
5 Comments (Open | Close)
Article printed from Undocumented Matlab: https://undocumentedmatlab.com
URL to article: https://undocumentedmatlab.com/articles/customizing-uifigures-part-2
URLs in this post:
[1] Turbomachinery and Heat Transfer Laboratory: http://bcukurel.net.technion.ac.il
[2] uicontrol: http://www.mathworks.com/help/matlab/ref/uicontrol-properties.html
[3] uicheckbox: http://www.mathworks.com/help/matlab/ref/checkboxappdesigner-properties.html
[4] previous post on this blog: http://undocumentedmatlab.com/blog/customizing-uifigures-part-1
[5] Image: http://undocumentedmatlab.com/images/uifigure_css1.gif
[6] the official webpage: https://www.w3.org/standards/webdesign/htmlcss#whatcss
[7] suggested read: http://vanseodesign.com/css/css-specificity-inheritance-cascaade/
[8] other forms: http://www.tutorialspoint.com/css/css_syntax.htm
[9] uitextarea: http://www.mathworks.com/help/matlab/ref/uitextarea.html
[10] thanks to Perttu: https://undocumentedmatlab.com/articles/matlab-callbacks-for-uifigure-javascript-events#comment-454673
[11] Image: http://undocumentedmatlab.com/images/uifigure_css3.png
[12] a list of CSS properties: http://www.w3schools.com/cssref/default.asp
[13] dojo.style(): https://dojotoolkit.org/reference-guide/1.7/dojo/style.html#tips
[14] f4fef70: https://github.com/dojo/dojo/commit/f4fef7077dde783156c1ddd3b06464d236e5537c
[15] dojo.query(): https://dojotoolkit.org/reference-guide/1.10/dojo/query.html
[16] dojo.style(): https://dojotoolkit.org/reference-guide/1.7/dojo/style.html
[17] dojo.setAttr: https://dojotoolkit.org/reference-guide/1.7/dojo/setAttr.html
[18] uilistbox: http://www.mathworks.com/help/matlab/ref/uilistbox.html
[19] described here: http://stackoverflow.com/questions/38933254/how-to-customize-app-designer-figures-in-more-ways-than-officially-documented
[20] another method: https://github.com/StackOverflowMATLABchat/mlapptools/issues/1#issuecomment-241252137
[21] XSS: https://en.wikipedia.org/wiki/Cross-site_scripting
[22] related: https://en.wikipedia.org/wiki/Cross-site_scripting#Related_vulnerabilities
[23] GitHub project: https://github.com/StackOverflowMATLABchat/mlapptools
[24] MathML: https://www.w3.org/Math/
[25] Customizing uifigures part 3 : https://undocumentedmatlab.com/articles/customizing-uifigures-part-3
[26] Customizing uifigures part 1 : https://undocumentedmatlab.com/articles/customizing-uifigures-part-1
[27] Customizing web-GUI uipanel : https://undocumentedmatlab.com/articles/customizing-web-gui-uipanel
[28] Matlab callbacks for uifigure JavaScript events : https://undocumentedmatlab.com/articles/matlab-callbacks-for-uifigure-javascript-events
[29] Customizing help popup contents : https://undocumentedmatlab.com/articles/customizing-help-popup-contents
[30] Customizing uitree nodes – part 1 : https://undocumentedmatlab.com/articles/customizing-uitree-nodes
Click here to print.
5 Comments To "Customizing uifigures part 2"
#1 Comment By Felix On May 23, 2017 @ 10:56
Hello Iliya & Yair,
did you already try it in R2017a?
It seems that the ‘Container’ property of the figure controler is not longer available.
#2 Comment By Yair Altman On June 29, 2017 @ 00:43
@Felix – this works in R2017a:
#3 Comment By Vishal Sharma On November 4, 2019 @ 06:22
Great help. thanks. I was struggling with Appdesigner, but this post and the links given for mlapp,solved quite a lot of issues.
#4 Comment By João Neves On March 20, 2023 @ 12:47
I am on matlab 2021a, this still works:
url = struct(struct(struct(struct(hFig).Controller).PlatformHost).CEF).URL;but the html document is empty. Is there still a way to do this?
#5 Comment By Yair Altman On April 11, 2023 @ 12:42
To debug uifigures in an external browser (e.g. to see the internal HTML/CSS) we need to first set Matlab to enable such debugging. I updated Step #2 above with the relevant information for this.