Don’t fix it, if it ain’t broke…

In Applied Materials Israel (PDC) we use Matlab code for both algorithm development and deployment (production). As part of the dev-ops build system, which builds our product software versions, we build Matlab artifacts (binaries) from the Matlab source code.

A typical software version has several hundreds Matlab artifacts that are automatically rebuilt on a daily basis, and we have many such versions – totaling many thousands of compilations each day.

This process takes a long time, so we were looking for a way to make it more efficient.

The idea that we chose to implement sounds simple – take a single binary module in any software version (Ex. foo.exe – Matlab-compiled exe) and check it: if the source code for this module has not changed since the last compilation then simply don’t compile it, just copy it from previous software version repository. Since most of our code doesn’t change daily (some of it hasn’t changed in years), we can skip the compilation time of most binaries and just copy them from some repository of previously compiled binaries.

In a broader look, avoiding lengthy compilations cycles by not compiling unchanged code is a common programming practice, implemented by all modern compilers. For example, the ‘make’ utility uses a ‘makefile’ to check the time stamps of all dependencies of every object file in order to decide which object requires recompilation. In reality, this is not always the best solution as time stamps may be incorrect, but it works well in the vast majority of cases.

Coming back to Matlab, now comes the hard part – how could our build system know that nothing has changed in module X and that something has changed in module Y? How does it even know which source files it needs to ensure didn’t change?

The credit for the idea goes to my manager, Lior Cohen, as follows: You can actually check the dependency of a given binary after compilation. The basis of the solution is that a Matlab executable is in fact a compressed (zip) file. The idea is then to:

1. Compile the binary once
2. Unzip the binary and “see” all your dependencies (source files are encrypted and resources are not, but we only need the list of file names – not their content).
3. Now build a list of all your dependency files and compute the CRC value of each from the source control. Save it for the next time you are required to compile this module.
4. In the next compilation cycle, find this dependency list, review it, dependency source file at a time and make sure CRC of the dependency hasn’t changed since last time.
5. If no dependency CRC has changed, then copy the binary from the repository of previous software version, without compiling.
6. Otherwise, recompile the binary and rebuild the CRC list of all dependencies again, in preparation for the next compilation cycle.

That’s it! That simple? Well… not really – the reality is a bit more complex since there are many other dependencies that need to be checked. Some of them are:

1. Did the requested Matlab version of the binary change since the last compilation?
2. Did the compilation instructions themselves (we have a sort of ‘makefile’) change?

Basically, I implemented a policy that if anything changed, or if the dependency check itself failed, then we don’t take any chances and just compile this binary. Keeping in mind that this dependencies check and file copying is much faster than a Matlab compilation, we save a lot of actual compilation time using this method.

Bottom line: Given a software version containing hundreds of compilation instructions to execute and assuming not much has changed in the version (which is often the case), we skip over 90% of compilations altogether and only rebuild what really changed. The result is a version build that takes about half an hour, instead of many hours. Moreover, since the compilation process is working significantly less, we get fewer failures, fewer stuck or crashed mcc processes, and [not less importantly] less maintenance required by me.

Note that in our implementation we rely on the undocumented fact that Matlab binaries are in fact compressed zip archives. If and when a future Matlab release will change the implementation such that the binaries will no longer be zip archives, another way will need to be devised in order to ensure the consistency of the target executable with its dependent source files.

Don’t kill it, if it ain’t bad…

I want to share a very weird issue I investigated over a year ago when using Matlab compiled exe. It started with a user showed me a Matlab compiled exe that didn’t run – I’m not talking about a regular Matlab exception: the process was crashing with an MS Windows popup window popping, stating something very obscure.

It was a very weird behavior that I couldn’t explain – the compiler seemed to work well but the compiled executable process kept crashing. Compiling completely different code showed the same behavior.

This issue has to do with the system compiler configuration that is being used. As you might know, when installing the Matlab compiler, before the first compilation is ever made, the user has to state the C compiler that the Matlab compiler should use in its compilation process. This is done by command ‘mbuild –setup’. This command asks the users to choose the C compiler and saves the configuration (batch file back then, xml in the newer versions of Matlab) in the user’s prefdir folder. At the time we were using Microsoft Visual C++ compiler 9.0 SP1.

The breakthrough in the investigation came when I ran mcc command with –verbose flag, which outputs much more compilation info than I would typically ever want… I discovered that although the target executable file had been created, a post compilation step failed to execute, while issuing a very cryptic error message:

mt.exe : general error c101008d: Failed to write the updated manifest to the resource of file “…”. Access is denied.

cryptic compilation error (click to zoom)

The failure was in one of the ‘post link’ commands in the configuration batch file – something obscure such as this:

set POSTLINK_CMDS2=mt.exe -outputresource: %MBUILD_OUTPUT_FILE_NAME%;%MANIFEST_RESOURCE% -manifest "%MANIFEST_FILE_NAME%"

This line of code takes an XML manifest file and inserts it into the generated binary file (additional details).

If you open a valid R2010a (and probably other old versions as well) Matlab-generated exe in a text editor you can actually see a small XML code embedded in it, while in a non-functioning exe I could not see this XML code.

So why would this command fail?

It turned out, as funny as it sounds, to be an antivirus issue – our IT department updated its antivirus policies and this ‘post link’ command suddenly became an illegal operation. Once our IT eased the policy, this command worked well again and the compiled executables stopped crashing, to our great joy.

I work for Applied Materials Israel (PDC) in the algorithm development department. My group provides Matlab software solutions across all of our products. I am a big fan of Yair’s blog and was happy to receive an invitation to be a guest blogger.

In PDC we are using Matlab compiler and Java Builder to deploy our algorithmic products at the customer sites. The software we produce includes binaries of three sorts: exe, C++ dlls, and jar files, all compiled using Matlab deployment tools.

C++ dlls are not as common and are a relatively new kind of binary in production so there’s not as much experience using them in PDC and from time to time we discover weird behaviors that might be interesting to the readers. Here are three of the latest quirks that we encountered:

Leftover tmp files

Recently I faced a problem that googling revealed that I might be the first to encounter it (outside MathWorks development team) as there was no mention of this anywhere. It all started when I got a mail from a fellow developer with content of the type: “Hi Hanan, what are these dump files for ???” and a screenshot of tempdir showing multiple tmp files that all had a naming convention of ‘mathworks_tmp_XXX_YYY’ (xxx and yyy being some numbers, presumably process ids):

multiple leftover tmp files in tmpdir (click for details)

Each file was about 43MB in size, and combined they consumed the entire free space in the hard-disk, preventing applications from lunching.

At first I looked in our code for the code that will produce files with this naming convention and once I was sure there is no such code I was comfortable to assume that they are MathWorks internal-use files (indeed – who other than MathWorkers would call their files mathworks_tmp_…).

Compiling and running a small ‘hello world’ C++ dll showed that every time that a Matlab-compiled C++ dll executed and ran, a temp file is created and then deleted in the %tmp% dir (tempdir). If the process that runs this dll is ‘killed’ during the run, the temp file is not deleted and remains in the %tmp% folder, accumulating over time until the hard disk becomes completely full.

On another machine I found a whole bunch of different temp files with different extensions. Opening them in editor didn’t reveal their purpose. They all had two things in common:

1. All had a naming convention mathworks_tmp_XXX__YYY, whatever the extension of the file may be.
2. They are all useless when the Matlab application is done running.

Once this was clear, the solution was very simple:

• When lunching the process that runs Matlab compiled dll, change the %tmp% directory to a new one.
• Delete this directory once done running or on the next process start.

mclInitializeApplication

As a separate discussion, our compiled algorithm is embarrassingly parallel in nature so our C++ team is running it in parallel processes across several machines and many cores.

As with all Matlab-compiled C++ DLLs, it needs to be initialized with a call to mclInitializeApplication per running process. This worked well when we had a relatively small number of processes running concurrently but lately we encountered a problem that out of dozens of calls to this function from time to time a call is stuck (not failed – exactly that: stuck) and the process needs to be killed.

We don’t know why this happens and the solution we are currently using is pretty “shaky” at best – we retry the call to this function several times until it eventually works (and it does work after several tries).

I have a suspicion that something is not entirely parallel with this function and there exists some hidden internal shared resource among different processes but no way to know exactly as this is an internal MathWorks function.

Implicit multithreading

This is relatively old issue we encountered at the beginning when we just started using Matlab-compiled DLLs, so this might not surprise some readers. While Matlab’s builtin (automatic) multithreading is great when used in the MATLAB IDE (non-compiled), it creates a problem when running compiled DLLs in parallel across multiple processes: this multithreading ‘starves’ for cores because they are busy at running other processes.

The solution is simple and well documented – when calling mclInitializeApplication before launching the DLL, pass it the singleCompThread flag to prevent implicit multithreading. We found this single threaded DLL to run faster because it reduced CPU ‘starvation’ drastically in a multi process environment.

Addendum (by Yair again):

I am delighted to announce that a few days ago I received an apparently-automated email from MathWorks telling me that 4 of the issues that I have reported early last year were fixed in R2015b. IMHO, this marks a very important step of closing the loop with the original issue reporter, something that I have suggested publicly back in December 2014 (and also privately over the years). I believe that such automated feedbacks increase user motivation to report misbehaving or missing features, which will ultimately make Matlab better for the benefit of us all. I also believe that this proves once again my claim that beneath the corporate jargon, MathWorks is indeed a company run by engineers like us, who cares about its users and the interaction with them more than typical in the corporate world. I can only praise this email and hope that it is now part of the ongoing development release process, rather than being an isolated case. If I may suggest a followup improvement, it would be nice to receive such emails before the new release is out (i.e., during the beta phase) so that the original reporter could have a chance to test it on the beta before it actually goes live. But in any case, thanks MathWorks for listening and making yet another improvement

Happy New Year of the Monkey everybody!

Error using strtrim
Input should be a string or a cell array of strings.
Error in updateGUI (line 121)

Sounds simple enough to debug right? Just go to updateGUI.m line #121 and fix the call to strtrim(), correct?

Well, not so fast… It turns out that updateGUI.m line #121 is an empty line surrounded on either side by one-line comments. This is certainly not the line that caused the error. The actual call to strtrim() only occurs in line #147!

What’s going on?

The answer is that the Matlab compiler has a bug with comment blocks – lines surrounded by %{ and %}:

15   %{
16       This is a comment block that is
17       ignored by the Matlab engine,
18       and causes a line-numbering
19       error in the Matlab compiler!
20   %}
21 
22   a = strtrim(3.1416);  % this generates an error

In the example above, the 6-line comment block is ignored as expected by the Matlab engine in both interactive and compiled modes. However, whereas in interactive mode the error is correctly reported for line #22, in compiled mode it is reported for line #17. Apparently the compiler replaces any block comment with a single comment line before parsing the m-file.

The workaround is simple: count all the block-comment lines that precede the reported error line in the original m-file, and add that number to the reported line. In the example above, 17 + (6-1) = 22. Note that the source code could have multiple block-comments that precede the reported line, and they should all be counted. Here is a basic code snippet that does this parsing and opens the relevant m-file at the correct location:

% Read the source file (*.m)
str = fileread(filename,'*char');
 
% Split the contents into separate lines
lines = strtrim(strsplit(str',10));
 
% Search for comment block tokens
start_idx = find(strcmp(lines,'%{'));
end_idx   = find(strcmp(lines,'%}'));
 
% Count the number of block-comment lines in the source code
delta_idx = end_idx - start_idx;
relevant_idx = sum(start_idx < reported_line_number);
total_comment_block_lines = sum(delta_idx(1:relevant_idx));
 
% Open the source file at the correct line
opentoline(filename, reported_line_number + total_comment_block_lines);

This code snippet can fairly easily be wrapped in a stand-alone utility by anyone who wishes to do so. You’d need to generate the proper full-path source-code (m-file) filename of course, since this is not reported by the deployed application in its error message. You’d also need to take care of edge-cases such as p-coded or mex files, or missing source-code m-files. You’d also need to parse the input parameters (presumably filename and reported_line_number, but possibly also other inputs).

This is another example of a bug that does not officially exist in Matlab’s public bug parade (at least, I couldn’t find it). I’ve reported other such undocumented bugs in previous articles (here and here). Please don’t start another comments tirade on MathWorks’ bug-publication policies. I think that issue has already been discussed ad nauseam.

Update: according to Tom’s comment below, this bug was apparently fixed in R2012b.

Another related bug related to block comments is that at least on some Matlab releases (I haven’t tested properly to determine which releases), block comments are NOT properly ignored by the Editor’s publish functionality. Instead, at least on those releases where the bug occurs, publishing code that includes block comments parses the block’s contents as if it was runnable code. In other words, publish treats %{ and %} as one-line comments rather than as tokens marking the ends of a block comment.

Reminder: I will be visiting several US cities (Minneapolis, Boston and New York) in July 22-31 (details). Let me know if you’d like to meet me there.

Introduction

Using a pure-Java Swing-based Graphical User Interface (GUI) has several important advantages compared to using a pure-Matlab GUI:

• Java GUI widget toolkit provides a large collection of components (scrollbar, slider, etc…) and layouts (card, spring, etc…)
• Multiple event handling options
• Mature third-party Java Swing IDEs for quick and easy GUI development (Netbeans, Eclipse, etc…)
• Supports pluggable look and feel that allows applications to have a look and feel that is unrelated to the underlying platform.
• Java Swing’s window icon can be modified, whereas Mathworks apparently places an [entirely unreasonable] license restriction on modifying the Matlab figure window icon.

When Matlab is in development (non-deployed) mode, we can load and display a Java JFrame using the following Matlab code:

% Add Java library to dynamic Java classpath
javaaddpath([pwd '\ExampleWindow.jar']);
 
% Get example Java window from the library
jFrame = examplewindow.JavaWindow();
 
% Get Java buttons
% Note: see https://undocumentedmatlab.com/blog/matlab-callbacks-for-java-events-in-r2014a
plotMeshButton = handle(jFrame.getPlotMeshButton(),    'CallbackProperties');
showWarnButton = handle(jFrame.getShowWarnDlgButton(), 'CallbackProperties');
 
% Set Java button callbacks
set(plotMeshButton, 'ActionPerformedCallback', @myPlotMeshCallback);
set(showWarnButton, 'ActionPerformedCallback', @myShowWarnCallback);
 
% Display the Java window
jFrame.setVisible(true);

Java JFrame created in Matlab (click for full-size image)

The problem

All this works great when ran within the Matlab environment. However, when the Matlab source code file (.m) is compiled as a Windows standalone executable application, when running the resulting executable the Java window appears for a short period and then terminates (exits). Without a visible Matlab figure, the compiled Matlab application does not retain the Java window open. The JFrame flashes on for a split-second, and then vanishes.

The solution

To fix this problem, it turns out that we only need to add the Matlab waitfor command after making the Java window visible:

% (earlier code as above)
 
% Display the Java window
jFrame.setVisible(true);
if isdeployed
    waitfor(jFrame);
end

Java JFrame in deployed Matlab (click for full-size image)

The waitfor command holds the Java window open, but still enables full interactivity with the window: When a control event is triggered, the specified callback is executed, and then Matlab returns to the waitfor.

A zip file containing the Matlab source-code and its resulting executable can be downloaded from here. You might get a security notification when downloading the zip file, due to the contained exe file. Also note that to run the executable, you need to install Matlab MCR R2012b (8.0) or later, otherwise you will get a run-time error.

Finally, note that closing the Java GUI exits the JVM and by extension the entire containing program (Matlab). Naturally, this has no impact in deployed mode, but to prevent it from exiting the Matlab environment in non-deployed mode, don’t close the Java window by clicking its “X” button etc., but rather by calling jFrame.dispose(); in your Matlab code or the Matlab Desktop’s Command Window.

As Yair pointed out in many occasions, the power of Matlab could be greatly enhanced using the underlying Java mechanism. To me, while developing a larger standalone tool for technical calculations, these hints are worth a mint (as I guess for many of you).

One of these very useful hints is the ability to dock figure windows in standalone applications. This perfectly fits to my understanding of a “clean desktop”, i.e., having as less as possible separate windows. Since in many calculations dozens of figures are generated, the desktop gets up crowded very fast – if these are not grouped. So docking is essential (at least for me). Unfortunately there seems to be a serious problem with the resulting menu entries (at least in R2011b on Win XP), leading to a crash of the standalone application. Based on the posts by Yair, I will sketch a possible route to avoid this issue.

The symptom

In the compiled application, docking could be accomplished by accessing the figure frame’s underlying Java level:

% get java frame for sophisticated modifications
jframe = get(handle(figure_handle), 'JavaFrame');
 
% allow docking
jframe.fHG1Client.setClientDockable(true)

Using this modification, the user is now allowed to dock and undock the figures manually. For initial docking of the figure,

javaFrame.fHG1Client.setClientWindowStyle(true,false)

could be used during figure creation. Unfortunately, there are menu entries in the Figures container which are either unwanted (since not usable) or even directly crash the standalone applications:

Useless Debug menu items in deployed applications

Menu items crashing deployed applications

Since crashing menu entries will be found and used by end-users (though these are somehow hidden), these prohibit the usage of the docking feature as long as these could be invoked. So how can we disable / remove these menu items?

The unsuccessful solution

Unfortunately, the straight forward solution of getting the handle to the Figures containers’ menu bar and remove the unwanted items does not work. The reason for this is the (to me unexpected behavior) that the menu bar seems to be rebuilt whenever a figure is docked/undocked.

This is actually the same behavior that automatically rebuilds the Editor’s menu bar whenever an editor file is added/removed. The Editor container is basically the same docking container as the Figures container, as shown by Yair’s setFigDockGroup utility:

Docking a figure in the Editor container (group)

Therefore, removing unwanted menu items only helps until the next figure docking/undocking. To make it even worse: also pressing any of the buttons within the document bar (if having more than one figure) somehow rebuilds the entire menu structure, reverting our changes. So the solution becomes a bit more complex.

The working solution

For the working solution, many pieces presented by Yair should be put together. The first piece results from the question how to detect a dock/undock event. Since no such callback is defined, we need to use a property listener as Yair showed in his post about the continuous slider callback:

% listen to the WindowStyle property to detect docking / undocking events
hProp = findprop(handle(figure_handle),'WindowStyle');  % a schema.prop object
 
% if the event occurs, invoke the callback
hlistener = handle.listener(handle(figure_handle), hProp, 'PropertyPostSet',{@(source, event) Callback_DockingFcn});
 
% attach listener to the GUI since it needs to be known (as long as the figure exists)
setappdata(figure_handle, 'Handle_Listener', hlistener);

Now, whenever the figure’s WindowStyle property (which controls the docking state) is changed, our docking callback is invoked.

The next piece of the puzzle takes care of the menu rebuild whenever any document bar button is pressed. To overcome this behavior, the idea is to define the MousePressed callback of theses buttons to (again) invoke the docking callback. This is necessary for two reasons: First, pressing the button (i.e., changing the current figure) rebuilds the menu, overwriting our changed menu entries. Secondly, all other buttons are also somehow rebuilt and the callbacks are removed if a new figure is docked.

The handles to the document bar buttons could be found using Yair’s findjobj utility. We have already seen that the Editor container is analogous to the Figures docking container. So let’s use the method described by Yair for accessing the Editor container, to access the Figures container:

figures_container = javaObjectEDT(matlab_instance.getGroupContainer('Figures'));
figures_frame = javaObjectEDT(figures_container.getTopLevelAncestor);

Once we get the Java Frame for the Figures container, the buttons could be found by digging through its children. This finally allows to set the callback using

DTDocumentBar = javaObjectEDT(figures_frame.getRootPane.getLayeredPane.getComponent(1).getComponent(1).getComponent(0).getComponent(0).getComponent(1).getComponent(0));
ContentPanel = javaObjectEDT(DTDocumentBar.getComponent(0).getComponent(0).getViewport.getView);
 
if ~isempty(ContentPanel.getComponents) % less than two documents are open and no DTDocumentbar exists
    drawnow; pause(0.05)
    GroupPanel = javaObjectEDT(ContentPanel.getComponent(0));
    GroupPanel_Elements = javaObjectEDT(GroupPanel.getComponents);
 
    % change the MousePressed Callback for each of the buttons to invoke the function which disables the menu
    for n = 1 : GroupPanel.getComponentCount
        thisElement = GroupPanel_Elements(n);
        if isequal(char(thisElement.getClass.toString), 'class com.mathworks.widgets.desk.DTDocumentBar$DocumentButton')
            set(handle(thisElement, 'CallbackProperties'), 'MousePressedCallback', {@(source, event) Cbrake_Callback_Diagrams_DockingFcn})
        end
    end
    drawnow; pause(0.05)
end

where the loop runs through the current buttons in the document bar.

As the last step of our procedure, we finally remove (or disable) the menu entries which are unwanted. This is achieved by extracting the handle to the Figures menu by:

figures_menu = javaObjectEDT(figures_frame.getJMenuBar);

Running through the menu items, searching for the unwanted entries (as long as they have pre-defined menu-item names) at the end sets us into the position to take care of the menu items:

% run through top-level menu items
for n = 1 : figures_menu.getMenuCount
    % completely deactivate Debugging options
    if isequal(char(figures_menu.getMenu(n-1).getName), 'DesktopDebugMenu')
        DesktopDebugMenuPos = n - 1;
    end
 
    % Remove some items from the Desktop menu
    if isequal(char(figures_menu.getMenu(n-1).getName), 'DesktopMenu')
        desktop_menu = javaObjectEDT(figures_menu.getMenu(n-1));
 
        DeletePos = [];
        for m = 1: desktop_menu.getMenuComponentCount
            if ismember({char(desktop_menu.getMenuComponent(m-1).getName)}, ...
                        {'ToggleFigure PaletteCheckBoxMenuItem', 'TogglePlot BrowserCheckBoxMenuItem', 'ToggleProperty EditorCheckBoxMenuItem'})
                DeletePos(end+1) = m - 1;
            end
        end
 
        for m = length(DeletePos) : -1 : 1
            desktop_menu.remove(DeletePos(m))
        end
    end
end
 
% finally remove the "Debug" menu
if ~isempty(DesktopDebugMenuPos)
    figures_menu.remove(DesktopDebugMenuPos)
end

Since this callback is invoked whenever a figure is docked/undocked, or the currently shown figure is changed (by pressing the document bar button), all unwanted menu items within the Figures menu could be removed.

As a result, the new Figures container menu looks like:

Deployed menu without unwanted items

Remarks

I must admit that above solution is still imperfect. For instance, sometimes there is a larger delay between the docking (or button press event) and the removing of the menu item. Nevertheless, this solution allows me to distribute my standalone with docked figures without having menu items directly leading to a fatal error.

Obviously, the solution has some positive side effects:

• As could be seen from the screen shot, the Matlab desktop becomes available also within your compiled applications. This might be wanted. If not, it could be removed the same way as the other menu items. One drawback of making the desktop available should be mentioned: In my tests, the standalone Matlab desktop shows the whole list of recent files I have in the Matlab editor at compile time. This is somehow ugly but not that problematic.
• Additional menu items could be added, giving more possibilities for modifications.

I have uploaded a first version of the docking and creation functions, together with a small test project, to the Matlab file Exchange. Readers are welcome to download the code and send me improvement suggestions. Or you could simply leave a comment below.

Of the numerous official resources containing unofficial information (I can’t help but smile), do you have a personal favorite? If so, please post a link in a comment below, with a short description of what it is about.

Today, I show one such technical solution (http://www.mathworks.com/support/solutions/en/data/1-1UOGW7/) that solves a privacy issue in deployed (compiled) Matlab applications. IMHO, the privacy issue is generally harmless, but users can judge by themselves. Obviously, if someone is stupid enough to store real secret information in the Matlab preferences, then the privacy issue could become more important.

The solution refers to Matlab release 7.1 (R14SP3), but is also relevant for later releases. I will let the text speak for itself:

When I execute an application compiled using MATLAB Compiler 4.3 (R14SP3), how can I prevent personal information such as my MATLAB preferences information from being extracted?

Problem Description:

I have compiled an application ‘test’ using MATLAB Compiler 4.3 (R14SP3). Upon execution I noticed that a new folder gets created in the ‘test_mcr’ folder. This folder is named ‘test_‘.

Opening this folder revealed that some of the files were stored in my MATLAB preferences directory, including my MATLAB preferences file.

I want to know if there is any way to avoid the creation of this directory when my application executes.

Solution:

It is not possible to avoid having MATLAB Compiler include the preferences directory in each compiled application. The Compiler includes the user’s preference information because the application may rely on it. There are two possible workarounds:

1) On UNIX, use SETENV and the ! operator to compile. For example:

!setenv MATLAB_PREFDIR /tmp/emptydir;
mcc -Nmgv test

On Windows, open a DOS command window and execute

set -- USERPROFILE = C:\Temp\emptydir

Start MATLAB from this DOS prompt (to use the appropriate environment variable in the MATLAB session) and execute the MCC command at the MATLAB command prompt.

Note: If you are using MATLAB Compiler 4.4 (R2006a) you can also use the SETENV function at the MATLAB command prompt

setenv('-- USERPROFILE','C:\Temp\emptydir');
mcc -Nmgv test

This temporarily sets the preference directory to /tmp/emptydir or C:\Temp\emptydir, and the compiled application will therefore only have the default settings (or the preference directory settings from your installation of MATLAB 7.0.4 (R14SP2)).

2) Move the contents of the preference directory, or rename the directory, before compilation, and restore afterwards. To do this on Windows, for example, execute the following command to compile the function test.m:

mymcc -m test.m

The code of the function mymcc is:

function mymcc(varargin)
% MYMCC does the same as MCC. The only difference is that the created
% application doesn't contain information of the MATLAB 
% prefdir anymore.
%
% For the description of possible options please look at the help of 
% the command MCC.
 
command = 'mcc ';
for i = 1 : length(varargin)
   command = [command,varargin{i},' '];
end
 
a = prefdir;
b = regexp(a,'\','start');
c = a(b(end)+1:end);
 
p = cd;
cd(prefdir)
cd ..
!md tmp
cd(prefdir)
!move * ../tmp
cd(p)
 
% Because of the displacement MATLAB cannot access the saved preferences
% and that's why you have to indicate each time, which compiler shall be used.
try
   eval(command);
catch
   warning('WarnTests:Warning', ...
           'Calling mcc causes Error. Please check your input arguments.\n No file was compiled ...\n')
end
 
cd(prefdir)
cd ..
cd tmp
eval(['!move * ../',c]);
cd ..
!rmdir tmp
cd(p);

A similar function could also be written for the UNIX platform.

Note that the above workaround is not officially supported by MathWorks.

Related Solutions:

Why are my preference files included in my stand-alone application created with MATLAB Compiler 4.9 (R2008b)?

Problem Description:
If I look in the cache folder that my compiled application creates, I can see that the .matlab folder contains my preference files with my last development activities in plain text. The files are: cwdhistory.m, history.m, matlab.prf, MATLAB_Editor_State.xml, etc. How can I avoid the inclusion of these files in the stand-alone application?

Solution:
The stand-alone does not include your preference files. You can see the preference files in the cache folder because you are running the stand-alone in your development environment. At runtime, the stand-alone determines those preferences from a local MATLAB installation and copies them to the cache folder for it’s own use. If you run your stand-alone application on a machine without a MATLAB installation, you will see that the cache folder does not contain your preference files.

The following has been reported to me by a reader of this blog. Please note that I cannot independently confirm the correctness of the report (if anyone can let me know I would be grateful):

Normally, the MCR and the stand-alone executable is unpacked upon every startup in the user’s temp dir, and deleted when the user logs out. Apparently, when the MCR_CACHE_ROOT environment variable is set, these files are only unpacked once and kept for later reuse. If this report is indeed true, this could significantly speed up the startup time of a compiled application in subsequent invocations.

On Linux:

export MCR_CACHE_ROOT=/tmp/mcr_cache_root_$-- USER   # local to host
mkdir -p @MCR_CACHE_ROOT
./myExecutable

On Windows:

REM set MCR_CACHE_ROOT=%TEMP%
set MCR_CACHE_ROOT="C:\Documents and Settings\Yair\Matlab Cache\"
myExecutable.exe

If you wish to set this env variable permanently on Windows, look at the explanation provided here.

Setting MCR_CACHE_ROOT is especially important when running the executable from a network (NFS) location, since unpacking onto a network location could be quite slow. If the executable is run in parallel on different machines (for example, a computer cluster running a parallel program), then this might even cause lock-outs when different clusters try to access the same network location. In both cases, the solution is to set MCR_CACHE_ROOT to a local folder (e.g., /tmp or %TEMP%). If you plan to reuse the extracted files again, then perhaps you should not delete the extracted files but reuse them. Otherwise, simply delete the temporary folder after the executable ends. In the following example, $RANDOM is a bash function that returns a random number:

export MCR_CACHE_ROOT=/tmp/mcr$RANDOM
./matlab_executable
rm -rf $MCR_CACHE_ROOT

Setting MCR_CACHE_ROOT can also be used to solve other performance bottlenecks in deployed applications, as explained in a MathWorks technical solution and a related article here.

In a related matter, compiled Matlab executables may fail with a Could not access the MCR component cache error, when Matlab cannot write in the MCR cache directory due to missing permission rights. This can be avoided by setting MCR_CACHE_ROOT to a non-existent directory, or to a folder in which there is global access permissions (/tmp or %TEMP% are usually such writable folders) – see related posts here and here.

I have seen many users double-click the application icon again and again in frustration, thinking that perhaps the application did not start when they first tried. In fact, this launches multiple applications and MCR loads at the same time, which only prolongs the overall time that it takes the application to actually load and display.

The “normal” solution for this would be to display a splash figure or start-up message at the very beginning of the application. Unfortunately, this doesn’t help because the vast majority of the launch time is due to MCR load time, which happens before any user code is executed. The result is that our application’s splash figure or msgbox will only be displayed after the very long time in which the application appears unresponsive.

Splash-window solution

I developed a splash-screen wrapper application for deployed applications that solves this problem on Windows platforms.

The idea is to create a stand-alone non-Matlab application that displays a user-specified image (splash) in a dedicated window at the center of the screen, and then immediately launches the requested deployed Matlab application in the background. The wrapper then waits for the Matlab application to load, and automatically closes the splash window when it detects that the target Matlab GUI has finally displayed.

The effect is that upon launching the application, the user sees immediate feedback in the form of a dedicated splash window (for example, displaying an image of the application or company logo with the message “Loading – please wait…”). This does not make the target application load any faster, but at least users are given immediate feedback that prevents them from re-launching the application. Users also report that this makes the application appear to load faster. This is due to a psychological effect called “perceived performance” – we also see this effect with phone ringtones and elevator floor-level indicators.

My splash wrapper application receives the following input parameters:

1. splash image filepath (BMP/JPG/GIF – note that other formats are NOT supported)
2. splash image width in pixels (will resize the image if necessary)
3. splash image height in pixels (will resize the image if necessary)
4. target window title string
5. title matching mode:
   • 1 = match at the beginning of the title (case sensitive)
   • 2 = match at any place within the title (case sensitive)
   • 3 = match the entire title (case sensitive)
   • -1, -2, -3 = case-insensitive version of the above values
6. target application filepath (this is the actual deployed application executable)
7. optional input parameters for the target application

Sample usage:

splash MySplashImage.bmp 600 450 "My GUI Title" 1 "C:\Program Files\MyApps\MyTargetApplication.exe" param1 param2

This will display the MySplashImage.bmp image in a 600×450 window (resizing the image if needed) until a window with a title of “My GUI Title” appears. In the example above, the parameters to the splash application were as follows:

1. image filepath – MySplashImage.bmp
2. image width – 600 pixels
3. image height – 450 pixels
4. target window title string – “My GUI Title”
5. title matching mode – 1 (=match at the beginning of the title)
6. target application – “C:\Program Files\MyApps\MyTargetApplication.exe”
7. optional input parameters to your target application – param1, param2

You can create a shortcut that runs the above command-line and then when the user clicks the shortcut it will automatically launch splash.exe, which will automatically display the splash screen and then load your main application.

I’ve contributed dozens of useful open-source utilities to the Matlab user community over the years (look at my File Exchange profile). This utility is different because it directly targets compiled applications, which are typically used commercially. For this reason, my splash wrapper application is not free (but also not very expensive). You can purchase a copy here, and I will email it to you.

Please note: this is a Windows application. It will not work on Macs or Linux.

Interested in learning more about actual and perceived performance improvement tricks in Matlab? Consider attending my Matlab Performance Tuning course – email me (altmany at gmail dot com) for details.

A trivial solution to this question is to use the built-in license function. Unfortunately, this does not work with a multi-system network/floating Matlab license, nor on deployed (compiled) systems.

While Matlab doesn’t have a built-in solution, we can use simple Java to access system information. There are several possible approaches. Here are several alternatives:

Windows SID

A Windows-specific approach is to return the Window Domain Controller’s SID (Security ID). This is a unique identifier that changes with each computer/user. Java enables direct access to this identifier, and we can run this directly in Matlab:

>> sid = get(com.sun.security.auth.module.NTSystem,'DomainSID')
sid =
S-1-5-21-292311649-1610625687-3346456317

The exact same value can also be gotten directly from the Windows Registry:

% Note: scanning HKEY_-- USERS node names is better, but Matlab's winqueryreg() can't do that...
rootkey = 'HKEY_CURRENT_-- USER';
subkey = 'Software\Microsoft\Windows\CurrentVersion\Group Policy\GroupMembership';
count = winqueryreg(rootkey,subkey,'Count');
for idx = 0 : double(count)-1   % Note: double() is needed for Matlab 6 compatibility
    val = winqueryreg(rootkey,subkey,['Group' char('0'+idx)]);
    dashes = find(val=='-');
    if length(dashes) > 4,  sid = val(1:dashes(end)-1);  break;  end   % short ids are phoney
end

This later version, although less simple than the first alternative (Java-based approach) above, has the benefit of working on old Matlab releases such as R12 (6.0) where the Java approach fails. For this reason, my getsid utility on the File Exchange uses the registry approach.

Other platforms

For non-Windows systems, both of the above approaches fail. For such platforms, we can use the Ethernet addresses of the computer’s network cards. This should be pretty unique for any practical effect:

sid = '';
ni = java.net.NetworkInterface.getNetworkInterfaces;
while ni.hasMoreElements
    addr = ni.nextElement.getHardwareAddress;
    if ~isempty(addr)
        addrStr = dec2hex(int16(addr)+128);
        sid = [sid, '.', reshape(addrStr,1,2*length(addr))];
    end
end
 
>> sid
sid = 
.89ECC872C85C.8AFE928FEDB4.8262278A1CCA.899919B50FC9

This again uses Java (which unfortunately fails on R12 aka 6.0). The benefit is that it is entirely cross-platform, working wherever Matlab runs (including Windows), on any Matlab release that supports Java (I think R13 aka 6.5 should be the earliest).

Do you have another way to generate unique identifiers? If so, please share your experience in a comment.

Well, at least not officially

The following trick restores the docking controls to figures in R2008a-compiled applications, enabling figure docking. Simply add one or both of the following alternatives in your application, after the figure has been created:

% Alternative #1 - uses pure Matlab
set(hFig, 'DockControls', 'on');
 
% Alternative #2 - uses the underlying Java frame
jFrame = get(handle(hFig), 'JavaFrame');
try
   % This works up to R2011a
   jFrame.fFigureClient.setClientDockable(true);
catch
   % This works from R2008b and up
   jFrame.fHG1Client.setClientDockable(true);
end

where hFig is the figure handle. This will have no effect for the regular interpreted (non-compiled) run of the application, where these controls are ‘on’ by default. But in the compiled application, although it may erroneously report that the controls are ‘on’, they are in fact ‘off’, so turning them ‘on’ fixes the problem.

Matlab figure docking control

Note: the two variants in alternative #2 above are actually identical, it is simply that the relevant field name has changed: Up to R2008a, only the fFigureClient existed; in R2008b, the fHG1Client field was added, which was simply an alias for fFigureClient, holding the same reference handle, so either of these fields could be used (a corresponding fHG2Client was also added – more on HG1 and HG2 here). In R2011b (at least the pre-release), the fFigureClient alias field was dropped and only fHG1Client remained. While the field name has changed, the underlying docking functionality appears to have remained stable over all these releases. For the record, in answer to a user question below, these fields can be listed using the built in fieldnames function:

% R2008b - R2011a:
>> fieldnames(jFrame)
ans = 
    'fFigureClient'
    'fHG2Client'
    'fHG1Client'
    'fUseHG2'
    'UICONTROLBACKGROUND_OS'
    'UICONTROLBACKGROUND_COMPATIBLE'

I was reminded of this trick by Aurélien’s recent comment, where he mentions MathWorks so-called workaround for this problem, which (IMHO) is really not a work-around at all: MathWorks advises to modify our application to use – would you believe this – tabbed panels to “dock” the separate figures contents onto separate panels. Not to mention the fact that this so-called “solution” relies on undocumented and unsupported Matlab functionality (that of tabbed-panels) and requires major rework of existing applications, it also results in far inferior look-and-feel than simple docking as G-d intended…

Since I have demonstrated above that the docking functionality actually exists in compiled apps just as in the interpreted m-file apps, I do not understand why MathWorks took such great pains to prevent this important functionality in the compiler. There must be some important reason for this, but I cannot think of any. Perhaps if there is enough public demand, MathWorks will agree to return the docking functionality.

Unfortunately, I recently discovered that in the most recent compiler, that ships with R2011a, alternative #1 above (which uses pure Matlab) no longer works. Sometime between R2008a and R2011a MathWorks discovered my first back-door and closed it. Such a pity…

Luckily, alternative #2 (which uses the underlying Java frame object) seems to still work, even on R2011a.

I still haven’t tested this on R2011b’s compiler (whose pre-release has become available for download yesterday), but hopefully the trick above will continue to work on R2011b and on subsequent releases – please tell me if you find out otherwise.

Addendum: Since 2013, possibly as a direct result of this post, MathWorks have prevented this workaround. MathWorks officially states that docking figures is not possible in deployed applications. I do not know the reason for this, and I have still not discovered if another workaround for this annoying limitation is possible.