- Undocumented Matlab - https://undocumentedmatlab.com -

Speeding up compiled apps startup

Posted By Yair Altman On August 22, 2012 | 32 Comments

A few weeks ago, I discussed a splash wrapper application [1] that alleviates much of the pain of the slow startup of deployed (compiled) Matlab applications. While such a splash wrapper is indeed useful, it may also be possible to achieve an actual speedup of the compiled app’s startup using the MCR_CACHE_ROOT [2] environment variable.
The following has been reported to me by a reader of this blog. Please note that I cannot independently confirm the correctness of the report (if anyone can let me know I would be grateful):
Normally, the MCR and the stand-alone executable is unpacked upon every startup in the user’s temp dir, and deleted when the user logs out. Apparently, when the MCR_CACHE_ROOT environment variable is set, these files are only unpacked once and kept for later reuse. If this report is indeed true, this could significantly speed up the startup time of a compiled application in subsequent invocations.
On Linux:

export MCR_CACHE_ROOT=/tmp/mcr_cache_root_$-- USER   # local to host
mkdir -p @MCR_CACHE_ROOT
./myExecutable

On Windows:

REM set MCR_CACHE_ROOT=%TEMP%
set MCR_CACHE_ROOT="C:\Documents and Settings\Yair\Matlab Cache\"
myExecutable.exe


If you wish to set this env variable permanently on Windows, look at the explanation provided here [3].
Setting MCR_CACHE_ROOT is especially important when running the executable from a network (NFS) location, since unpacking onto a network location could be quite slow. If the executable is run in parallel on different machines (for example, a computer cluster running a parallel program), then this might even cause lock-outs [4] when different clusters try to access the same network location. In both cases, the solution is to set MCR_CACHE_ROOT to a local folder (e.g., /tmp or %TEMP%). If you plan to reuse the extracted files again, then perhaps you should not delete the extracted files but reuse them. Otherwise, simply delete the temporary folder after the executable ends. In the following example, $RANDOM is a bash function that returns a random number:

export MCR_CACHE_ROOT=/tmp/mcr$RANDOM
./matlab_executable
rm -rf $MCR_CACHE_ROOT

Setting MCR_CACHE_ROOT can also be used to solve other performance bottlenecks in deployed applications, as explained in a MathWorks technical solution [5] and a related article here [6].
In a related matter, compiled Matlab executables may fail with a Could not access the MCR component cache error, when Matlab cannot write in the MCR cache directory due to missing permission rights. This can be avoided by setting MCR_CACHE_ROOT to a non-existent directory [7], or to a folder in which there is global access permissions (/tmp or %TEMP% are usually such writable folders) – see related posts here [8] and here [9].

Categories: Low risk of breaking in future versions, Undocumented feature


32 Comments (Open | Close)

32 Comments To "Speeding up compiled apps startup"

#1 Comment By Andi On August 22, 2012 @ 17:04

Thanks for the suggestion. In addition, I have found that running the following snippet before compiling significantly reduces the size of the executable and the time to unpack the executable.

f = dir(prefdir);
for index = 1: numel(f)
    if f(index).isdir && ~strcmpi(f(index).name, '.') && ~strcmpi(f(index).name, '..')
        fprintf('deleting %s ...\n', fullfile(prefdir, f(index).name));
        rmdir(fullfile(prefdir, f(index).name), 's');
    end
end

#2 Comment By Yair Altman On August 22, 2012 @ 17:16

@Andi – this is strange. Can you tell how you came across this and/or what the logic behind deleting all preference files is? This is obviously a very destructive operation and I would not advise doing it without a very good reason…

#3 Comment By Andi On September 1, 2012 @ 12:12

Yair, please note that I do not delete the preference files in the preference root directory. I only delete the subdirectories, which I believe are created by the compiler. Furthermore, I have not found anything in the MatLab documentation, which would suggest that these subdirectories are important.

I have noticed that the executables of small programs got bigger and bigger after each compilation (> 10 MB) without any obvious reason. In the log of the compiler I saw that the compiler includes many files from these subdirectories. I then deleted these subdirectories and found that the executables are suddenly small again (a few hundred KB). I have not observed any adverse effects.

#4 Comment By Yair Altman On September 1, 2012 @ 12:17

Ah! – it makes sense now. Strange behavior on the part of the compiler, I must say

#5 Comment By alvaro562003 On December 18, 2016 @ 12:10

As of today, December 18 2016, prefdir is: C:\Users\admin\AppData\Roaming\MathWorks\MATLAB\R2016a

The content of this directory is:

18/12/2016  09:23              .
18/12/2016  09:23              ..
18/12/2016  10:51             1ÿ291 publish_configurations.m
18/12/2016  10:51               220 run_commands.m
18/12/2016  09:14         1ÿ817ÿ222 History.bak
18/12/2016  11:01         1ÿ817ÿ797 History.xml
18/12/2016  10:49             4ÿ668 matlab.prf
18/12/2016  09:23            18ÿ143 matlab.settings
18/12/2016  11:01            10ÿ081 MATLABDesktop.xml
18/12/2016  09:20             9ÿ092 MATLABDesktop.xml.prev
18/12/2016  09:20             7ÿ420 MATLAB_Editor_State.xml
14/04/2016  18:30             4ÿ130 comparisons.settings
07/04/2016  09:03                 0 creation.timestamp
10/03/2016  12:44               731 dastudio.prf
24/03/2016  20:42             2ÿ040 database.settings
07/04/2016  09:03              HtmlPanel
10/03/2016  10:14                87 javaclasspath.txt
19/10/2016  10:23               618 matlabprefs.mat
29/07/2015  09:37             8ÿ963 MBUILD_C_win64.xml
29/07/2015  09:37             8ÿ978 mex_C++_win64.xml
03/09/2015  19:50             8ÿ521 mex_C_win64.xml
22/02/2016  10:44                40 MLintDefaultSettings.txt
07/04/2016  09:13             1ÿ298 mwKeyStore
07/04/2016  09:13             6ÿ144 parallel.settings
27/01/2016  22:38               201 shortcuts_2.xml
10/03/2016  12:29               138 systemtest.cfg
07/04/2016  09:13               772 thisMatlab.pem

I would advise not doing it. Personally, I will not do this operation.

#6 Comment By Sam On May 14, 2020 @ 06:12

Doesn’t work for me! I am using Matlab 2018a, no different at all, even the exe file size is the same.

#7 Comment By Cris On August 23, 2012 @ 02:12

Some years ago I used to manually unpack the archive file using the $MATLABROOT/toolbox/compiler/deploy/$ARCH/extractCTF utility. Things have changed in the compiler, but the utility still exists, so this technique might still work.

#8 Comment By Ofek Shilon On March 29, 2017 @ 17:43

As of matlab 2016a, the path to the utility (at least on windows 10) is $MATLABROOT/bin/$ARCH/

#9 Comment By Hao Deng On December 10, 2018 @ 11:13

Hi, I found that extractCTF did work. But after extraction, what should I do? Put the extracted files into MCR cache dir? It didn’t work in this way since MCR simply ignore the extracted files and did extraction again.

#10 Comment By Matt W On August 23, 2012 @ 07:41

Hi Yair,
I can confirm that setting the MCR_CACHE_ROOT variable is indeed the way to go. Just ran accross this on a .NET deployment of one of our apps and it improved the startup time and a number of other headaches considerably.
On our desktop apps we generally just compile with the ctf archive separately (-C switch on mcc). This is for two reasons 1) We unpack once to a folder under the executable. 2) Our CTF contains a number of resource files and preferences . Although one can’t swap out any .m code you are free to manipulate your other resources included in there. This was also another reason we went to the MCR_CACHE_ROOT for the .NET deployment. For example we use the Matlab report generator for some reports. Unfortunately the stylehseet (.rgs- actually an xml) seems to only take absolute file paths for resources like bitmaps we use for report headers . So on first time startup we have the software figure out where it is installed and then where the absolute location of the resources are and then edit the stylesheet.

Best Regards
Matt

#11 Comment By Dirk Engel On October 3, 2012 @ 04:38

Yair:
I cannot confirm! Where did you find the information that extracted CTF files are “normally deleted when the user logs out”? With my setup (Win 7, MATLAB 2010b, compiled without -C flag, MCR_CACHE_ROOT not set) unpacked files in the user’s temp directoy (%LocalAppData%\Temp\\mcrCache7.14\ in my case) do not get deleted on log-out or reboot but are kept for later use. Up to now this was “normal” behavior for me. But I don’t like the idea of getting this behavior just by chance, so if anybody has an explaination…

#12 Comment By Hao Deng On December 10, 2018 @ 09:43

In my case, things go like this: Without setting the MCR_CACHE_ROOT env variable, compiled exe will be extracted to path like this %LocalAppData%\Temp\\mcrCache83\XXX. It will usually not be deleted after closing. However, when I open another app which call matlab compiled dll(not exe) , the previous extracted files are totally removed. This phenomenon seems remain exactly the same when I set the MCR_CACHE_ROOT. So the question should be: how to prevent MCR from cleaning extracted files when switching between extracting exe and extracting dll?

#13 Comment By Dani On October 6, 2012 @ 08:42

I just tried this with 2011b on Windows 7. I compiled my application with the -C switch, extracted the content of the CTF with extractCTF.exe and set the MCR_CACHE_ROOT to the directory with the extract. The startup time is the same if I start the application like this or if I do it the standard way (i.e. no -C, no extract, no MCR_CACHE_ROOT). It takes about 7 seconds until I see first window appear. Maybe this is due to the SSD disk.

#14 Comment By ECS_VUW_Kevin On November 8, 2012 @ 16:04

The link you provide under the word MCR_CACHE_ROOT in the first paragraph

[16]

no longer seems to exist.

Matlab’s web presence first gave me a “you need an account to access” message (presumably because it’s top secret information?) and, having set one up, I then got told:

The page you were looking for is not available in this version of the Documentation. 
You can use the search box or browse the products below to find related information. 
You can also view archive documentation for prior releases.

however, when I try to search the previous version, I get told

Access to archived documentation is a benefit for licensed users of MathWorks products.
Your account must be associated with a license. 

so, just wanted to say thanks for your resource, which I found when trying to find the content from the top secret pages that Matlab no longer want anyone to see.

#15 Comment By Yair Altman On November 8, 2012 @ 16:15

@Kevin – the webpage that you mentioned can be accessed [17] (no MathWorks license required). I discussed this doc-archiving mechanism some weeks ago, [18]. In this day and age, once a webpage has been posted, it is difficult to get rid of it completely…

#16 Comment By Yair Altman On July 24, 2013 @ 13:26

MathWorks moved the contents of the webpage that explains MCR_CACHE_ROOT and the CTF embedding feature to here: [2]

Too bad they didn’t automatically redirect traffic from the old link, but at least the information has not been deleted.

#17 Comment By Ofek Shilon On March 29, 2017 @ 17:35

@Yair link is broken again

#18 Comment By Yair Altman On April 2, 2017 @ 12:25

Here are the updated links on the Mathworks website (note the minute differences between the pages):
[19]
[20]

#19 Comment By xiaogp13 On July 8, 2013 @ 04:39

It doesn’t work!

#20 Comment By Yair Altman On July 8, 2013 @ 04:42

@xiaogp13 – what doesn’t work? please be more specific…

#21 Comment By xiaogp13 On July 8, 2013 @ 04:59

I have set MCR_CACHE_ROOT, but my compiled application still startup too slow!

#22 Comment By Yair Altman On July 8, 2013 @ 05:03

The MCR_CACHE_ROOT trick is apparently platform-dependent. As you can see from the other comments here, it does work for some people.

#23 Comment By eric On August 4, 2013 @ 10:28

How do you introduce the set MCR_CLASS_ROOT command, Matlab doesn’t like it, is it a batch file?

#24 Comment By Yair Altman On August 4, 2013 @ 10:30

@Eric – it is an [21], not a Matlab variable…

#25 Comment By eric On August 4, 2013 @ 17:02

alright, unfortunately I’m a student w/ no license, so I can’t access the mathworks page, I’m not entirely sure how to set the variable in windows for my program

#26 Comment By Yair Altman On August 4, 2013 @ 23:58

Look at the link that I provided above

#27 Comment By Matthias On August 5, 2013 @ 06:16

This might be a bit offtopic but I was having a problem with my MCRCache. For some reason when I deploy my executable to a different system (using a 3rd Party Installer, though all that one really does is copying some files) when I run the executable for the first time the MCRCache is created. However something’s wrong with it and the first time I get an error message from the Visual C++ Runtime (probably from the redistributables that the MCR installs during its setup process).
Anyway, after the first start the error message no longer appears and the executable works as intended.

Is there any way I could ‘pregenerate’ the mcrcache and deploy it alongside the executable? This is not just about the .CTF (at least from my understanding). The CTF File (if I generate one as opposed to just compiling it into the executable) is only 8MB whereas the MCRCache is 23MB.

#28 Comment By Yair Altman On August 5, 2013 @ 06:31

@Matthias – this sounds like a compiler or installer bug. I suggest that you contact MathWorks customer support about this. Please place a followup comment here if you learn anything new.

#29 Comment By Andy On November 21, 2013 @ 09:42

Hi,
I succesfully changed the mcr cache folder to ‘C:\matlab_mcr_cache’ by setting the MCR_CACHE_ROOT as a system variable. Files added to the exe (some automatically by mcc, and some manually by -a) do indeed exctract to something like ‘C:\matlab_mcr_cache\mcrCache7.17’, but they are automatically deleted when the program is finished, thus giving no potential gain in speed. Am I doing something wrong? I do not use -c, since I want a single exe-file.
/Andy

#30 Comment By Clayton Chu On April 28, 2014 @ 14:32

Hi Yair –

I was wondering if this long start-up time still exists in MATLAB R2013b or newer. They switched to a shiny new deploytool, but I wonder if anything was done on the backend to improve startup time. It looks like MATLAB Compiler jumped from v4.18.1 to v5.0 between R2013a and R2013b.

#31 Pingback By Deployed Matlab application using significantly more memory than Matlab scripts | Ngoding On November 19, 2014 @ 05:50

[…] archive and startup the MCR runtime, just like the delay you see when you start the MATLAB IDE): undocumentedmatlab.com/blog/speeding-up-compiled-apps-startup. –  Amro Jun 28 '13 at […]

#32 Comment By dariush On September 23, 2015 @ 17:14

Hi Yair.
I wrote a fairly complex GUI in R2014b and i have the problem of very slow startup. Could you please explain a little more on the solution you mentioned here. What steps should i take to set MCR_CACHE_ROOT and so on? Where can i set this options?


Article printed from Undocumented Matlab: https://undocumentedmatlab.com

URL to article: https://undocumentedmatlab.com/articles/speeding-up-compiled-apps-startup

URLs in this post:

[1] splash wrapper application: http://undocumentedmatlab.com/blog/splash-window-for-deployed-applications/

[2] MCR_CACHE_ROOT: http://www.mathworks.com/help/compiler/mcr-component-cache-and-ctf-archive-embedding.html

[3] here: http://comp.uark.edu/~npai/StreBanD/FAQ.html

[4] cause lock-outs: http://bioinfo.ittc.ku.edu/faq.php?type=11#47

[5] MathWorks technical solution: http://www.mathworks.com/support/solutions/en/data/1-7RH0IV/

[6] here: http://matlab4compbio.blogspot.co.il/2010/04/how-to-solve-mcr-cache-access-problems.html

[7] non-existent directory: http://wiki.rcs.manchester.ac.uk/community/MatlabWithCondor#MCR_CACHE_ROOT

[8] here: https://www.mathworks.com/matlabcentral/newsreader/view_thread/283432

[9] here: https://www.mathworks.com/matlabcentral/newsreader/view_thread/255547

[10] Removing user preferences from deployed apps : https://undocumentedmatlab.com/articles/removing-user-preferences-from-deployed-apps

[11] Quirks with compiled Matlab DLLs : https://undocumentedmatlab.com/articles/quirks-with-compiled-matlab-dlls

[12] Using pure Java GUI in deployed Matlab apps : https://undocumentedmatlab.com/articles/using-pure-java-gui-in-deployed-matlab-apps

[13] Docking figures in compiled applications : https://undocumentedmatlab.com/articles/docking-figures-in-compiled-applications

[14] Speeding up Matlab-JDBC SQL queries : https://undocumentedmatlab.com/articles/speeding-up-matlab-jdbc-sql-queries

[15] Speeding-up builtin Matlab functions – part 1 : https://undocumentedmatlab.com/articles/speeding-up-builtin-matlab-functions-part-1

[16] : http://www.mathworks.com/help/toolbox/compiler/brl4_f1-1.html

[17] : http://web.archive.org/web/20100922055526/http://www.mathworks.com/help/toolbox/compiler/brl4_f1-1.html

[18] : https://undocumentedmatlab.com/blog/changes-in-the-online-doc-urls/

[19] : http://mathworks.com/help/compiler_sdk/ml_code/mcr-component-cache-and-ctf-archive-embedding.html

[20] : http://mathworks.com/help/compiler_sdk/cxx/matlab-runtime-component-cache-and-deployable-archive-embedding.html

[21] : https://en.wikipedia.org/wiki/Environment_variable

Copyright © Yair Altman - Undocumented Matlab. All rights reserved.