A few days ago I encountered a situation where I needed to incorporate a JAR file into Matlab’s static Java classpath. I came across several relevant hacks that I thought could be useful for others:
- The documented approach
- Fixing javaclasspath.txt in run-time
- Preloading Java class files in javaclasspath.txt
- Loading Java class files into the static classpath in run-time (!)
- Analyzing loaded Java classes in run-time
The documented approach
In most use-cases, adding Java classes (or ZIP/JAR files) to the dynamic Java classpath is good enough. This can easily be done in run-time using the javaaddpath function. In certain cases, for example where the Java code uses asynchronous events, the Java files need to be added to the static, rather than the dynamic, Java classpath. In other cases, the Java code misbehaves when loaded using Matlab’s dynamic Java classloader, rather than the system classloader (which is used for the static classpath (some additional info). One example of this are the JAR/ZIP files used to connect to various databases (each database has its own Java JDBC connector, but they all need to reside in the static Java classpath to work properly).
Adding class-file folders or ZIP/JAR files to the static Java classpath can be done in several manners: we can update the Matlab installation’s classpath.txt file, or (starting in R2012b) a user-prepared javaclasspath.txt file. Either of these files can be placed in the Matlab (or deployed application’s) startup folder, or the user’s Matlab preferences folder (prefdir). This is all documented.
There are several important drawbacks to this approach:
- Both classpath.txt and javaclasspath.txt do not accept relative paths, only full path-names. In other words, we cannot specify ../../libs/mylib.jar but only C:\Yair\libs\mylib.jar or similar variants. We can use the
$jre_homeplace-holder macros, but not user folders. We can easily do this in our personal computer where the full path is known, but this is difficult to ensure if we deploy the application onto other computers.
Matlab’s compiler only supports adding Java classes and JARs to the dynamic classpath, but not to the static one – we need to manually add a classpath.txt or javaclasspath.txt to the deployment folder, complete with the deployment target’s correct full path. Naturally this is problematic when we wish to grant the user the flexibility of installing the deployed application anywhere on their computer disk.
- The javaclasspath.txt file is only supported on R2012b onward. The classpath.txt file is supported in both old and new releases, but (1) modifying it in Matlab’s installation folder may require system privileges, and (2) require update upon each and every Matlab reinstallation or upgrade. A classpath.txt file from one Matlab release will NOT be compatible with another Matlab release – if we try using an incorrect file Matlab will crash or hang. If you are not using just a single matlab release, this quickly turns into a maintenance nightmare.
- The javaclasspath.txt file loads the user-specified classes after those loaded in Matlab’s pre-defined classpath.txt. If we edit The classpath.txt file, we can place our classes at the top of the file, but this again entails the maintenance nightmare described above
Luckily, there are a few hacks that can alleviate some of the pain when dealing with deployed applications that use static Java classes:
Fixing javaclasspath.txt in run-time
At the very top of our main function, we could fix (or create) the javaclasspath.txt file to include the current folder’s actual full path, and then relaunch the application and exit the current session. The newly-launched application will use this javaclasspath.txt file and everything should work well from that moment onward. Here’s a bare-bones implementation example:
function mainProgram() if ~exist('javaclasspath.txt','file') try fid = fopen('javaclasspath.txt', 'wt'); fprintf(fid, [pwd '/classesFolder/']); % add folder of *.class files fprintf(fid, [pwd '/subFolder/classes.jar']); % add jar/zip file fclose(fid); system(['./' mfilename ' &']); catch err msg = sprintf('Could not create %s/javaclasspath.txt - error was: %s', pwd, err.message); uiwait(msgbox(msg, 'Error', 'warn')); % need uiwait since otherwise exit below will delete the msgbox end exit end % If we reached this point, then everything is ok - proceed normally ...
A related solution is to recreate the classpath.txt file using the build-generation script, as described by Andrew Janke.
Preloading Java class files in javaclasspath.txt
Class folders and JAR/ZIP files can be loaded before those in classpath.txt, without having to modify the system’s classpath.txt, as exposed by Christopher Barber. All we need to do is to add a line containing the
<before> string as a separate line in our javaclasspath.txt user-file. All entries beneath this line will be loaded into the front (rather than the end) of the static Java classpath. For example:
# This will be placed at the end of the static Java classpath C:\placed\at\the\end\of\the\static\classpath\mylib1.jar <before> # This will be placed at the beginning of the static Java classpath C:\placed\at\the\top\of\the\static\classpath\mylib2.jar
Loading Java class files into the static classpath in run-time (!)
The crown jewels in Java static classpath hacking belong without a doubt to power-users Andrew Janke and Amro. Based on Amro’s tip (based on a Java forum post from back in 2002), Andrew created a Matlab function that loads a Matlab class into the static classpath at run-time (i.e., without needing to modify/create either classpath.txt or javaclasspath.txt), slightly modified by me to include an optional classname input arg. Here’s the resulting javaaddpathstatic function:
function javaaddpathstatic(file, classname) %JAVAADDPATHSTATIC Add an entry to the static classpath at run time % % javaaddpathstatic(file, classname) % % Adds the given file to the static classpath. This is in contrast to the % regular javaaddpath, which adds a file to the dynamic classpath. % % Files added to the path will not show up in the output of % javaclasspath(), but they will still actually be on there, and classes % from it will be picked up. % % Caveats: % * This is a HACK and bound to be unsupported. % * You need to call this before attempting to reference any class in it, % or Matlab may "remember" that the symbols could not be resolved. Use % the optional classname input arg to let Matlab know this class exists. % * There is no way to remove the new path entry once it is added. % Andrew Janke 20/3/2014 http://stackoverflow.com/questions/19625073/how-to-run-clojure-from-matlab/22524112#22524112 parms = javaArray('java.lang.Class', 1); parms(1) = java.lang.Class.forName('java.net.URL'); loaderClass = java.lang.Class.forName('java.net.URLClassLoader'); addUrlMeth = loaderClass.getDeclaredMethod('addURL', parms); addUrlMeth.setAccessible(1); sysClassLoader = java.lang.ClassLoader.getSystemClassLoader(); argArray = javaArray('java.lang.Object', 1); jFile = java.io.File(file); argArray(1) = jFile.toURI().toURL(); addUrlMeth.invoke(sysClassLoader, argArray); if nargin > 1 % load the class into Matlab's memory (a no-args public constructor is expected for classname) eval(classname); end
The usage is super simple: instead of using javaaddpath to load a class-files folder (or ZIP/JAR file) into Matlab’s dynamic Java classpath, we’d use javaaddpathstatic to load the same input into the static classpath – in runtime.
As Andrew notes, there are important caveats to using the new javaaddpathstatic function. One of the important caveats is that the class needs to be loaded before Matlab gets around to “discover” that it does not exist, because from then on even if we load the class into the static classpath, Matlab will report the class as missing. Therefore, we need to call javaaddpathstatic at the very top of the program (possibly even in our startup.m file). Moreover, if we directly call the class anywhere in the same m-file as our call to javaaddpathstatic, it will fail. The reason is that Matlab’s built-in interpreter processes the direct Java call when it pre-parses the file, before the javaaddpathstatic had a chance to run:
function this_Will_Fail() % mylib.jar contains the com.app.MyClass class javaaddpathstatic([pwd '/mylib.jar']); obj = com.app.MyClass; % this will fail!
Instead, we need to prevent the interpreter from processing the class before run-time, by loading the class at run-time:
function this_Should_Work() % mylib.jar contains the com.app.MyClass class javaaddpathstatic([pwd '/mylib.jar']); obj = javaObject('com.app.MyClass'); % this should work ok
At least in one case, for a recent client project that required deploying a compiled Matlab application that used JDBC to connect to a database (and therefore relied on the static classpath), this saved the day for me.
I’m sad I wasn’t aware of this years ago, but now that I have posted it, we have less of an excuse to complain about static classpath files in Matlab. Hopefully MathWorks will incorporate this idea into Matlab so that we can more easily load classes into the static classpath in runtime, and remove the need for ugly deployment hacks (and yes, this includes MathWorks’ own toolboxes), hopefully removing the caveats listed above along the way.
Analyzing loaded Java classes in run-time
A related Matlab function, whereisjavaclassloadingfrom, was posted by Andrew Janke to report information about loaded classes, including whether they were loaded by the static (=standard Java) classloader, or the dynamic (=MathWorks’ custom) classloader, and from which location. For example:
% This is loaded into the static classpath as part of Matlab's standard classpath.txt file: >> whereisjavaclassloadingfrom('java.util.HashMap') Version: 220.127.116.11648 (R2015b) Class: java.util.HashMap ClassLoader: sun.misc.Launcher$AppClassLoader@69365360 URL: jar:file:/C:/Program%20Files/Matlab/R2015b/sys/java/jre/win64/jre/lib/rt.jar!/java/util/HashMap.class % This is loaded into the dynamic classpath using MathWorks' custom classloader >> javaaddpath('C:\Yair\Work\MultiClassTableModel') >> whereisjavaclassloadingfrom('MultiClassTableModel') Version: 18.104.22.168648 (R2015b) Class: MultiClassTableModel ClassLoader: com.mathworks.jmi.CustomURLClassLoader@5270d6ad URL: file:/C:/Yair/Work/MultiClassTableModel/MultiClassTableModel.class % This is an example of a class that is not loaded >> whereisjavaclassloadingfrom('no.such.Class') Error using javaArray No class no.such.Class can be located on the Java class path Error in whereisjavaclassloadingfrom (line 25)
How risky is all this in terms of future compatibility? well, I’m not a prophet, but I’d say that using these hacks is pretty risky in this regard. Matlab’s custom classloader is way-deep in undocumented territory. This does not mean that the functionality will change tomorrow, but don’t be surprised if and when it does.
Have you ever used any additional undocumented hack relating to using the Java classpaths in Matlab? If so then please post a comment below.
Users who have endured my post this far, should probably also read my related article about Java class access pitfalls.
Specifying relative paths in classpath.txt may not be supported, but for me it works just fine in R2011b.
Simply add paths relative to the location of classpath.txt (i.e. relative to matlabs startup directory):
The corresponding entries in the javaclasspath within matlab are not expanded properly, but all jars are properly loaded.
I would say with regard to loading classes through the java (root) classloader directly should not be that risky… As long as MATLAB is using Java, the root classloader should behave exactly the way it is designed. But you are right if you load classes through the MATLAB classloader, it is definitely prone to changes in the future. That’s why the utility whereisjavaclassloadingfrom from Andrew is wonderful, a necessity to tell what classloader a class comes from (in order to debug class loading issues).
I think I have a method of adding and removing from static class path, but I am unable to make it stick on subsequent restarts of Matlab. The idea is to use
java.lang.getPropertyto set and edit the static class path.
1. Get the string from static class path and store it as a char
2. manipulate the string either through string concatenate (strcat), string replace (strrep)
3. put the string back into the
If you could help add the last bit where loaded class paths can be made permanent. This would be great.
On a secondary note thank you for sharing whereisclassloadingfrom from Andrew, it has saved so much time.
It is ‘
java.lang.System.getProperty‘ and ‘
java.lang.System.setProperty‘. I read that JVM uses ‘java.class.path’ content only at startup to create its
SystemClassLoaderand ignores it later on. I read that
SystemClassLoadercould be made to reinitialize by setting its sys_paths-field to null (https://stackoverflow.com/questions/15409223/adding-new-paths-for-native-libraries-at-runtime-in-java), but nothing happens unfortunately:
Yair and others,
I found yet another way to add files to Java’s class path. See the function java_class_path_add_before_static(file) below. Benefit of this new method is that the classes are dynamically set before MATLAB’s static ones and its classes will be found first. This allows me to, for instance, provide newer version of *.jar of same jar libraries MATLAB uses (i.e. commons-compress.jar). It worked in at least R2019b and R2011a in short test.
I also wrote few other useful functions, available at my MATLAB toolbox repository under free BSD license: