I would like to welcome guest blogger Pavel Holoborodko of Advanpix, maker of the Multiprecision Computing Toolbox. Today, Pavel discusses undocumented MEX functions that he encountered in his work to improve the toolbox performance.
It is not a secret that Matlab has many undocumented (or deliberately hidden) features and commands. After all, Yair’s website & book are devoted specifically to this topic.
However most of the findings are related to Matlab language itself and investigations on undocumented MEX API seems to be scarce.
During development of our toolbox we have found lots of hidden functions which can be helpful for creating speed-efficient extensions for Matlab using native languages.
Here we want to explain some of them in detail and provide complete list of undocumented MEX API functions.
Please note that there is a risk in using the functions – MathWorks can change / remove some of them in next versions. It is an additional burden for developers to stay tuned and update their toolboxes on time.
Reduced OOP capabilities of Matlab
Starting with R2008b Matlab allows user to introduce custom-type classes by the classdef keyword. Matlab was late on adding object oriented features – I can only image how hard it was for developers at MathWorks to add OOP constructs to a purely procedural language, which follows entirely different philosophy. (Yes, objects could be created using structs and special folder structure in earlier version of Matlab – but that was just ugly design, MathWorks will not support it in the future).
They still don’t have full support for OOP features though. The most important missing features are:
- It is prohibited to have a destructor for custom non-handle classes
- It is not possible to overload assignment-without-subscripts operator (e.g. A = B)
I don’t know the reasons why these fundamental OOP paradigms are not implemented in Matlab – but they surely prevent creating powerful virtual machine-type of toolboxes.
In that case Matlab objects would have only one property field – ‘id’, identifier of variable stored in MEX module – virtual machine (e.g. pointer to C++/C object). MEX module would only need to know ‘id’ of objects and what operation to conduct with them (+, -, *, etc.) – all processing would be done in MEX. Heavy data exchange between Matlab and MEX libraries would be completely unnecessary. Matlab would act as just an interpreter in such scenario. Moreover MEX API could be simplified to several functions only.
Access to object properties from MEX
Unfortunately we are restricted to current architecture – where all the data are allocated / stored on Matlab side and we have to transfer it from Matlab to MEX library in order to work with it. The official MEX API provides two functions to access object properties from within MEX library: mxGetProperty
and mxSetProperty
.
Both functions share the same problem – they create a deep copy of the data!
Imagine the situation when your object is a huge matrix with high-precision elements and it occupies 800MB of RAM. If we want to access it in MEX library (e.g. transpose) we would call mxGetProperty
which will do ENTIRE COPY of your object’s property – wasting another 800MB!
Obviously this cannot be accepted, not speaking of totally reduced performance (copying could take a while for such amount too).
In search for remedy we found a similar (but) undocumented functions we can use to get shared access to objects properties (32-bit):
extern mxArray* mxGetPropertyShared(const mxArray* pa, unsigned int index, const char * propname); extern void mxSetPropertyShared(mxArray* pa, unsigned int index, const char * propname, const mxArray* value); |
These functions can be used as one-to-one replacement of the official functions: mxGetPropertyShared
just returns pointer to existing property without any copying; mxDestroyArray
can still be called on returned pointer (thanks to James Tursa for the correction).
Full list of MEX API functions
We have extracted the full list of usable MEX functions from libmx.dll and libmex.dll (Matlab R2012b Windows 32-bit) – the two main MEX API dynamic libraries. It includes functions from the official API as well as undocumented ones (which are in fact the majority):
The distinctive feature of the list – it provides de-mangled C++ names of functions, with type of arguments and return value. This makes usage of undocumented functions much easier. You can also easily see this list using tools such as DLL Export Viewer or the well-known Dependency Walker. This list was not previously published to the knowledge of this author; a much shorter unannotated list was posted by Yair in 2011.
Take a look at the function list – there are a lot of interesting ones, like mxEye
, mxIsInf
, mxFastZeros
, mxGetReferenceCount
and many others.
Moreover it is possible to see high level C++ classes MathWorks developers use for work. Peter Li has posted an article about mxArray_tag
‘s evolution and internal structure in 2012. Now it is clear that the fundamental data type mxArray_tag
is not a plain-old-struct anymore, it has member-functions and behaves more like a class. It even has custom new
/delete
heap management functions and overloaded assignment operator=
. Reverse-engineering of these functions might reveal the exact & complete data structure of mxArray_tag
, but perhaps this is not allowed by the Matlab license agreement.
Anyway, with some effort, the internal mxArray_tag
class from MathWorks might be used in third-party MEX files. How much more easier this would be instead of clumsy mx****
functions!
Please feel free to leave your requests or comments below.
Note: This article was originally posted on the Advanpix blog; reposted here with kind permission and a few changes.
Thank you. Very interesting. I don’t know why some of these aren’t documented. However,
mxIsInf
is documented: http://www.mathworks.com/help/matlab/apiref/mxisinf.htmlWould you open a Google+ Page?
It will make following you much easier.
Thanks.
Yair,
mxGetPropertyShared is working great on Windows. Do you know whether there is a way to use it on LINUX as well ? The generic solution solution by James Tursa works consisting of mxGetPropertyPtr works for LINUX but slows processing speed if called many times.
Best Regards,
Ilan.
@Ilan – I don’t have access to Linux at the moment so I’m afraid someone else would need to answer this…
Interesting news from the R2015a release notes: mxCreateUninitNumericMatrix and mxCreateUninitNumericArray are now *documented* functions.
Correct signatures for these two functions are:
(Note that
mwIndex
has a different basic type depending on the architecture.)Noteworthy is that the MATLAB code for the property setter and getter methods will be executed here, so these functions are not necessarily quick. It is sad that we cannot create class methods as MEX-files.