Comments on: Matlab's internal memory representation

By: Cris Luengo

Cris Luengo — Sun, 29 Apr 2018 05:14:10 +0000

Finding the size of the mxArray header is much simpler than what is shown here:

similar work of mine to github, hopefully it adds value to anyone trying to watch mxArray's (at least from visual studio).

By: Undocumented Matlab MEX API | Undocumented Matlab

Undocumented Matlab MEX API | Undocumented Matlab — Wed, 24 Jul 2013 18:02:30 +0000

[…] it is possible to see high level C++ classes MathWorks developers use for work. Peter Li has posted an article about mxArray_tag‘s evolution and internal structure in 2012. […]

By: Christian Di

Christian Di — Wed, 29 May 2013 20:40:17 +0000

Dear Mr. Altman,
First of all, I wish to congratulate you for the excellent work done with this “Undocumented Matlab” portal. There are lots of things actually undocumented in Matlab and that need to be clarified. Currenlty, I am an engineer student that is working with coders (H.264-HDTV) and therefore working at the bit level with Matlab. Bit level operations and specially algorithms that demand to consider byte alignments are quiet difficult in order to improve speed. While reading this article I was animated to consider to “break” Matlab’s FILE structure. Matlab has an option in the fread function to consider bit reding called “ubit”. This property is not seen in C, nor C++, nor in Octave. I wanted to ask all the persons involved in this blog if you believe that there is a special pointer “hidden” in Matlab’s “fread” structure in order to save the bit position. This is a topic that I been trying to discover in the last months and where I was not able to find information. If you could know something about this, I will appreciate your help.

Once more, congratulations for the great work done.

Best regards,

Christian Di

By: SK Mody

SK Mody — Sun, 16 Sep 2012 00:01:31 +0000

I stumbled upon this blog when trying to find out how property ‘set’ methods are impacted if the property is an array. For example:

classdef Collection
   properties
      array;
   end
   methods
      function This = set.array(This, s)
         % do something
         This.array = s;
      end
   end
end

C = Collection;
C.array = rand(2,2,1000);
C.array(1,1,3) = 5;

‘set’ is called for the indexed assignment, with an argument ‘s’ which is a ‘copy’ of This.array with exactly one element that is different. I was wondering Matlab could implement this sort of thing with a basic Copy on Write without causing some terrible inefficiencies. But reading the above I see that a more sophisticated COW could work reasonably.

Thanks.

By: Peter

Peter — Tue, 27 Mar 2012 20:18:28 +0000

In reply to Martin.

Sounds very interesting Martin. Indeed, the “reference” field is another particularly important subject that I believe was covered in part for older Matlab versions in a post by Benjamin Schubert: http://www.mk.tu-berlin.de/Members/Benjamin/mex_sharedArrays

I’m not aware of any more up-to-date write-up of this however. I would be particularly interested in better understanding how this interacts with mxUnshareArray and other copy-on-write mechanisms, some of which Benjamin’s article goes into.

If my goal here was successful, perhaps you will find some of the simple tools here useful either in your investigations or else just in demonstrating your findings succinctly.

By: Martin

Martin — Tue, 27 Mar 2012 14:51:40 +0000

In fact I have something like headerdump which works well with recent versions of MATLAB. I created it in order to reverse-engineer the internal representation of mxArray. My original motivation for doing this was to be able to track MATLAB's memory allocation for big arrays because I'm on a NUMA system and I wanted to be able to check/enforce that often used variables reside on node-local memory. For example, MATLAB distinguishes between normal variables (as shown in this post), temporary variables (when you pass immediate values to a function), global/persistent variables, and variables embedded in cell arrays and structs. Concerning the latter, given this post you could assume that

x = repmat({ones(200)}, 1, 100);

creates 100 doubly-linked instances of mxArray (sharing the same data -- a 200×200 matrix) for the entries of x, but in fact there will be only one mxArray with a reference count set to 100. If there's sufficient interest I could imagine sharing my code (after some cleanup). Maybe I could also write a blog post similar to this one, but digging deeper into the internals (as far as I managed to understand them).

By: Peter

Peter — Wed, 21 Mar 2012 19:54:22 +0000

In reply to Laurens Bakker. Thanks Laurens. Yes what you say makes sense. For some reason I was thinking about this in terms of traversing the cycle, but of course when we modify one element we must delete it from the cycle and that is the operation where the double linking will save you time (if your cycle is irrationally large :)).

By: Laurens Bakker

Laurens Bakker — Fri, 16 Mar 2012 18:40:35 +0000

Hi Peter,

Thanks for this nifty bit of detective work. You wondered why the cycle of identical arrays is stored as a doubly linked list. My guess is that it is probably done for time-efficiency. Suppose for some reason you create a million copies of a single array, and then modify a single one. In order to re-close the loop, you would have to iterate over the entire cycle to get both loose ends. With a doubly-linked list this can be done in constant time.

Cheers,

Laurens