MATLAB\R2017b\toolbox\distcomp\distcomp\distcompserialize64.mexw64

There is a function distcompdeserialize() which reconstructs the matlab data from the bytes. Fwiw, getArrayFromByteStream() appears to work on the serialized bytes produced by distcompserialize64 as well. Performance seems to be equivalent for both discompdeserialize() and getArrayFromByteStream().

Thanks!

@Roc – thanks for the follow-up for the benefit of other readers

I solved the problem. The problem is that when you compile to DLL, if you use the following command

mcc -W cpplib:testDeserialized -T link:lib testDeserialized.m -C;

the compiler can’t accurately package all the functions that need to be dependent on all the deserialization. At this time, create an empty object that needs to be deserialized, save it, such as nullObj.mat and then pack it with the -a option, and the compiler will analyze the mat file and automatically find the fully dependent functions. As shown below

mcc -W cpplib:testDeserialized -T link:lib testDeserialized.m -a nullObj.mat -C.

Thanks again @Yair

@Yair, Thank you very much. I tried your suggestion, but it still has the same result as previous. It is noteworthy that the following code works fine in MATLAB:

function testDeserialized(matfilename)
   fileData = load(matfilename);
   netByte2 = uint8(fileData.netByteSaved);  % convert back to uint8
   net2 = getArrayFromByteStream(netByte2);
   save D:\net00.mat net2

but when I use the following code

mcc -W cpplib:testDeserialized -T link:lib testDeserialized.m  -C;

generate deployed files, such as testDeserialized.dll, testDeserialized.lib, testDeserialized.ctf and call testDeserialized.dll from C++ it has error when I tried to load D:\net00.mat. The error message is as follows:

load('D:\net00.mat')
Error using load
Cannot read file D:\net00.mat. 
try open('load(''D:\net00.mat'')
          ↑
Error: Character vector is not terminated properly.

when double click net00.mat,the error message changed to be

Warning: Unable to read some of the variables due to unknown MAT-file error.
 
> In matfinfo (line 9)
  In finfo (line 118)
  In internal.matlab.importtool.ImportableFileIdentifier.isTextFile (line 113)
  In internal.matlab.importtool.ImportableFileIdentifier.useTextImportTool (line 91)
  In uiimport>useTextImportTool (line 998)
  In uiimport (line 237) 
Warning: Unable to read some of the variables due to unknown MAT-file error.
 
> In matfinfo (line 9)
  In finfo (line 118)
  In uiimport/gatherFilePreviewData (line 416)
  In uiimport (line 245) 
Error using load
Number of columns on line 2 of ASCII file D:\net00.mat must be the same as previous lines.
Error in uiimport/runImportdata (line 467)
                    datastruct = load('-ascii', fileAbsolutePath);
Error in uiimport/gatherFilePreviewData (line 435)
        [datastruct, textDelimiter, headerLines]= runImportdata(fileAbsolutePath,type);
Error in uiimport (line 245)
    gatherFilePreviewData(fileAbsolutePath); 

Do you have any good Suggestions? Thanks.

@Roc – try to convert your data to int16 before saving, and then convert back to uint8 after loading:

netByte = getByteStreamFromArray(net);
netByteSaved = int16(netByte);  % convert to int16
save(filename, 'netByteSaved');
...
fileData = load(filename);
netByte2 = uint8(fileData.netByteSaved);  % convert back to uint8
net2 = getArrayFromByteStream(netByte2);

net = alexnet;
netByte = getByteStreamFromArray(net);
net2 = getArrayFromByteStream(netByte);

it performs well in both MATLAB or deployed mode.

but if we save netByte in a mat file in advance, then it does not work in dll mode:

netByte = load('netByte.mat');
netByte = netByte.netByte;
net2 = getArrayFromByteStream(netByte);
save D:\net2.mat net2

We can not load the net2.mat. It doesn’t look like to be created correctly.

Do you have any ideas for solving this problem?

Thanks!

Analogously, getArrayFromByteStream also calls loadobj.

@Andre – The limit is either 2 or 4 GB depending on the type of data, since the format uses 32-bit signed integers in some places and 32-bit unsigned integers in other places. If we stick to plain arrays, the limit is 2^32-1 bytes or entries in one dimensions, i.e. this stays within the limit (and thus works)

getByteStreamFromArray(zeros(1, 2 ^ 31 - 1, 'uint8'));

while

getByteStreamFromArray(zeros(1, 2 ^ 31, 'uint8'));
getByteStreamFromArray(zeros(0, 2 ^ 31, 'uint8'));

both fail.

If you use aggregate data types, e.g. cells or structs, then the limit is 4 GB. You can put 3 arrays of size 1 GB into a cell array and then successfully it.

(However, there’s an elegant way to get beyond this limitation. Watch out for one of my future comments.)