**- Undocumented Matlab - https://undocumentedmatlab.com -**

rmfield performance

Posted By Yair Altman On May 25, 2016 | __5 Comments__

*Once again I would like to introduce guest blogger Hanan Kavitz ^{[1]} of Applied Materials ^{[2]}. Several months ago Hanan discussed some quirks with compiled Matlab DLLs ^{[3]}. Today Hanan will discuss how they overcame a performance bottleneck with Matlab’s builtin rmfield function, exemplifying the general idea that we can sometimes improve performance by profiling the core functionality that causes a performance hotspot and optimizing it, even when it is part of a builtin Matlab function. For additional ideas of improving Matlab peformance, search this blog for “Performance” articles ^{[4]}, and/or get the book “Accelerating MATLAB Performance ^{[5]}“.*

I’ve been using Matlab for many years now and from time to time I need to profile low-throughput code. When I profile this code sometimes I realize that a computational ‘bottleneck’ is due to a builtin Matlab function (part of the core language). I can often find ways to accelerate such builtin functions and get significant speedup in my code.

I recently found Matlab’s builtin

It started when a co-worker asked me to look at a code that looked just slightly more intelligent than this:

```
for i = 1:5000
myStruct = rmfield(myStruct,fieldNames{i});
end
```

Running this code within a tic/toc pair yielded the following results:

```
>> tic; myFunc(); t1 = toc
t1 =
25.7713
```

In my opinion 25.77 secs for such a simple functionality seems like an eternity…

The obvious thing was to change the code to the documented ^{[6]} faster (vectorized) version:

```
>> tic; myStruct = rmfield(myStruct,fieldNames); t2 = toc
t2 =
0.6097
```

This is obviously much better but since rmfield is called many times in my application, I needed something even better. So I profiled rmfield and was not happy with the result.

The original code of * rmfield* (

```
function t = rmfield(s,field)
% get fieldnames of struct
f = fieldnames(s);
% Determine which fieldnames to delete.
idxremove = [];
for i=1:length(field)
j = find(strcmp(field{i},f) == true);
idxremove = [idxremove;j];
end
% set indices of fields to keep
idxkeep = 1:length(f);
idxkeep(idxremove) = [];
% remove the specified fieldnames from the list of fieldnames.
f(idxremove,:) = [];
% convert struct to cell array
c = struct2cell(s);
% find size of cell array
sizeofarray = size(c);
newsizeofarray = sizeofarray;
% adjust size for fields to be removed
newsizeofarray(1) = sizeofarray(1) - length(idxremove);
% rebuild struct
t = cell2struct(reshape(c(idxkeep,:),newsizeofarray),f);
```

When I profiled the code, the highlighted row was the bottleneck I was looking for.

First, I noticed the string comparison equals to `true`

part – while `'==true'`

is not the cause of the bottleneck, it does leave an impression of bad coding style 🙁 Perhaps this code was created as some apprentice project, which might also explain its suboptimal performance.

The real performance problem here is that for each field that we wish to remove, * rmfield* compares it to all existing fields to find its location in a cell array of field names. This is algorithmically inefficient and makes the code hard to understand (just try – it took me hard, long minutes).

So, I created a variant of

```
function t = fast_rmfield(s,field)
% get fieldnames of struct
f = fieldnames(s);
[f,ia] = setdiff(f,field,'R2012a');
% convert struct to cell array
c = squeeze(struct2cell(s));
% rebuild struct
t = cell2struct(c(ia,:),f)';
```

This code is much shorter, easier to explain and maintain, but also (and most importantly) much faster:

```
>> tic; myStruct = fast_rmfield(myStruct,fieldNames); t3 = toc
t3 =
0.0302
>> t2/t3
ans =
20.1893
```

This resulted in a speedup of ~850x compared to the original version (of 25.77 secs), and ~20x compared to the vectorized version. A nice improvement in my humble opinion…

The point in all this is that we can and should rewrite Matlab builtin functions when they are too slow for our needs, whether it is found to be an algorithmic flaw (as in this case), extraneous sanity checks (as in the case of **ismember**^{[7]} or **datenum**^{[8]}), bad default parameters (as in the case of **fopen/fwrite**^{[9]} or **scatter**^{[10]}), or merely slow implementation (as in the case of **save**^{[11]}, **cellfun**^{[12]}, or the * conv* family of functions

A good pattern is to save such code pieces in file names that hint to the original code. In our case, I used

Categories: Guest bloggers, Low risk of breaking in future versions, Stock Matlab function

5 Comments (Open | Close)

Article printed from Undocumented Matlab: **https://undocumentedmatlab.com**

URL to article: **https://undocumentedmatlab.com/articles/rmfield-performance**

URLs in this post:

[1] Hanan Kavitz: **http://www.mathworks.com/matlabcentral/profile/authors/2392999-hanan-kavitz**

[2] Applied Materials: **http://appliedmaterials.com**

[3] quirks with compiled Matlab DLLs: **http://undocumentedmatlab.com/blog/quirks-with-compiled-matlab-dlls**

[4] “Performance” articles: **http://undocumentedmatlab.com/tag/performance**

[5] Accelerating MATLAB Performance: **http://undocumentedmatlab.com/books/matlab-performance**

[6] documented: **http://www.mathworks.com/help/matlab/ref/rmfield.html**

[7] * ismember*:

[8] * datenum*:

[9] * fopen/fwrite*:

[10] * scatter*:

[11] * save*:

[12] * cellfun*:

[13] * conv* family of functions:

[14] Matlab-Java memory leaks, performance : **https://undocumentedmatlab.com/articles/matlab-java-memory-leaks-performance**

[15] Callback functions performance : **https://undocumentedmatlab.com/articles/callback-functions-performance**

[16] Array resizing performance : **https://undocumentedmatlab.com/articles/array-resizing-performance**

[17] Improving save performance : **https://undocumentedmatlab.com/articles/improving-save-performance**

[18] Plot performance : **https://undocumentedmatlab.com/articles/plot-performance**

[19] uicontextmenu performance : **https://undocumentedmatlab.com/articles/uicontextmenu-performance**

[20] : **http://www.mathworks.com/matlabcentral/fileexchange/23606-fast-and-efficient-kronecker-multiplication**

Click here to print.

Copyright © Yair Altman - Undocumented Matlab. All rights reserved.

5 Comments To "rmfield performance"

#1 CommentByPeterOn May 26, 2016 @ 03:08I recently wrote a function that needed to calculate many many thousands of dot products. When I profiled my function, it was spending a ton of time in the

function. When I openeddotdot.m, it was mostly sanity checks I didn’t need, so I just inlined the dot calculation. The function went from minutes to about a second to complete.#2 CommentByYair AltmanOn May 26, 2016 @ 13:02@Peter – excellent usage example. Thanks for sharing.

#3 CommentByFernandoOn June 5, 2016 @ 21:36This is really cool. I remember years ago having to accelerate an algorithm that used Matlab’s bultin kronecker tensor product. Luckily, I was able to find this:

^{[20]}#4 CommentByMalcolm LidierthOn June 6, 2016 @ 11:34I found this too with a package that was heavily profiled up to R2012a. Maybe things have changed as JIT acceleration has improved but there were two ‘tricks’ I used often.

Conditional statements were frequently the bottleneck, but served little purpose in the specific context e.g.

could often safely be replaced with

Also, a try-catch sequence in place of conditional tests was often faster.

ML

#5 CommentByHoi WongOn January 30, 2017 @ 21:32Thanks for pointing that out. I didn’t even notice (or expect) that rmfield() is not a built-in low level function.

Since the rmfield() code calls struct2cell() then cell2struct(), it looks like it’s saying that behind the scene, struct() is basically a high level wrapper around cells using hash keys to map the name to indices: a useful piece of information to keep in mind for performance tuning. Actually, table() or dataset() object deals with cells under the hood, I just wasn’t expecting struct() to be the same given its origins in C.

I found rmfield() so slow that I’ve actually written a keepField() long time ago for the exact same reason as your application scenario: if I need to remove 5000 fields, I might as well keep what I want by adding to an empty (fieldless) struct one-field at a time. i.e.

It turned out to be much faster too because there are no names to search for. Dynamic field names are done with hash table (checked with TMW, it’s not documented), so it’s on average O(1) time. It boils down to the same O(nlog(n)) time as the setdiff() proposed if you ultimately have to identify the fields to remove instead.

Unfortunately MATLAB has only rmfield(), so I suspect a lot of people might have done a set-op (spent O(nlog(n)) time) to get the list to keep (the complementary set to remove) then run through the O(n) algorithm in rmfield() when they could have done it in average O(1) time by just transferring the wanted fields.