I recently consulted to a very large industrial client. Following a merger/acquisition, the relevant R&D department had two development groups using different technologies: one group uses Matlab, the other does not. My client wanted both groups to use Matlab, something that would naturally please MathWorks.
Unfortunately, it turns out that a technical challenge was preventing this move: the other technology enabled data field names (identifiers) longer than Matlab’s namelengthmax=63 characters, and these names also sometimes contained illegal identifier characters, such as spaces or symbols. This prevented an easy code transition, indefinitely delaying the migration to Matlab.
Accessing improper fieldnames
I suggested to this client to use Matlab class objects that overloaded the subsref() and subsasgn() methods: the long original identifiers would be stored in some internal container (cell array or containers.Map etc.), and they would be accessed not directly (which would be impossible since they are not valid Matlab identifiers) but via the overloaded methods. Something along the following lines:
classdef ExtendedFieldsClass
% The internal data implementation is not publicly exposed
properties (Access = 'protected')
props = containers.Map;
end
methods (Access = 'public', Hidden=true)
% Overload property assignment
function obj = subsasgn(obj, subStruct, value)
if strcmp(subStruct.type,'.')
try
obj.props(subStruct.subs) = value;
catch
error('Could not assign "%s" property value', subStruct.subs);
end
else % '()' or '{}'
error('not supported');
end
end
% Overload property retrieval (referencing)
function value = subsref(obj, subStruct)
if strcmp(subStruct.type,'.')
try
value = obj.props(subStruct.subs);
catch
error('"%s" is not defined as a property', subStruct.subs);
end
else % '()' or '{}'
error('not supported');
end
end
end
end |
This works splendidly, as the following snippet shows:
>> c = ExtendedFieldsClass c = ExtendedFieldsClass with no properties. >> c.(' asd f @#$^$%&') = -13.5; % no error >> c.(' asd f @#$^$%&') ans = -13.5 >> c.(' asd f @#$^$%& xyz') % note the extra "xyz" Error using ExtendedFieldsClass/subsref (line 27) " asd f @#$^$%& xyz" is not defined as a property
Note how we need to use the () parentheses in order to access the “properties” as dynamic fieldnames. We would naturally get an error if we tried to directly access the field:
>> c. asd f @#$^$%&
c. asd f @#$^$%&
|
Error: Unexpected MATLAB expression.
Tab completion
So far so good.
The problem is that we would also like to see the defined “properties” when in the desktop’s tab completion. i.e., when I type “c.” and then click <tab> in the Matlab command prompt, I’d like to see the list of defined “properties” in a tooltip (in the example above: ” asd f @#$^$%&”). Instead, I get the message “No Completions Found.”:
I described the hack for desktop tab-completion a few years ago. Unfortunately, that hack only works for functions. We need to find another solution for Matlab class objects.
The solution is to overload the fieldnames() function as well, such that it would return a cell-array of the relevant strings:
classdef ExtendedFieldsClass
% The internal data implementation is not publicly exposed
properties (Access = 'protected')
props = containers.Map;
end
methods (Access = 'public', Hidden=true)
% Overload property assignment
function obj = subsasgn(obj, subStruct, value)
... (as above)
end
% Overload property retrieval (referencing)
function value = subsref(obj, subStruct)
... (as above)
end
% Overload fieldnames retrieval
function names = fieldnames(obj)
names = sort(obj.props.keys); % return in sorted order
end
end
end |
When we now run this in the command prompt, we get the expected behavior:
R2014a
Unfortunately, this works only up to and including Matlab release R2013b. In R2014a, MathWorks made some internal change that prevents overloading the fieldnames function. To be more precise, we can still overload it as above, and it will indeed work if we directly call fieldnames(c), but it no longer has any effect on the tab completion. On R2014a, the tab-completion remains broken and returns “No Completions Found.” When this was reported to MathWorks some time ago, the official response was that the previous behavior was considered a “bug”, and this was “fixed” in R2014a (don’t bother searching for it in the official bugs parade). Go figure…
So what do you think I should now do? Remember: this is a large client, who knows how many licenses are at stake. Should I suggest to my client not to switch to Matlab? Or should I suggest that they keep using R2013b across the entire organization and cancel their annual maintenance? Or maybe I should simply tell them to accept the fact that some important functionality should be expected to get broken whenever Matlab is upgraded?
These sort of things just blow me away. Sometimes I feel as if I am swimming against the current, and that’s frustrating. I admit it doesn’t happen very often. Then again, I guess if things were straight-forward, nobody would need me to consult them…
Don’t mind me – just blowing off some steam. I’m allowed to, every now and then, aren’t I? 🙂
Addendum July 21, 2014: I found out today that on R2014a+ we can simply overload the properties method. This is a function that returns the properties of a class, and so it makes perfect sense for a class object’s tab-completion to use properties rather than fieldnames. So I can now indeed see why the past behavior was considered by MathWorks to be a bug that should be fixed. Still, it would have been nice if for backward-compatibility considerations, Matlab (or at least mlint) would have detected the fact that fieldnames is being overloaded in a user class and warned/alerted regarding the fact that we should now overload properties. In any case, to be fully backward compatible, simply overload both methods, and make one of them call the other. For example:
% Overload property names retrieval function names = properties(obj) names = fieldnames(obj); end |
My client would be quite happy to hear of this new development 🙂
Related odds and ends
Michal Kutil described a mechanism for overloading the methods function, which is also part of the tab-completion tooltip. The problem here is that we cannot simply overload methods in our class, since Matlab calls methods with the class name (not the class object reference) when it wants to determine the relevant methods to display in the tooltip. Michal’s solution was to create a wrapper function that calls the overloaded variant. This wrapper function can then be placed within a @char folder somewhere in the Matlab path. I used a similar trick for my profile_history utility last month.
Related newsgroup posts by Eric Salemi here and here.
Similarly, in order to overload the data-value tooltip (when hovering over the object in the editor), or when displaying the object in the Matlab command prompt, simply overload the disp() function (see related):
% Overload class object display function disp(obj) disp([obj.props.keys', obj.props.values']); % display as a cell-array end |
In a related matter, we can limit the values that a property can accept using the matlab.system.StringSet class of the matlab.System package, as recently discovered by Oleg Komarov (additional details; a different way to constrict property data type):
classdef foo < matlab.System properties Coordinates end properties(Hidden,Transient) CoordinatesSet = matlab.system.StringSet({'north','south','east','west'}); end end |
This blog will now take a short vacation for a few weeks, due to my U.S. trip. I will return with some fresh material in August – stay tuned!
Out of curiosity – if that doesn’t contradict any NDA:
What technology allows “fieldnames” or property names to contain special characters and even whitespaces?
I’d go for a consulting project helping them to get rid of that mess :>
@Seb – I believe it was names of data channels that contained spaces, dots (.) and dashes (-). These could be valid channel names yet still illegal Matlab identifiers. In the real world, such channel names are quite common. I added the special symbols in my example just to illustrate the point.
I’m sorry but this is just terrible. Why on earth did you develop such a hack? Your time would have been far better spent sorting out their mess like Seb suggested. Now they don’t have any reason to tidy up!
Shouldn’t you be advising them to use the solution that best suits their needs? Sometimes that isn’t Matlab. If I was hiring a consultant that is what I would want them to do.
Ouch, that hurt. It’s a pity that after so many years, you underestimate me so much. My client could not modify the legacy system. The options were either to keep that system and the discrepancies vs. Matlab (which made R&D difficult across the two groups), or to find a way to migrate to Matlab, or to redo everything from scratch (at a huge R&D cost). My client decided to migrate to Matlab before they contacted me, but hit the problem I described above. They came to me asking for a solution after MathWorks support and other consultants were not able to help them solve the problem. I was able to give them a pointer in this direction which nobody else was able to do before then. Having said that, I did warn them that it might not be cost-effective. But now at least my client has an alternative that they can consider vs. their current ongoing development difficulties – an alternative that they did not have beforehand.
Fair enough. However, from those two paragraphs, you seem to put a lot of focus on what would benefit you. I hope you can see how that sort of thing could give the wrong impression.
I would like to apologize. I am sure you are trying your best to give the best advice possible to us and your clients. I do think those paragraphs are easy to misunderstand and I would probably have phrased them differently. That was really my ultimate point, but considering what a terrible job I did expressing myself I am probably the last person who should be giving that sort of advice. So please accept my apology and believe me when I say I do not doubt your integrity.
Thanks for letting me know about this. I can see how this might indeed be seen in that light and so I have removed the relevant phrases.
All of these restrictions are on the MATLAB side, the underlying MEX API actually does not impose such limitations.
I implemented a MEX-function “my_setfield.cpp” to illustrate: http://pastebin.com/j69kQEur
To keep it simple, I only handle scalar structures, but you can extend the code to work with struct arrays..
Here is an example showing the use of illegal field names:
(Note: you can workaround the “namelengthmax” restriction by similarly implementing a C++ MEX-function that uses mxGetField/mxGetFieldByNumber).
I find it completely unprofessional that the official “bug list” doesn’t list all known bugs.
[…] additional aspects of class property tab-completion […]
I am currently using R2014a and are using Michal’s solution to attempt to hide handle superclass methods (addlistener.m, etc) when using tab-completion. I’m not getting the desired result using R2014a.
In 2013a and 2013b, I am able to hide handles’ methods by inheriting from a class that overloading the functions (addlistener.m, etc) and gives them access of ‘Hidden’. As seen in the stackoverflow questions below:
http://stackoverflow.com/questions/6621850/is-it-possible-to-hide-the-methods-inherited-from-the-handle-class-in-matlab
My question is: Is anyone aware of changes made to R2014a+ that would not allow overloading of ‘methods’ and thus not resolve the hiding of handles’ methods when using the tab-completion tool tip?
I am creating some custom classes for a project. I am overloading the subsasgn and subsref methods and that is working wonderfully. I tried overloading the properties method so I could provide a list of specific properties and I would say it is 50/50 working and not working. This is what I mean by 50/50 working:
I also tried overloading the fieldnames method as well but I still get the no completions found message. Any ideas of how I can get this to work?
Being I am already overloading the subsasgn and subsref methods I do realize I could change them such so that instead of doing
obj(i).property_nameI could useobj.property_name(i)and then I think the tab completion would work. However, my custom class (obj) actually contains an array of a different custom class. So it seems more natural to specify the objects I want first,obj(i)and then the property to get/set,obj(i).property_name.Hi Yair,
I came across a use case where sadly I needed to use this approach to support arbitrary variable names. Any idea on how to support tab completion within the parentheses? e.g.:
c.(‘ %tab complete after the quote
Thanks,
Jim
Matlab seems to catch on to
mustBeMemberand offers tab completion options when it is used.