Class object tab completion & improper field names

I recently consulted to a very large industrial client. Following a merger/acquisition, the relevant R&D department had two development groups using different technologies: one group uses Matlab, the other does not. My client wanted both groups to use Matlab, something that would naturally please MathWorks.

Unfortunately, it turns out that a technical challenge was preventing this move: the other technology enabled data field names (identifiers) longer than Matlab’s namelengthmax=63 characters, and these names also sometimes contained illegal identifier characters, such as spaces or symbols. This prevented an easy code transition, indefinitely delaying the migration to Matlab.

Accessing improper fieldnames

I suggested to this client to use Matlab class objects that overloaded the subsref() and subsasgn() methods: the long original identifiers would be stored in some internal container (cell array or containers.Map etc.), and they would be accessed not directly (which would be impossible since they are not valid Matlab identifiers) but via the overloaded methods. Something along the following lines:

classdef ExtendedFieldsClass
    % The internal data implementation is not publicly exposed
    properties (Access = 'protected')
        props = containers.Map;
    end
 
    methods (Access = 'public', Hidden=true)
 
        % Overload property assignment
        function obj = subsasgn(obj, subStruct, value)
            if strcmp(subStruct.type,'.')
                try
                    obj.props(subStruct.subs) = value;
                catch
                    error('Could not assign "%s" property value', subStruct.subs);
                end
            else  % '()' or '{}'
                error('not supported');
            end
        end
 
        % Overload property retrieval (referencing)
        function value = subsref(obj, subStruct)
            if strcmp(subStruct.type,'.')
                try
                    value = obj.props(subStruct.subs);
                catch
                    error('"%s" is not defined as a property', subStruct.subs);
                end
            else  % '()' or '{}'
                error('not supported');
            end
        end
    end
end

This works splendidly, as the following snippet shows:

>> c = ExtendedFieldsClass
c =
  ExtendedFieldsClass with no properties.
 
>> c.(' asd f @#$^$%&') = -13.5;  % no error
 
>> c.(' asd f @#$^$%&')
ans =
                     -13.5
 
>> c.(' asd f @#$^$%& xyz')  % note the extra "xyz"
Error using ExtendedFieldsClass/subsref (line 27)
" asd f @#$^$%& xyz" is not defined as a property

Note how we need to use the () parentheses in order to access the “properties” as dynamic fieldnames. We would naturally get an error if we tried to directly access the field:

>> c. asd f @#$^$%&
 c. asd f @#$^$%&
        |
Error: Unexpected MATLAB expression.

Tab completion

So far so good.

The problem is that we would also like to see the defined “properties” when in the desktop’s tab completion. i.e., when I type “c.” and then click <tab> in the Matlab command prompt, I’d like to see the list of defined “properties” in a tooltip (in the example above: ” asd f @#$^$%&”). Instead, I get the message “No Completions Found.”:

Missing tab completion

Missing tab completion

I described the hack for desktop tab-completion a few years ago. Unfortunately, that hack only works for functions. We need to find another solution for Matlab class objects.

The solution is to overload the fieldnames() function as well, such that it would return a cell-array of the relevant strings:

classdef ExtendedFieldsClass
    % The internal data implementation is not publicly exposed
    properties (Access = 'protected')
        props = containers.Map;
    end
 
    methods (Access = 'public', Hidden=true)
 
        % Overload property assignment
        function obj = subsasgn(obj, subStruct, value)
            ...  (as above)
        end
 
        % Overload property retrieval (referencing)
        function value = subsref(obj, subStruct)
            ...  (as above)
        end
 
        % Overload fieldnames retrieval
        function names = fieldnames(obj)            names = sort(obj.props.keys);  % return in sorted order        end    end
end

When we now run this in the command prompt, we get the expected behavior:

Working tab completion

Working tab completion

R2014a

Unfortunately, this works only up to and including Matlab release R2013b. In R2014a, MathWorks made some internal change that prevents overloading the fieldnames function. To be more precise, we can still overload it as above, and it will indeed work if we directly call fieldnames(c), but it no longer has any effect on the tab completion. On R2014a, the tab-completion remains broken and returns “No Completions Found.” When this was reported to MathWorks some time ago, the official response was that the previous behavior was considered a “bug”, and this was “fixed” in R2014a (don’t bother searching for it in the official bugs parade). Go figure…

So what do you think I should now do? Remember: this is a large client, who knows how many licenses are at stake. Should I suggest to my client not to switch to Matlab? Or should I suggest that they keep using R2013b across the entire organization and cancel their annual maintenance? Or maybe I should simply tell them to accept the fact that some important functionality should be expected to get broken whenever Matlab is upgraded?

These sort of things just blow me away. Sometimes I feel as if I am swimming against the current, and that’s frustrating. I admit it doesn’t happen very often. Then again, I guess if things were straight-forward, nobody would need me to consult them…

Don’t mind me – just blowing off some steam. I’m allowed to, every now and then, aren’t I? :-)

Addendum July 21, 2014: I found out today that on R2014a+ we can simply overload the properties method. This is a function that returns the properties of a class, and so it makes perfect sense for a class object’s tab-completion to use properties rather than fieldnames. So I can now indeed see why the past behavior was considered by MathWorks to be a bug that should be fixed. Still, it would have been nice if for backward-compatibility considerations, Matlab (or at least mlint) would have detected the fact that fieldnames is being overloaded in a user class and warned/alerted regarding the fact that we should now overload properties. In any case, to be fully backward compatible, simply overload both methods, and make one of them call the other. For example:

        % Overload property names retrieval
        function names = properties(obj)
            names = fieldnames(obj);
        end

My client would be quite happy to hear of this new development :-)

Michal Kutil described a mechanism for overloading the methods function, which is also part of the tab-completion tooltip. The problem here is that we cannot simply overload methods in our class, since Matlab calls methods with the class name (not the class object reference) when it wants to determine the relevant methods to display in the tooltip. Michal’s solution was to create a wrapper function that calls the overloaded variant. This wrapper function can then be placed within a @char folder somewhere in the Matlab path. I used a similar trick for my profile_history utility last month.

Related newsgroup posts by Eric Salemi here and here.

Similarly, in order to overload the data-value tooltip (when hovering over the object in the editor), or when displaying the object in the Matlab command prompt, simply overload the disp() function (see related):

        % Overload class object display
        function disp(obj)
            disp([obj.props.keys', obj.props.values']);  % display as a cell-array
        end

In a related matter, we can limit the values that a property can accept using the matlab.system.StringSet class of the matlab.System package, as recently discovered by Oleg Komarov (additional details; a different way to constrict property data type):

classdef foo < matlab.System
    properties
        Coordinates
    end
    properties(Hidden,Transient)
        CoordinatesSet = matlab.system.StringSet({'north','south','east','west'});
    end
end

restricting Matlab property values

This blog will now take a short vacation for a few weeks, due to my U.S. trip. I will return with some fresh material in August – stay tuned!

Related posts:

  1. Class object creation performance Performance aspects of Matlab class object creation are discussed, with specific suggestions. ...
  2. Setting class property types Matlab's class properties have a simple and effective mechanism for setting their type....
  3. Accessing private object properties Private properties of Matlab class objects can be accessed (read and write) using some undocumented techniques. ...
  4. getundoc – get undocumented object properties getundoc is a very simple utility that displays the hidden (undocumented) properties of a specified handle object....
  5. Creating a simple UDD class This article explains how to create and test custom UDD packages, classes and objects...
  6. Extending a Java class with UDD Java classes can easily be extended in Matlab, using pure Matlab code. ...

Categories: Desktop, Medium risk of breaking in future versions, Undocumented feature

Tags: , , ,

Bookmark and SharePrint Print

12 Responses to Class object tab completion & improper field names

  1. seb says:

    Out of curiosity – if that doesn’t contradict any NDA:
    What technology allows “fieldnames” or property names to contain special characters and even whitespaces?

    I’d go for a consulting project helping them to get rid of that mess :>

    • @Seb – I believe it was names of data channels that contained spaces, dots (.) and dashes (-). These could be valid channel names yet still illegal Matlab identifiers. In the real world, such channel names are quite common. I added the special symbols in my example just to illustrate the point.

  2. David says:

    I’m sorry but this is just terrible. Why on earth did you develop such a hack? Your time would have been far better spent sorting out their mess like Seb suggested. Now they don’t have any reason to tidy up!

  3. TheBlackCat says:

    Shouldn’t you be advising them to use the solution that best suits their needs? Sometimes that isn’t Matlab. If I was hiring a consultant that is what I would want them to do.

    • Ouch, that hurt. It’s a pity that after so many years, you underestimate me so much. My client could not modify the legacy system. The options were either to keep that system and the discrepancies vs. Matlab (which made R&D difficult across the two groups), or to find a way to migrate to Matlab, or to redo everything from scratch (at a huge R&D cost). My client decided to migrate to Matlab before they contacted me, but hit the problem I described above. They came to me asking for a solution after MathWorks support and other consultants were not able to help them solve the problem. I was able to give them a pointer in this direction which nobody else was able to do before then. Having said that, I did warn them that it might not be cost-effective. But now at least my client has an alternative that they can consider vs. their current ongoing development difficulties – an alternative that they did not have beforehand.

    • TheBlackCat says:

      Fair enough. However, from those two paragraphs, you seem to put a lot of focus on what would benefit you. I hope you can see how that sort of thing could give the wrong impression.

    • TheBlackCat says:

      I would like to apologize. I am sure you are trying your best to give the best advice possible to us and your clients. I do think those paragraphs are easy to misunderstand and I would probably have phrased them differently. That was really my ultimate point, but considering what a terrible job I did expressing myself I am probably the last person who should be giving that sort of advice. So please accept my apology and believe me when I say I do not doubt your integrity.

    • Thanks for letting me know about this. I can see how this might indeed be seen in that light and so I have removed the relevant phrases.

  4. Amro says:

    All of these restrictions are on the MATLAB side, the underlying MEX API actually does not impose such limitations.

    I implemented a MEX-function “my_setfield.cpp” to illustrate: http://pastebin.com/j69kQEur
    To keep it simple, I only handle scalar structures, but you can extend the code to work with struct arrays..

    Here is an example showing the use of illegal field names:

    % create struct
    s = struct();
    s = my_setfield(s, 'a a', 1);
    s = my_setfield(s, 'b.b', 2);
    s = my_setfield(s, 'c-c', 3);
    s = my_setfield(s, '1dd', 4);
    s = my_setfield(s, repmat('e',1,70), 5);
    s = my_setfield(s, ' asd f @#$^$%&amp;', 6);
     
    % retrieve fields using "dynamic field names" syntax
    disp(s)
    a = s.('a a')
    b = s.('b.b')
    c = s.('c-c')
    d = s.('1dd')
    %e = s.(repmat('e',1,70))   % this one didn't work :)
    f = s.(' asd f @#$^$%&amp;')

    (Note: you can workaround the “namelengthmax” restriction by similarly implementing a C++ MEX-function that uses mxGetField/mxGetFieldByNumber).

  5. Daniel says:

    I find it completely unprofessional that the official “bug list” doesn’t list all known bugs.

  6. Pingback: Setting class property types | Undocumented Matlab

  7. Lorin says:

    I am currently using R2014a and are using Michal’s solution to attempt to hide handle superclass methods (addlistener.m, etc) when using tab-completion. I’m not getting the desired result using R2014a.

    In 2013a and 2013b, I am able to hide handles’ methods by inheriting from a class that overloading the functions (addlistener.m, etc) and gives them access of ‘Hidden’. As seen in the stackoverflow questions below:
    http://stackoverflow.com/questions/6621850/is-it-possible-to-hide-the-methods-inherited-from-the-handle-class-in-matlab

    My question is: Is anyone aware of changes made to R2014a+ that would not allow overloading of ‘methods’ and thus not resolve the hiding of handles’ methods when using the tab-completion tool tip?

Leave a Reply

Your email address will not be published. Required fields are marked *

*

<pre lang="matlab">
a = magic(3);
sum(a)
</pre>