User-defined tab completions – take 2

Back in 2010, I posted about Matlab’s undocumented mechanism for setting Matlab desktop tab-completions. That mechanism used a couple of internal files (TC.xml and TC.xsd) to describe the various possible options that are auto-completed (or displayed in a small tooltip window) when the user clicks the <Tab> key on partially-entered function input parameters.

Using TabComplete for user-defined functions

Using TabComplete for user-defined functions

Unfortunately, this mechanism apparently broke in R2016a and was replaced with a new mechanism, as explained below.

The new mechanism relies on a file called functionSignatures.json which exists in every single folder that contains Matlab files that have functions whose input parameters ought to be tab-completable.

The new mechanism offers far greater versatility and flexability in defining the input types and inter-relationsships compared to the old TC.*-based mechanism. Another important benefit is that we can now add custom user-defined functionSignatures.json files to our user folders, next to our m-files, without having to modify any Matlab system file.

Note that you may need to restart Matlab the first time that you create a functionSignatures.json file. But once it’s created, you can modify it within a Matlab session and the changes take effect immediately.

Note: Credit for first posting about this new mechanism goes to Nicholas Mati. I’ve known about this new mechanism for over a year, but I never found the time to write about it until now, so Nicholas gets credit for breaking the scoop. The discussion below uses and expands Nicholas’ original post.

Syntax

The functionSignatures.json file has the general form:

{
	"FunctionName1":
	{
		"key1":"val1",
		"key2":"val2",
		"keyn":"valn"
	},
	"FunctionName2":
	{
		"key1":"val1",
		"key2":"val2",
		"keyn":"valn"
	}
}

A number of keys are supported including “platform“, “setsAns“, “inputs“, and “outputs“, although inputs and outputs are by far the most common (and presumably only inputs are relevant for tab-completion). These keys take an array of (or a single) object value(s). The objects typically take one of the following forms:

{"name":"variable_name", "kind":"kind_option", "type":"string_or_array_of_type"}
{"mutuallyExclusiveGroup":
	[
		...
	]
}
{"name":"varargin", "kind":"optional", "multiplicity":"append"}

The value for “kind” can be “required”, “optional”, “positional”, “flag”, “namevalue” or “platform” (and perhaps a few other lesser-used kinds):

  • required” means that the specified input is mandatory
  • optional” means that it can be added or omitted
  • positional” means that it’s an optional input but if it is specified then it must appear at the specified position relative to the previous (earlier) inputs
  • flag” means that it’s an optional input flag, from a predefined list of one or more single-token strings. For example, in regexp(s1,s2,'once') the last input arg ('once') is such a flag.
  • namevalue” means that it follows Matlab’s standard practice of using P-V pairs (parameter name followed by its value). For example, func('propName',propValue)
  • platform” indicates that this input is only available on the specified platform(s)

These “kind”s are all explained below.

The value for “type” can be a string such as “char” or “numeric” or “filepath”, or a more complicated JSON array (see below).

In addition to “name”, “kind” and “type”, we can also define a “default” value (e.g. "default":"false") and a “display” string. While these are [currently] not used by Desktop tab-completion, they might be used by other components such as the JIT compiler or the Editor, if not today then perhaps in a future release.

Note that while pure JSON format does not accept comments, Matlab’s functionSignatures.json does accept C++-style comments, as discovered by Heiko in a comment below. To add a comment, simply add // comment text at the end of any line, or /* comment text */ anywhere within a line.

Usage examples

Multiple examples of functionSignatures.json files can be found in subfolders of %matlabroot%/toolbox/matlab. For example, here’s the tab-completion definition for the visdiff function, which displays a visual comparison between two files, and resides in %matlabroot%/toolbox/shared/comparisons/functionSignatures.json:

{
"visdiff":
{
    "inputs":
    [
        {"name":"filename1", "kind":"required",   "type":"filepath"},
        {"name":"filename2", "kind":"required",   "type":"filepath"},
        {"name":"type",      "kind":"positional", "type":"choices={'text', 'binary'}"}
    ]
}
}

As can be seen in this example, the first and second inputs are expected to be a filename, whereas the third input is one of the two predefined strings ‘text’ or ‘binary’. This third input has “kind”:”positional”, meaning that it is optional, but if it is provided then it must be in the 3rd position and cannot appear sooner. Moreover, if the user specifies any input argument to the “right” of a positional input, then the positional argument becomes required, not optional.

Whereas a “positional” parameter has a specific position in the args list (#3 in the case of visdiff above), an “optional” parameter may appear anywhere in the list of inputs.

Here’s a more complex example, for the built-in regexprep function (in %matlabroot%/toolbox/matlab/strfun/functionSignatures.json). This example shows how to limit the input to certain data types and how to specify optional input flags with pre-defined choices:

"regexprep":
{
	"inputs":
	[
		{"name":"str",               "kind":"required",  "type":[["char"], ["cell"], ["string"]]},
		{"name":"expression",        "kind":"required",  "type":[["char"], ["cell"], ["string"]]},
		{"name":"replace",           "kind":"required",  "type":[["char"], ["cell"], ["string"]]},
		{"name":"optMatch",          "kind":"flag",      "display":"", "type":[["char", "choices={'all','once'}"], ["numeric", "scalar"]],   "default":"'all'"},
		{"name":"optWarnings",       "kind":"flag",      "display":"", "type":["char", "choices={'nowarnings','warnings'}"],                 "default":"'nowarnings'"},
		{"name":"optCase",           "kind":"flag",      "display":"", "type":["char", "choices={'matchcase','ignorecase','preservecase'}"], "default":"'matchcase'"},
		{"name":"optEmptyMatch",     "kind":"flag",      "display":"", "type":["char", "choices={'noemptymatch','emptymatch'}"],             "default":"'noemptymatch'"},
		{"name":"optDotAll",         "kind":"flag",      "display":"", "type":["char", "choices={'dotall','dotexceptnewline'}"],             "default":"'dotall'"},
		{"name":"optStringAnchors",  "kind":"flag",      "display":"", "type":["char", "choices={'stringanchors','lineanchors'}"],           "default":"'stringanchors'"},
		{"name":"optSpacing",        "kind":"flag",      "display":"", "type":["char", "choices={'literalspacing','freespacing'}"],          "default":"'literalspacing'"}
	],
	"outputs":
	[
		{"name":"newStr", "type":[["char"], ["cell"], ["string"]]}
	]
},

Here’s an even more complex example, this time for the codegen function (in %matlabroot%/toolbox/coder/matlabcoder/functionSignatures.json, part of the Matlab Coder toolbox). This example shows how to limit the filenames to certain extensions and how to specify name-value input pairs:

"codegen":
{
	"inputs":
	[
		{"name":"compile_only",  "kind":"flag",       "type":"choices={'-c'}"},
		{"name":"config_flag",   "kind":"flag",       "type":"choices={'-config:mex','-config:lib','-config:dll','-config:exe','-config:hdl'}"},
		{"name":"debug",         "kind":"flag",       "type":"choices={'-g'}"},
		{"name":"report",        "kind":"flag",       "type":"choices={'-report'}"},
		{"name":"launchreport",  "kind":"flag",       "type":"choices={'-launchreport'}"},
		{"name":"file",          "kind":"flag",       "type":"filepath=*.m,*.mlx,*.c,*.cpp,*.h,*.o,*.obj,*.a,*.so,*.lib,*.tmf", "multiplicity":"append"},
		{"name":"-d",            "kind":"namevalue",  "type":"folderpath"},
		{"name":"-I",            "kind":"namevalue",  "type":"folderpath"},
		{"name":"-globals",      "kind":"namevalue"},
		{"name":"-o",            "kind":"namevalue",  "type":[["char"], ["filepath"]]},
		{"name":"-O",            "kind":"namevalue",  "type":"choices={'enable:inline','disable:inline','enable:blas','disable:blas','enable:openmp','disable:openmp'}"},
		{"name":"-args",         "kind":"namevalue",  "type":[["identifier=variable"], ["char"]]},
		{"name":"-config",       "kind":"namevalue",  "type":[["identifier=variable"], ["char"]]},
		{"name":"verbose",       "kind":"flag",       "type":"choices={'-v'}"},
		{"name":"singleC",       "kind":"flag",       "type":"choices={'-singleC'}"},
		{"name":"-test",         "kind":"namevalue",  "type":"identifier=function"}
	]
},

Argument types

As noted above, we use "type":... to specify the expected data type of each parameter. This can be a simple string such as “char”, “cellstr”, “numeric”, “table”, “categorical”, “filepath”, “folderpath”, “matlabpath”, class name, or a more complicated JSON array. For example:

  • "type":["numeric","scalar"]
  • "type":["numeric","numel=3",">=4"]
  • "type":[["char"], ["cellstr"], ["numeric"], ["logical","vector"]]
  • "type":[["char", "choices={'-ascii'}"]]
  • "type":[["filepath"], ["matlabpath=*.m,*.mlx"], ["char"]]
  • "type":"identifier=variable,function,localfunction,package,classdef"
  • "type":"matlab.graphics.axis.Axes"
  • "type":"choices={'yes','no','maybe'}"

We can even specify on-the-fly Matlab computation that returns a cell-array of values, for example a list of available fonts via "type":"choices=listfonts". A more complex example is the definition of the rmfield function, where the possible input choices for the second input arg (highlighted) depend on the struct that is provided in the first input arg (by running the fieldnames function on it):

"rmfield":
{
	"inputs":
	[
		{"name":"s",     "kind":"required", "type":"struct"},
		{"name":"field", "kind":"required", "type":"choices=fieldnames(s)"}	],
	"outputs":
	[
		{"name":"s", "type":"struct"}
	]
},

Alternative inputs

Multiple alternative inputs can be specified in the functionSignatures.json file. The easiest way to do so is to simply create multiple different definitions for the same function, one beneath the other. Matlab’s tab-completion parser is smart enough to combine those definitions and proceed with the most appropriate one based on the user-entered inputs.

For example, in the same Coder file above we find 6 alternative definitions. If (for example) we start typing coder('-ecoder', and click <Tab>, Matlab would automatically auto-complete the second input to “false”, and then another <Tab> click would set the third input to the required ‘-new’ parameter (see highlighted lines below):

...
"coder":
{
	"inputs":
	[
		{"name":"projectname", "kind":"required", "type":"filepath=*.prj"}
	]
},
"coder":
{
	"inputs":
	[
		{"name":"-open", "kind":"namevalue", "type":"filepath=*.prj"}
	]
},
"coder":
{
	"inputs":
	[
		{"name":"-build", "kind":"namevalue", "type":"filepath=*.prj"}
	]
},
"coder":
{
	"inputs":
	[
		{"name":"-new", "kind":"namevalue", "type":[["filepath=*.prj"], ["char"]]}
	]
},
"coder":
{
	"inputs":
	[
		{"name":"ecoderFlag",  "kind":"required", "type":"choices={'-ecoder'}"},		{"name":"ecoderValue", "kind":"required", "type":[["logical"], ["choices={'false'}"]]},		{"name":"newFlag",     "kind":"required", "type":"choices={'-new'}"},		{"name":"newvalue",    "kind":"required", "type":[["filepath=*.prj"], ["char"]]}	]
},
"coder":
{
	"inputs":
	[
		{"name":"tocodeFlag",  "kind":"required", "type":"choices={'-tocode'}"},
		{"name":"tocodevalue", "kind":"required", "type":"filepath=*.prj"},
		{"mutuallyExclusiveGroup":
			[
				[],
				[
					{"name":"scriptFlag", "kind":"required", "type":"choices={'-script'}"},
					{"name":"scriptname", "kind":"required", "type":[["filepath=*.m"], ["char"]]}
				]
			]
		}
	]
}

This example also shows, in the last definition for the coder function, another mechanism for specifying alternative inputs, using “mutuallyExclusiveGroup” (aka “MEGs”). A MEG is defined using an array of options, enclosed in square brackets ([]). Each of the MEG options is exclusive to each of the others, meaning that we can only work with one of them and not the others. This is equivalent to duplicating the definition as we saw above, and saves us some copy-paste (in some cases a lot of copy-pastes, especially with multiple and/or nested MEGs). However, MEGs have a major drawback of reduced readability. I believe that in most cases we only have a single MEG and few input args, and in such cases it makes more sense to use repeated function defs rather than a MEG. The Matlab signature files contain numerous usage examples for either of these two mechanisms.

Platform dependencies

If a specific function (or a specific signature variant) depends on the running platform, this can be specified via the “platform” directive. For example, the winopen function only works on Windows, but not on Linux/Mac. Its corresponding signature definition is:

"winopen":
{
	"platform":"win32,win64",	"inputs":
	[
		{"name":"filename", "kind":"required", "type":"filepath"},
		{"name":"varargin", "kind":"optional", "multiplicity":"append"}
	]
}

Platform dependence could also be specified at the parameter level, not just the entire function level. For example, in the xlsread function (defined in %matlabroot%/toolbox/matlab/iofun/functionSignatures.json), we see that the usage variant xlsread(filename,-1) is only available on Windows (note that the numeric value is defined as "<=-1", not necessarily -1), and so is the “functionHandle” input (which is called processFcn in the documentation – for some reason that escapes me the names of many input args do not match in the documentation and functionSignature):

"xlsread":
{
	"inputs":
	[
		{"name":"filename", "kind":"required", "type":"filepath=*.xls,*.xlsx,*.xlsb,*.csv"},
		{"mutuallyExclusiveGroup":
			[
				{"name":"openExcel", "kind":"required", "display":"", "type":["numeric", "<=-1"], "platform":"win64,win32"},				{"name":"xlRange",   "kind":"required", "type":["char", "@(x) isempty(x) || ~isempty(strfind(x, ':'))"], "default":"''"},
				[
					{"name":"sheet",          "kind":"positional", "type":[["char", "choices=matlab.internal.language.introspective.tabcompletion.xlsread_vsheet(filename)"], ["numeric", ">=1"]], "default":"1"},
					{"name":"xlRange",        "kind":"positional", "type":"char", "default":"''"},
					{"name":"basic",          "kind":"positional", "display":"", "type":["char", "choices={'basic',''}"]},
					{"name":"functionHandle", "kind":"positional", "type":"function_handle", "platform":"win64,win32"}				]
			]
		}
	],
        ...

Parsing errors

The new mechanism is not very user-friendly when you get something wrong. In the best case, it issues a cryptic error message (see below), and in the worst case it simply ignores the changes and the user has no idea why the new custom tab-completion is not working as intended.

The most likely causes of such problems are:

  • The most common problem is that you placed the functionSignatures.json file in a different folder than the Matlab function. For example, if the myFunction() function is defined in myFunction.m, then the tab-completion of this function MUST be located in a functionSignatures.json file that resides in the same folder, not anywhere else on the Matlab path. In other words, the Matlab path is NOT relevant for tab-completion.
  • Your functionSignatures.json file does not follow the [extremely strict] syntax rules above, to the letter. For example, forgetting the top or final curly braces, forgetting a comma or adding an extra one, or not closing all brackets/braces properly.
  • You mistyped one or more of the input parameters, types or options.

In case of a parsing error, you’d see a red error message on the Matlab console the next time that you try to use tab-completion:

Error parsing JSON data; Boost reports "(189): expected ',' or ']'".

Unfortunately the error message only tells us the problematic line location within the functionSignatures.json file, but not the file’s location, so if we haven’t recently edited this file we’d need to find it in the relevant folder. For example:

edit(fullfile(fileparts(which('myFunction')), 'functionSignatures.json')

Moreover, when a JSON syntax error (such as the one above) occurs, the entire file is not parsed, not just the definition that caused the error.

Another limitation of tab-completion is that it does not work while the main Matlab thread is working (e.g., during a uiwait or waitfor). This may be somewhat misleading since most editor/debugging actions do work.

Arguably, this new tab-completion mechanism could be made more programmer-friendly. Perhaps this will improve in a future Matlab release.

For a related mechanism, see my article on tab-completion for class properties and methods from 2014, which is apparently still relevant and functional.

Categories: Desktop, Medium risk of breaking in future versions, Undocumented feature

Tags: , ,

Bookmark and SharePrint Print

5 Responses to User-defined tab completions – take 2

  1. Heiko says:

    Very nice! I’ve been waiting for a feature like this since we offered our toolbox. Especially the possibility to let MatLab determine the choices is gold. Even though it seems to depend on the type. I could not yet get it to accept other choices than a cell array. Looking forward to somebody taking the time to document a more or less complete list of options for each parameter…

    About JSON comments: doing my first steps in ML R2016b I found that comments are possible. ML accidently opened the json file “R2016b\toolbox\matlab\graph2d\functionSignatures.json” which contained C-like inline comments starting with “//”. I also tried “/* */” comments successfully.

    Thanks for the very useful help!!
    Heiko

    • @Heiko – nice catch about the comments! Apparently functionSignatures.json is not a strictly-compliant JSON file, and undergoes some pre-processing by the Matlab engine.

  2. Greg says:

    Thanks, Yair, this is awesome!

    One added detail that I discovered when implementing this feature this morning is where MATLAB expects the .json file and the appropriate syntax for functions in package folders (i.e. +directories).

    It looks like the appropriate thing is to place the .json file in the root of the package (the folder above the top-level +folder) and then reference the function names with their full package-specific function name. As an example, check out the functionSignatures.json file in %matlabroot/toolbox/shared/io/, which contains signatures for functions like matlab.io.TextVariableImportOptions.

    By contrast, class methods seem to get referenced just by their method names, though class methods in @-folders are still described in the higher-level .json file. For example, in %matlabroot/toolbox/matlab/datatypes/functionSignatures.json, the method “histogram” of the categorical class is referenced simply as “histogram,” even though is a regular (non-static) method stored in the @categorical folder.

  3. Sky Sartorius says:

    This is a post I have been waiting for, Yair. Thank you. I was spending too much time digging through the Matlab functionSignatures.json files to find examples similar to what I was trying to do, so this summary is an excellent resource.

  4. I have been encountering some interesting behavior in 2017b. Signatures that were working in 2017a now do not, though the error message does look like it is trying to be helpful: “Unable to load signatures for 'foo'. 'namevalue' and 'flag' arguments may not appear both inside and outside of groups.” So, it appears that this is a new “rule” for signatures (and violating the rule was giving me the desired behavior before).

    TMW seems to get around this (see signatures for e.g. join) by putting a copy of each ‘outside’ arguments in every member of an existing MEG. Another workaround that seems to work is to put each ‘outside’ argument in its own (lonely) MEG, which has the benefit of avoiding duplicates.

Leave a Reply to Greg Cancel reply

Your email address will not be published. Required fields are marked *