JSON-Matlab integration

I would like to once again welcome guest blogger Mark Mikofski. Mark has written here last year about JGIT-Matlab integration (p.s., in the recently-released R2014a, MathWorks added GIT support to Simulink projects, although for some reason not to Matlab projects). Today, Mark discusses how to integrate JSON with Matlab.

What is JSON

It’s often necessary to save objects and arrays to a file, for lots of reasons. In Matlab, you would just save the desired objects in a mat-file, but that’s a binary format that in general only Matlab can read. One reason you might want to cache objects is to pass to a non-Matlab API. True there are libraries to import mat-files (for example: JMatIO), but there are already accepted standards for passing objects such as XML and JSON (JavaScript Object Notation, http://json.org) that most APIs readily understand. Both of these methods attempt to serialize data into a text based format that limits the types of objects they can contain. Because sometimes the API is a human being who needs to read and edit the file, one of the JSON’s goals is to be “easy for humans to read and write”.

Here’s a sample of a JSON object:

{
    "students": ["Dick", "Jane"],
    "assignments": ["essay", "term paper"]
    "scores": {
        "essay": {"Dick": 86, "Jane": 88},
        "term paper":  {"Dick": 89, "Jane": 87}
    },
    "cool": {"Dick": true, "Jane": true},
    "misc": null
}

Many web services, such as Twitter, use JSON in their APIs, because another JSON goal is to be “easy for machines to parse and generate”. JSON is based on the JavaScript ECMA standard in which it is native, and is extremely well documented.

JSON Primer

The documentation on json.org’s main page includes several train-track style graphics that best explain JSON usage, but here is a short primer:

• There are two main types of JSON objects, a JSONObject and a JSONArray.
• A JSONObject is enclosed in braces and consists of a collection of unordered key-value pairs separated by a commas. Each key-value pair is separated by a colon, with the key given before the value. In the example above, “students” is a key and its value is the JSONArray [“Dick”, “Jane”]. Keys must be strings, as explained below.
• A JSONArray is enclosed in brackets and is ordered array of valid JSON types.
• Items in a JSONObject or JSONArray may be one of 7 types: string, number, JSONObject, JSONArray, true, false, or null.
• Strings are always enclosed in double-quotes and may contain backslashed escapes such as newline (\n), tab (\t), double-quotes (\”), backslash (\\) etc.
• Numbers can be integers, floats, and scientific notation.
• JSON interprets true and false as booleans (or logicals), and null as nothing.

There are several options to use JSON in Matlab, depending on what JSON parsing interface you wish to use. Matlab can use external libraries such as Java, .NET and C that allow us to use different implementations of the standard, including the original Java-based implementation by Douglas Crockford himself, the primary author of JavaScript of which JSON is a subset. A full list of JSON implementations is given on the main website along with its documentation. Here is a short list of some libraries:

• JSONlab (Matlab) (also on SourceForge)
• org.json (Java)
• Json.NET (.NET)
• fastJSON (.NET)

JSONlab

The most popular and mature Matlab implementation is JSONlab, which was started in 2011 by a Qianqian Fang, a medical researcher at Massachusetts General Hospital. It is available on the Matlab File Exchange, on SourceForge and via a Subversion repository. The latest version is 1.0beta that includes BSON, a variant of JSON used in MongoDB, which compresses JSON data into a binary format. JSONlab is based in part on earlier JSON-Matlab implementations that are now deprecated: JSON Parser (2009) by Joel Feenstra, another JSON Parser (2011) by François Glineur, and Highly portable JSON-input parser (2009) by Nedialko.

JSONlab converts both strings and files given by filename directly into Matlab structures and vice-versa. For example:

>> loadjson('{"this": "that", "foo": [1,2,3], "bar": ["a", "b", "c"]}')
ans = 
    this: 'that'
    foo: [1 2 3]
    bar: {'a'  'b'  'c'}
 
>> s = struct('this', {'a', 'b'}, 'that', {1,2})
s = 
1x2 struct array with fields:
    this
    that
 
>> j = savejson(s)
j =
{
    "s": [
        {
            "this": "a",
            "that": 1
        },
        {
            "this": "b",
            "that": 2
        }
    ]
}

JSONlab will nest structures as necessary and translates the JSON types into the appropriate Matlab types and vice-versa. JSONlab is well documented, easy and fast. It is reliable because it is well-maintained, has been around for several years and has many users. It is open source and issues and contributions are welcomed.

org.json (aka JSON-Java)

This is the Java implementation of JSON by the creator JSON and JavaScript, Douglas Crockford. This version is very well documented and under active development, but since there are no releases and only one branch, it could be considered the development branch. There are several other popular Java implementations listed on the JSON.org website.

The JSON library itself is called json.jar or org.json.jar. The easiest way to use it in Matlab is to add it to your java class path and then use it as if you were calling Java code. I put together a gist of examples and links for your reference. In particular take a look at orgJSONdemo.m which has several examples of creating JSON objects and then using them. I addition, take a look at this CSSM newsgroup thread.

How to add the jar file to your java class path depends on your version of Matlab: starting in R2012b, Matlab switched from using classpath.txt in your Matlab workspace to using javaclasspath.txt, which is much easier (however, see here). Make a file in your Matlab workspace, usually /home/MATLAB or /home/documents/matlab (e.g. on windows 7: c:\users\you\documents\matlab), called “javaclasspath.txt” and put the full path to your jar file. For example if you put the jar file in a MATLAB/JSON/ folder, then put the following line (on windows 7) in javaclasspath.txt:

the path specified in the javaclasspath.txt file should be the path to the actual jar file, so assuming your jar is called json.jar then you would put this path in your file:

C:\Users\your-user-name\Documents\MATLAB\JSON\json.jar

You must restart Matlab before this change takes effect. An alternate method is to load the jar file dynamically to Matlab’s Java classpath, using javaaddpath(‘path\to\json.jar’).

All this is pretty straight-forward in Matlab R2013b and newer, which use Java 7. json.org was implemented in Java 7 as javax.json, and so can be used with Matlab R2013b and newer. Matlab versions R2013a and earlier use Java 6, so another Java implementation must be used. You can either download a pre-compiled JAR file from my dropbox, or compile the source files on your system as follows:

1. Confirm the version of Java used in Matlab by typing

   >> version -java
   Java 1.6.0_17-b04 with Sun Microsystems Inc. Java HotSpot(TM) 64-Bit Server VM mixed mode

2. Download and install the corresponding Java JDK from Oracle or OpenJDK
3. Obtain the source by cloning the repository or downloading a tar/zip-ball
4. If necessary, extract the source then browse to the “src” folder
5. Compile the source into a class files from the command line:

   C:\> C:\Program Files\Java\jdk1.7.0_45\bin\javac -verbose -sourcepath org\json\ -source 1.6 -target 1.6 -d ..\bin\ org\json\*.java org\json\zip\*.java

6. Browse the created “bin” folder
7. Archive the class files into a jar file:

   C:\> C:\Program Files\Java\jdk1.7.0_45\bin\jar -cvf JSON.jar org\

8. Add the new jar file to the Matlab static Java classpath, which will depend on what version of Matlab you have:
   1. Matlab <= 2012a:
      • Find the system classpath.txt file by typing:

        >> [matlabroot '\toolbox\local\classpath.txt']
        C:\Program Files\MATLAB\R2012a\toolbox\local\classpath.txt

      • Copy the classpath to your working folder and append the path the jar file to the end. Use hash marks to comment lines.
         
   2. Matlab >= 2012b: create a new file called “javaclasspath.txt” in your working folder with the path of the new jar file in it. Use hash marks for comments.
       
9. Restart Matlab to enable the jar file within Matlab.

Using the Java library is the same as using any Java library in Matlab. One nice feature is that tabbing after typing “object.” gives a pop-up list of available properties and methods, including inherited ones. Here are some simple usage examples:

>> j1 = org.json.JSONObject('{"this": "that", "foo": [1,2,3], "bar": ["a", "b", "c"]}')
>> j2 = j1.put('hello', 'world')
j2 =
{"hello":"world", "foo":[1,2,3], "bar":["a","b","c"], "this":"that"}
 
>> w = j1.getString('hello')
ans =
world

Watch out: the output is actually a java.lang.String object; use char() to convert it into a Matlab string:

>> char(w)
ans =
world

There is no easy way to convert a JSONArray to an array, so you can either loop over it, or convert it to a string and parse it. Note that JSONArray indexes start from 0, as in Java (contrary to Matlab indexes, which start at 1)

>> f = j1.get('foo')
f =
[1,2,3]
 
>> for n = 1:f.length,  fa(n)=f.get(n-1);  end
>> fa
fa =
     1     2     3

Json.NET and fastJSON

These libraries require you to add add the path of the .NET dll to Matlab using NET.addAssembly. For fastJSON, since binaries are not available, you will also need to compile the .NET dll. I’ll leave it to the reader to investigate further, independently.