I would like to introduce guest blogger Mark Mikofski, of SunPower Corp. Mark has developed the JGit4MATLAB utility, which can be used to integrate Matlab with the popular JGit open-source version-control system.
Introduction
JGit is an open-source Java implementation of Git, a popular distributed version control system. Since Matlab is essentially a Java interpreter, JGit is an obvious candidate for integrating Git into Matlab. Luckily, JGit also comes with a porcelain class that has the most popular commands conveniently packaged. This is the starting point for JGit4MATLAB – a thin Matlab wrapper utility on the Matlab File Exchange.
The first step is to download and install JGit4MATLAB in your Matlab path. The first time that the JGit Matlab class is called, it downloads the latest version of org.eclipse.jgit.jar, adds it to Matlab’s static Java classpath and makes a backup of the existing javaclasspath.txt file as javaclasspath.JGitSaved. Matlab must then be restarted for the changes to the static Java class path to take effect. From now on, you can use the commands in the JGit Matlab class or the jgit.m wrapper function. These use basic JGit commands, which shall be described below:
Most of the important JGit methods are encapsulated in the Matlab JGit wrapper class and jgit.m function, but you can also use them directly if you wish. The Matlab wrapper currently supports ADD, BRANCH, CHECKOUT, CLONE, COMMIT, DIFF, INIT, LOG, MERGE & STATUS, but there are many more in org.eclipse.jgit.api.Git
that you can use directly. In fact you don’t really need to use the wrapper at all if you don’t want to – all you really need is the JAR file, which the wrapper downloads and installs for you. You can also download the JAR file independently from here (latest only, which is what JGit4MATLAB does), or here (latest and previous releases). If you’re using JGit4MATLAB, you can update the JAR file to the latest version at will, by issuing JGit.downloadJGitJar('@JGit\org.eclipse.jgit.jar')
.
The most basic Git commands are clone, init, add and commit. Let us explore how to use JGit’s “Git-Porcelain API” class in Matlab to perform these tasks.
Clone and Init
Clone and Init are both class methods of the Git-Porcelain API class and return a Command
API class object, which is what nearly all of the Git-Porcelain methods do:
import org.eclipse.jgit.api.Git % import the package cloneCMD = Git.cloneRepository; % return a CloneCommand object URI = 'git://github.com/mikofski/JGit4Matlab.git'; % a Git repository cloneCMD.setURI(URI) % set the repository to clone g = cloneCMD.call; % clone the repository |
or more compactly:
g = org.eclipse.jgit.api.Git.cloneRepository.setURI(URI).call; |
The call method executes the command with the options and arguments set by the other methods. Calling the clone method also returns a Git instance. The full API for the CloneCommand
class lists all the possible options and arguments for cloning. One option lets you set an optional progress monitor; there are some already included in JGit, but unfortunately, they use the carriage return (or ‘\r’) escape sequence which in Windows is the same as a newline (read here). So, I implemented a customized ProgressMonitor
class for Matlab that uses backspace (or ‘\b’) instead.
Init is very similar to clone:
g = org.eclipse.jgit.api.Git.init.call; |
Configuring author information
Before moving on, it is important to set the “author” and “email” fields in the Git config. JGit will use your computer’s info if you provide nothing. You probably want to set these values globally, so the place to put them is in your $HOME folder in the “.gitconfig” file.
file = java.io.File(fullfile(getenv('home'),'.gitconfig')); gitconfig = org.eclipse.jgit.storage.file.FileBasedConfig(file, org.eclipse.jgit.util.FS.DETECTED); gitconfig.setString('user',[], 'name','John Doe'); % place your name in here gitconfig.setString('user',[], 'email','john@doe.com'); % place your email in here gitconfig.save; |
Hopefully your commits will all have the right name and email address in them from now on.
Typical JGit Workflow
Now that you have a Git repository, you will want to start working with it. The typical workflow is to create a feature branch in which to work. But first JGit needs a Git instance. If you saved it from cloning or initializing a new repo then good for you! But if you need to recreate it, just pass the constructor a File
object that points to the .git folder inside your Git repo:
file = java.io.File(fullfile('path', 'to', 'repo', '.git')); g = org.eclipse.jgit.api.Git.open(file); g.checkout.setCreateBranch(true).setName('newbranch').call; |
Now you are no longer on the “master” branch, which is usually reserved for buildable/deployable code. Start working. But before you do too much, commit your work to your branch:
g.add.addFilepattern('file1.m').addFilepattern('file2.m').call; g.commit.setMessage('initial dump').call; |
All commits should have a message, so you know what work was completed in that commit. Branch and commit frequently. You can always clean them up later. There are lots of options for adding and committing files, so look at the API’s AddCommand
and CommitCommand
classes.
Everything else is pretty much the same as above: Get a Git instance; create a command; set the options; and finally call the command:
file = java.io.File(fullfile('path', 'to', 'repo', '.git')); g = org.eclipse.jgit.api.Git.open(file); r = g.getRepository; % a Repository instance can be very useful g.status.call; g.log.call; g.checkout.setName('master').call; g.merge.include(r.resolve('newbranch')).call; g.push.call; g.pull.call; |
Note: Repository.resolve()
is one of the most powerful methods.
The JGit log window on the left can be gotten using glog – the standalone JGit command line log GUI (slightly similar to gitk) that can be downloaded as a binary for systems with /bin/sh from the eclipse site (same place as the jar download).
Some tricks can be gleaned by what little work I’ve done so far on the JGit4MATLAB wrapper on the Matlab File Exchange. I’ll be happy to answer whatever questions I can there, or comments here. I’m also happy to accept pull requests on Github, of course using JGit4MATLAB!
Authentication
You can only use an SSH paired keys to authenticate remote repositories, the keys must **not** have a pass-phrase, they must be in the openSSH format and they must be in $HOME/.ssh. PuTTYgen let’s you do all of this, so it’s really not an issue. Of course you can still use remotes that do not require authentication with no problems.
All commands that deal with remotes and that require authentication, e.g. CLONE, FETCH, PULL, PUSH, will only work
- with SSH
- with *no* passphrase
- with keys in the openSSH format
- with keys and known hosts in $HOME\.ssh
Obviously remotes that do not require authentication will work fine. e.g. public, read-only and local repositories.
SSH is very easy to set up:
- Download puttygen (Intel x86). Disregard the “Intel x86” business, it doesn’t matter what processor you have or whether your os is 64-bit or 32-bit. Puttygen is a very mature well establish application that is used by many other applications, for example all of the TortoiseXXX scm clients use Putty, Plink and Pageant. Here is the actual download here (click to download then click to install): http://the.earth.li/~sgtatham/putty/latest/x86/puttygen.exe
- Create a folder called .ssh in your %– USERPROFILE%. On Windows 7 this is C:\Users\
and on XP it is C:\Documents and Settings\ . You can do this in Matlab: mkdir(fullfile(getenv('userprofile'), '.ssh'))
. - Create a key using puttygen, this is fairly self explanatory, and kind of fun, but do *not* set a pass-phrase; leave those fields blank!
- Using puttygen, find the conversion menu tab and export the key in the openSSH format to the .ssh folder that you created earlier.
- Now copy and paste your public key to your remote repositories (Github, Bitbucket, etc.) as needed. Jsch, which is the ssh client that JGit uses, should now work out of the box.
Unfortunately using SSL (HTTPS) isn’t possible with JGit4MATLAB. Sorry. but hopefully you’ll love the excitement of using ssh key pairs!
Further development
Possible future work on the Matlab-JGit integration includes assigning standard keyboard-shortcuts to common source-control tasks, integration as toolbar/toolstrip icons for easier access, and in the Matlab Editor in general. Only a few days ago, a reader on this blog has shown interest in this.
In the far future, integration with Matlab’s existing source-control system. Anyone wish to pick up the glove?
JGit is covered by the BSD 3-clause license which is very permissive, and should allow MathWorks or anyone for that matter to develop an integrated JGit application for Matlab. There are more packages in JGit than just the porcelain API, including Swing utilities and other utilities that can be used to create rich user interfaces and experiences, but the development would have to be in Java. Developers should look at the developer’s section of the JGit website and use either the Maven or Github repository to set up their forks.
Conclusion
JGit and Matlab are a natural combination. JGit is a mature and thorough open-source project. Most porcelain commands are available without any additional work. There are some arcanae, but the API documentation is sufficient to figure out most things using short high-level code. If you find yourself diving too deep, then scan the docs some more, and you will probably find a shortcut.
@Mark
Thanks for this article. I have been using Git for a while now to support the Waterloo code I blogged about over the past couple of weeks. The Java code is developed in NetBeans which has full Git support – it would be good to see that also in MATLAB.
Being far from expert in Git, I find the GitHub app GUIs useful (it is not specific to the GitHub site: I use SourceForge). It shows me the Git setup, supports SSH and it’s easy to exclude specific files/folders from the public repo. As all changes are made in the Git repo, they are honoured also in NetBeans/other IDEs/from the command line and I am guessing with JGit/JGit4MATLAB too (?).
MATLAB makes use of NetBeans RCP components such as the editor. It would be good to see the MATLAB file manager provide the sort of version control support that NetBeans provides. That could encourage fuller open-source collaboration amongst the MATLAB community too: a problem with the MATLAB FEX is that it does not presently support that.
@Malcolm,
Thanks for your comment.
In response to … “…and I am guessing with JGit/JGit4MATLAB too (?)”, yes, JGit/JGit4MATLAB works with with all Git repositories, both local and remote (i.e.: Github, Bitbucket), regardless of what Git client they were generated with or maintained by, and it inter-operates with the original (C/Perl/shell) Git and other Git-bindings as well (such as Python’s Dulwich and Java’s JGit).
That’s cool that NetBeans also has Git integration. I use both Eclipse and Spyder a lot, and they both have full SCM integration, e.g.: Git, SVN, Hg, etc. Maybe this will put some pressure on TMW to finally introduce SCM integration into their IDE, recognizing it’s importance in best coding practices.
In response to your comment about open-source collaboration and FEX, I made a similar comment on CSSM here.
Frankly, IMHO Github’s window’s app is not the best Git client, I prefer to use TortoiseGit on windows, it has a very intuitive and rich feature set that will be very familiar to TortoiseSVN users. Another client I think is better than Github’s is SourceTree by Atlassian, maintainers of BitBucket, another online Git/Hg repository host.
BTW: I think the Waterloo graphics are awesome! Can’t wait to try them out.
Just wanted to post an update – I have made some major improvements to JGit4MATLAB. Now it can use SSH passphrases (or not), and can pull, push and fetch. I put utilities in to change or view global config and to save your passphrase. Still more to come! Also check out new website: http://mikofski.github.io/JGit4MATLAB/.
The latest update is version 0.6 (Falcon):
It allows (subset of) Git syntax to be used.
Bugs fixes.
`javaclasspath.txt` is now in prefdir instead of userpath (thanks Volker).
Merge conflicts also output *.BASE, *.LOCAL, *.REMOTE and *.orig for use in merge tools such as meld and p4m.
Documentation is now at http://mikofski.github.io/JGit4MATLAB/.
Please submit bugs to https://github.com/mikofski/JGit4MATLAB.
Enjoy!
–Mark
[…] would like to once again welcome guest blogger Mark Mikofski. Mark has written here last year about JGIT-Matlab integration (p.s., in the recently-released R2014a, MathWorks added GIT support to Simulink projects, although […]
[…] Once again I welcome guest blogger Mark Mikofski. Mark has written here last year about JGIT-Matlab integration and earlier this year on JSON-Matlab integration. Today, Mark shows how to use the well-known memcached library […]