A few days ago, my ScreenCapture utility was selected as Matlab’s Pick of the Week (POTW). POTW selections are normally extremely useful, well-written and instructive utilities, which are both great to use in their own right, as well as a great source of knowledge about Matlab programming features and good practices. I follow the weekly POTW selections closely, and often learn new stuff from these utilities. I take pride in the fact that some of my utilities have been selected for inclusion in this unique set.
ScreenCapture enables Matlab users to take automated (programmatic) as well as interactive screen-captures of any Matlab GUI component or sub-region. This includes figure windows, axes, images, controls and even the Matlab Desktop. If the target handle for the capture is not specified, then ScreenCapture prompts the user to interactively select the capture region using an rbbox limiting box. ScreenCapture also includes a feature that plants a camera icon in the figure toolbar, such that clicking this icon will immediately trigger the interactive region-selection screen-capture.
Whichever manner the capture was made, the user then has the option of sending the output to Matlab (as a 3D RGB image matrix), or to one of the standard image file formats (e.g., JPG or PNG).
ScreenCapture has extensive help and is well-documented and relatively easy to use. For example:
figure; surf(peaks); imgData=screencapture(gcf); imshow(imgData);
Some additional usage examples:
imageData = screencapture; % interactively select screen-capture rectangle imageData = screencapture(hListbox); % capture image of a uicontrol imageData = screencapture(0, [20,30,40,50]); % capture a small desktop sub-region imageData = screencapture(gcf,[20,30,40,50]); % capture a small figure sub-region % capture a small sub-region of an axes imageData = screencapture(gca,[10,20,30,40]); imshow(imageData); % display the captured image in a matlab figure imwrite(imageData,'myImage.png'); % save the captured image to file % capture a sub-region of an image img = imread('cameraman.tif'); hImg = imshow(img); screencapture(hImg,[60,35,140,80]); % in data units, not pixel units screencapture(gcf,,'myFigure.jpg'); % capture the entire figure into file screencapture('handle',gcf,'filename','myFigure.jpg'); % same as previous screencapture('toolbar',gcf); % adds a screen-capture button to gcf's toolbar screencapture('toolbar',,'file','sc.bmp'); % same, using a default output filename
Over the course of the past few years I have submitted 40 utilities to the Matlab File Exchange, several of which have been selected for POTW. Unfortunately, since most of my utilities employ undocumented Matlab features to some extent (I can’t help myself…), they are ineligible for being selected as POTF, useful and deserving as they may be. In fact, my cprintf utility, which was selected as POTW, was quickly deselected as POTW because of this very issue. MathWorks fears (and I can certainly understand the concern) that highlighting a utility that relies on some undocumented feature as POTW might be considered as an official endorsement of these features.
ScreenCapture is different in this regard: it uses purely documented Matlab functionality to achieve its aims, and apparently still succeeds in providing useful functionality. This does not mean that ScreenCapture uses pure Matlab. In fact, it relies on the Java
Robot class‘s functionality of taking a screen-capture of a specified area of the screen. Using the Java
Robot class in such a way is an entirely documented Matlab feature. I have discussed the Java
Robot in two past articles on this blog, where guest blogger Kesh Ikuma explained (here and here) how it can be used to simulate mouse and keyboard actions programmatically.
Under the hood
ScreenCapture calculates the requested screen-capture rectangle coordinates and then invokes the Java
Robot to take the actual bitmap screen-capture. The output is then converted into a Matlab image matrix for output, or stored in an image file, based on the user’s choice of parameters.
Some difficulties that I overcame when programming ScreenCapture:
- The screen position of docked windows cannot be computed reliably. Docked windows therefore need to be automatically temporarily undocked for screen-capture.
- Undocking in Windows 7 with Aero transparency features causes the
Robotto take its screen-shot before the window becomes fully opaque. Adding a short delay in undocking solved this issue.
- Images use reversed Y-axis (Y=0 is at the axes top, not bottom). Also, when specifying a sub-region for capture, many users are used to handling images using data units (e.g., for imcrop) rather than ScreenCapture’s standard pixel units. Taking screen-captures of images proved to be a non-trivial challenge indeed.
- Different Matlab objects (controls, axes, figures) have different external borders and internal margins. I had to take these into account in order to achieve tight-fitting image captures of these objects.
- Performance was a problem, and it turned out that the bottleneck was trying to convert from the Java image data format to Matlab’s image data format. A couple of suggestions by Jan Simon and Urs (us) Schwartz significantly improved this performance hotspot. I’ve submitted the relevant code snippets to MathWorks for incorporation in a published technical solution, and I was happy to see that they have indeed incorporated them into that solution.
- Finally, I thought that adding a custom toolbar image would be a nice touch. Since I’m not much of an artist, creating the camera icon programmatically proved to be a bit of a challenge…
Please feel free to download ScreenCapture’s code and check how I chose to program around these issues.
Some potentially-useful features have so far eluded me in ScreenCapture’s implementation. Perhaps one day I will find a way to do them:
- Enable output of the image data, as an image object, to the system clipboard. It is easy to serialize the data and store it as a string in the clipboard, using the built-in clipboard function. But we would not be able to paste this data as an image into an external editor or image-processing utility. I have not yet found an easy way to store the image data as an object, although it should not be very difficult to do (here’s a starter).
- When interactively selecting a screen-capture region, rbbox‘s starting point needs to be somewhere within the boundaries of a Matlab figure. The box can extend beyond the figure’s borders, but it has to start somewhere within the figure. I would like to be able to use rbbox without this limitation.
Addendum Jan 28. 2013: A new version of ScreenCapture was uploaded to the File Exchange today which appears to solve both of the TODO issues above: The copy-to-clipboard feature relies on Jiro Doke’s imclipboard utility as mentioned by Matt below — simply specify the ‘clipboard’ string as the capture target (rather than a filename); the solution of the rbbox-anywhere feature relies on using a temporary transparent window that spans the entire desktop area, capturing the user’s rbbox clicks anywhere within the desktop area. Interested readers can easily adapt the code to fit multiple monitors (I didn’t bother).
- An interesting uitree utility ExploreStruct is a utility that shows how custom uitrees can be integrated in Matlab GUI...
- GUI automation using a Robot This article explains how Java's Robot class can be used to programmatically control mouse and keyboard actions...
- GUI automation utilities This article explains a couple of Matlab utilities that use Java's Robot class to programmatically control mouse and keyboard actions...
- Splash window for deployed applications Deployed (compiled) Matlab applications take a long time to load. I present a splash window that loads immadiately, solving this problem. ...
- Figure toolbar components Matlab's toolbars can be customized using a combination of undocumented Matlab and Java hacks. This article describes how to access existing toolbar icons and how to add non-button toolbar components....
- uitree This article describes the undocumented Matlab uitree function, which displays data in a GUI tree component...