A few days ago, my ScreenCapture utility was selected as Matlab’s Pick of the Week (POTW). POTW selections are normally extremely useful, well-written and instructive utilities, which are both great to use in their own right, as well as a great source of knowledge about Matlab programming features and good practices. I follow the weekly POTW selections closely, and often learn new stuff from these utilities. I take pride in the fact that some of my utilities have been selected for inclusion in this unique set.
ScreenCapture enables Matlab users to take automated (programmatic) as well as interactive screen-captures of any Matlab GUI component or sub-region. This includes figure windows, axes, images, controls and even the Matlab Desktop. If the target handle for the capture is not specified, then ScreenCapture prompts the user to interactively select the capture region using an rbbox limiting box. ScreenCapture also includes a feature that plants a camera icon in the figure toolbar, such that clicking this icon will immediately trigger the interactive region-selection screen-capture.
Whichever manner the capture was made, the user then has the option of sending the output to Matlab (as a 3D RGB image matrix), or to one of the standard image file formats (e.g., JPG or PNG).
ScreenCapture has extensive help and is well-documented and relatively easy to use. For example:
figure; surf(peaks); imgData=screencapture(gcf); imshow(imgData);
Some additional usage examples:
imageData = screencapture; % interactively select screen-capture rectangle imageData = screencapture(hListbox); % capture image of a uicontrol imageData = screencapture(0, [20,30,40,50]); % capture a small desktop sub-region imageData = screencapture(gcf,[20,30,40,50]); % capture a small figure sub-region % capture a small sub-region of an axes imageData = screencapture(gca,[10,20,30,40]); imshow(imageData); % display the captured image in a matlab figure imwrite(imageData,'myImage.png'); % save the captured image to file % capture a sub-region of an image img = imread('cameraman.tif'); hImg = imshow(img); screencapture(hImg,[60,35,140,80]); % in data units, not pixel units screencapture(gcf,,'myFigure.jpg'); % capture the entire figure into file screencapture('handle',gcf,'filename','myFigure.jpg'); % same as previous screencapture('toolbar',gcf); % adds a screen-capture button to gcf's toolbar screencapture('toolbar',,'file','sc.bmp'); % same, using a default output filename
Over the course of the past few years I have submitted 40 utilities to the Matlab File Exchange, several of which have been selected for POTW. Unfortunately, since most of my utilities employ undocumented Matlab features to some extent (I can’t help myself…), they are ineligible for being selected as POTF, useful and deserving as they may be. In fact, my cprintf utility, which was selected as POTW, was quickly deselected as POTW because of this very issue. MathWorks fears (and I can certainly understand the concern) that highlighting a utility that relies on some undocumented feature as POTW might be considered as an official endorsement of these features.
ScreenCapture is different in this regard: it uses purely documented Matlab functionality to achieve its aims, and apparently still succeeds in providing useful functionality. This does not mean that ScreenCapture uses pure Matlab. In fact, it relies on the Java
Robot class‘s functionality of taking a screen-capture of a specified area of the screen. Using the Java
Robot class in such a way is an entirely documented Matlab feature. I have discussed the Java
Robot in two past articles on this blog, where guest blogger Kesh Ikuma explained (here and here) how it can be used to simulate mouse and keyboard actions programmatically.
Under the hood
ScreenCapture calculates the requested screen-capture rectangle coordinates and then invokes the Java
Robot to take the actual bitmap screen-capture. The output is then converted into a Matlab image matrix for output, or stored in an image file, based on the user’s choice of parameters.
Some difficulties that I overcame when programming ScreenCapture:
- The screen position of docked windows cannot be computed reliably. Docked windows therefore need to be automatically temporarily undocked for screen-capture.
- Undocking in Windows 7 with Aero transparency features causes the
Robotto take its screen-shot before the window becomes fully opaque. Adding a short delay in undocking solved this issue.
- Images use reversed Y-axis (Y=0 is at the axes top, not bottom). Also, when specifying a sub-region for capture, many users are used to handling images using data units (e.g., for imcrop) rather than ScreenCapture’s standard pixel units. Taking screen-captures of images proved to be a non-trivial challenge indeed.
- Different Matlab objects (controls, axes, figures) have different external borders and internal margins. I had to take these into account in order to achieve tight-fitting image captures of these objects.
- Performance was a problem, and it turned out that the bottleneck was trying to convert from the Java image data format to Matlab’s image data format. A couple of suggestions by Jan Simon and Urs (us) Schwartz significantly improved this performance hotspot. I’ve submitted the relevant code snippets to MathWorks for incorporation in a published technical solution, and I was happy to see that they have indeed incorporated them into that solution.
- Finally, I thought that adding a custom toolbar image would be a nice touch. Since I’m not much of an artist, creating the camera icon programmatically proved to be a bit of a challenge…
Please feel free to download ScreenCapture’s code and check how I chose to program around these issues.
Some potentially-useful features have so far eluded me in ScreenCapture’s implementation. Perhaps one day I will find a way to do them:
- Enable output of the image data, as an image object, to the system clipboard. It is easy to serialize the data and store it as a string in the clipboard, using the built-in clipboard function. But we would not be able to paste this data as an image into an external editor or image-processing utility. I have not yet found an easy way to store the image data as an object, although it should not be very difficult to do (here’s a starter).
- When interactively selecting a screen-capture region, rbbox‘s starting point needs to be somewhere within the boundaries of a Matlab figure. The box can extend beyond the figure’s borders, but it has to start somewhere within the figure. I would like to be able to use rbbox without this limitation.
Addendum Jan 28. 2013: A new version of ScreenCapture was uploaded to the File Exchange today which appears to solve both of the TODO issues above: The copy-to-clipboard feature relies on Jiro Doke’s imclipboard utility as mentioned by Matt below — simply specify the ‘clipboard’ string as the capture target (rather than a filename); the solution of the rbbox-anywhere feature relies on using a temporary transparent window that spans the entire desktop area, capturing the user’s rbbox clicks anywhere within the desktop area. Interested readers can easily adapt the code to fit multiple monitors (I didn’t bother).
it would be nice if tooltip, active menu/submenu, ect can be captured.