ScreenCapture utility

August 15, 2012 35 Comments

A few days ago, my ScreenCapture utility was selected as Matlab’s Pick of the Week (POTW). POTW selections are normally extremely useful, well-written and instructive utilities, which are both great to use in their own right, as well as a great source of knowledge about Matlab programming features and good practices. I follow the weekly POTW selections closely, and often learn new stuff from these utilities. I take pride in the fact that some of my utilities have been selected for inclusion in this unique set.
ScreenCapture enables Matlab users to take automated (programmatic) as well as interactive screen-captures of any Matlab GUI component or sub-region. This includes figure windows, axes, images, controls and even the Matlab Desktop. If the target handle for the capture is not specified, then ScreenCapture prompts the user to interactively select the capture region using an rbbox limiting box. ScreenCapture also includes a feature that plants a camera icon in the figure toolbar, such that clicking this icon will immediately trigger the interactive region-selection screen-capture.
Whichever manner the capture was made, the user then has the option of sending the output to Matlab (as a 3D RGB image matrix), or to one of the standard image file formats (e.g., JPG or PNG).
ScreenCapture has extensive help and is well-documented and relatively easy to use. For example:

figure; surf(peaks); imgData=screencapture(gcf); imshow(imgData);

Some additional usage examples:

imageData = screencapture;                     % interactively select screen-capture rectangle
imageData = screencapture(hListbox);           % capture image of a uicontrol
imageData = screencapture(0,  [20,30,40,50]);  % capture a small desktop sub-region
imageData = screencapture(gcf,[20,30,40,50]);  % capture a small figure sub-region
% capture a small sub-region of an axes
imageData = screencapture(gca,[10,20,30,40]);
imshow(imageData);  % display the captured image in a matlab figure
imwrite(imageData,'myImage.png');  % save the captured image to file
% capture a sub-region of an image
img = imread('cameraman.tif');
hImg = imshow(img);
screencapture(hImg,[60,35,140,80]);  % in data units, not pixel units
screencapture(gcf,[],'myFigure.jpg');                   % capture the entire figure into file
screencapture('handle',gcf,'filename','myFigure.jpg');  % same as previous
screencapture('toolbar',gcf);                           % adds a screen-capture button to gcf's toolbar
screencapture('toolbar',[],'file','sc.bmp');            % same, using a default output filename

Purely documented

Over the course of the past few years I have submitted 40 utilities to the Matlab File Exchange, several of which have been selected for POTW. Unfortunately, since most of my utilities employ undocumented Matlab features to some extent (I can’t help myself…), they are ineligible for being selected as POTF, useful and deserving as they may be. In fact, my cprintf utility, which was selected as POTW, was quickly deselected as POTW because of this very issue. MathWorks fears (and I can certainly understand the concern) that highlighting a utility that relies on some undocumented feature as POTW might be considered as an official endorsement of these features.
ScreenCapture is different in this regard: it uses purely documented Matlab functionality to achieve its aims, and apparently still succeeds in providing useful functionality. This does not mean that ScreenCapture uses pure Matlab. In fact, it relies on the Java Robot class‘s functionality of taking a screen-capture of a specified area of the screen. Using the Java Robot class in such a way is an entirely documented Matlab feature. I have discussed the Java Robot in two past articles on this blog, where guest blogger Kesh Ikuma explained (here and here) how it can be used to simulate mouse and keyboard actions programmatically.

Under the hood

ScreenCapture calculates the requested screen-capture rectangle coordinates and then invokes the Java Robot to take the actual bitmap screen-capture. The output is then converted into a Matlab image matrix for output, or stored in an image file, based on the user’s choice of parameters.
Some difficulties that I overcame when programming ScreenCapture:

The screen position of docked windows cannot be computed reliably. Docked windows therefore need to be automatically temporarily undocked for screen-capture.
Undocking in Windows 7 with Aero transparency features causes the Robot to take its screen-shot before the window becomes fully opaque. Adding a short delay in undocking solved this issue.
Images use reversed Y-axis (Y=0 is at the axes top, not bottom). Also, when specifying a sub-region for capture, many users are used to handling images using data units (e.g., for imcrop) rather than ScreenCapture’s standard pixel units. Taking screen-captures of images proved to be a non-trivial challenge indeed.
Different Matlab objects (controls, axes, figures) have different external borders and internal margins. I had to take these into account in order to achieve tight-fitting image captures of these objects.
Performance was a problem, and it turned out that the bottleneck was trying to convert from the Java image data format to Matlab’s image data format. A couple of suggestions by Jan Simon and Urs (us) Schwartz significantly improved this performance hotspot. I’ve submitted the relevant code snippets to MathWorks for incorporation in a published technical solution, and I was happy to see that they have indeed incorporated them into that solution.
Finally, I thought that adding a custom toolbar image would be a nice touch. Since I’m not much of an artist, creating the camera icon programmatically proved to be a bit of a challenge…

Please feel free to download ScreenCapture’s code and check how I chose to program around these issues.

TODO list

Some potentially-useful features have so far eluded me in ScreenCapture’s implementation. Perhaps one day I will find a way to do them:

Enable output of the image data, as an image object, to the system clipboard. It is easy to serialize the data and store it as a string in the clipboard, using the built-in clipboard function. But we would not be able to paste this data as an image into an external editor or image-processing utility. I have not yet found an easy way to store the image data as an object, although it should not be very difficult to do (here’s a starter).
When interactively selecting a screen-capture region, rbbox‘s starting point needs to be somewhere within the boundaries of a Matlab figure. The box can extend beyond the figure’s borders, but it has to start somewhere within the figure. I would like to be able to use rbbox without this limitation.

Addendum Jan 28. 2013: A new version of ScreenCapture was uploaded to the File Exchange today which appears to solve both of the TODO issues above: The copy-to-clipboard feature relies on Jiro Doke’s imclipboard utility as mentioned by Matt below — simply specify the ‘clipboard’ string as the capture target (rather than a filename); the solution of the rbbox-anywhere feature relies on using a temporary transparent window that spans the entire desktop area, capturing the user’s rbbox clicks anywhere within the desktop area. Interested readers can easily adapt the code to fit multiple monitors (I didn’t bother).

35 Responses

wei August 17, 2012 at 08:24 Reply

it would be nice if tooltip, active menu/submenu, ect can be captured.

Yair Altman August 20, 2012 at 07:50 Reply

@wei – capturing tooltips and open menu items is of course impossible in interactive mode, but I’m pretty sure that it works ok in non-interactive (programmatic mode). Meaning, if a timer callback runs ScreenCapture when a menu-item is open or a tooltip is displayed, then I believe that it will indeed be captured.

Matt August 28, 2012 at 14:18 Reply

Yair,

In regards to your ‘ToDo #1’ Jiro Doke from Mathworks seems to have a solution
http://www.mathworks.com/matlabcentral/fileexchange/28708-imclipboard/content/imclipboard.m

Here’s the ‘paste’ code – the real transform seems to occur in when the ImageSelection object is created.

% Import necessary Java classes
import java.awt.Toolkit.*
import java.awt.image.BufferedImage
import java.awt.datatransfer.DataFlavor
 
% Get System Clipboard object (java.awt.Toolkit)
cb = getDefaultToolkit.getSystemClipboard();
 
% Get image size
ht = size(data, 1); 
wd = size(data, 2);
 
% Convert to Blue-Green-Red format
data = data(:, :, [3 2 1]);
 
% Convert to 3xWxH format
data = permute(data, [3, 2, 1]);
 
% Append Alpha data (not used)
data = cat(1, data, 255*ones(1, wd, ht, 'uint8'));
 
% Create image buffer
imBuffer = BufferedImage(wd, ht, BufferedImage.TYPE_INT_RGB);
imBuffer.setRGB(0, 0, wd, ht, typecast(data(:), 'int32'), 0, wd);
 
% Create ImageSelection object
%    % custom java class
imSelection = ImageSelection(imBuffer);
 
% Set clipboard content to the image
cb.setContents(imSelection, []);

Thanks so much for all your amazing matlab solutions.

Yair Altman August 28, 2012 at 16:23 Reply

@Matt – thanks, I missed that one 🙂
Matt August 29, 2012 at 17:16 Reply

Yair – On the topic of matlab clipboard interfaces- is there an easy way in these situations to check if the instance of matlab running the paste code has lost ownership of the system clipboard?

The ClipboardOwner interface has the method lostOwnership, but I’ve had difficulties figuring out how to use that method within the Matlab event handler. It would be sufficient to even be able to check the owner without a callback.
I currently use the FlavorsChangedCallback as a workaround to try to make automated plot generation more robust – but the ideal would be to know if the clipboard owner changed between the ‘copy’ function, and the ‘paste’ function.

manoj kumar July 30, 2014 at 13:20 Reply

I ran your screencapture.m im my computer and just included the following code to capture the image in the figure and it ran successfully.

imgData=screencapture(gcf,[790,275,750,600]);

But when I ran my code in another computer where I didn’t run your screencapture.m, i am facing error as :
Undefined function ‘screencapture’ for input arguments of
type ‘double’.

Error in manual_thresh>done (line 550)

imgData=screencapture(gcf,[790,275,750,600]);

Error while evaluating uicontrol Callback

what is the code that i need to include in my GUI code, so that it can run in any computer with running your screencapture.m file.

Thanks…

Yair Altman August 6, 2014 at 03:11 Reply

@Manoj – You need to include the screencapture.m file together with your code, otherwise the other computer will not know where this function is located (duh!).

Hoi Wong September 18, 2014 at 15:37 Reply

With ScreenCapture utility, is there a way to capture a figure without the borders and toolbar, title, etc?

This tool is awesome and super useful since TMW never managed to do ‘What You See Is What You Print (or export to EPS,PDF for publishing): fonts and text always gets changed, shifted around and multi-grid legend became an uncontrollable mess once I export/print. Thanks a lot for the great utility!

Yair Altman September 18, 2014 at 16:14 Reply

@Hoi – thanks. Please remember to add your feedback/vote to the FEX page.

You can run ScreenCapture on the figure’s top-level panel handle, assuming you have such a panel in your figure. ScreenCapture will then only take a screenshot of the panel and its contents, nothing outside it.

Hoi Wong September 18, 2014 at 16:34

Thanks. Just did that.

For simple figures done by plot(), do they have a top level panel handle? All the child objects are of the type ‘axes’.
Yair Altman September 18, 2014 at 16:36

Best thing is to have a top-level panel. If you don’t have it, try pointing ScreenCapture at the axes handle, but it may perhaps not be accurate because of the labels/tick-marks/title.
Hoi Wong September 18, 2014 at 17:00

Thanks for the reply. I have simple figures done by subplot() and plot() commands, so just pointing at one axes handle won’t work. Can you point me to a easy way to retroactively add a top-level panel to natively generated figures?

The reason I’m using the tool is for PowerPoint presentations, where the resolution doesn’t matter more than what I already seen on the screen because I am not going to zoom in the details. Having to add a uipanel then add the axes under it will be too much logistics.

I’m using Greenshot for regular screen capture, which I can use the mouse to select a window to capture (with or without toolbars+title bar). Just wondering if there’s a way to do that programmatically in MATLAB.
Yair Altman September 18, 2014 at 17:31

Of course it’s possible, it’s just that ScreenCapture doesn’t currently have this feature and I have no plans to add it in the near future. You’ll need to add a top-level uipanel. It doesn’t strike me as being such a great hassle.
Yair Altman September 18, 2014 at 17:39

Alternatively, use ScreenCapture’s ability to specify a pixel position vector (second input arg) and use the figure’s size to determine the position vector’s 3rd and 4th values. Something like [2,2,550,400].
Hoi Wong September 18, 2014 at 18:05

Thanks. I’ll try to figure out how to retroactively put the plotted figures into an uipanel. I’m not too used to it other than laying out a GUI. That’s why I thought it’s quite a hassle especially when I am trying to capture the plot windows generated by other people’s code. I’ll let you know if I figure out an adapter for ScreenCapture that does the trick 🙂

Another couple of Matlab bugs and workarounds | Undocumented Matlab November 26, 2014 at 11:04 Reply

[…] a simple printout of a screen-capture would be both much faster and 100% reliable.Which is where my ScreenCapture utility comes in. Unlike Matlab’s print and getframe, ScreenCapture takes an actual screen-capture of […]
Marc Lalancette September 4, 2015 at 14:55 Reply

Can you clarify the ‘position’ argument? When I supply get(gcf, ‘Position’), the screen capture is not in the right spot. And the help makes it sound like that’s the default… Like Hoi, I wanted to capture the inside of the figure window (which is a gui with 3 axes and other controls, but no “top-level pannel”). Thanks!

Yair Altman September 5, 2015 at 10:31 Reply

The position argument is relative to the bottom-left corner of the specified handle. For example, a value of [10,20,30,40] means a 30×40-pixel rectangle that starts at an offset of x=10, y=20 from the bottom-left corner of the specified handle.

Marc Lalancette September 9, 2015 at 14:50

Thanks so much! Thus using [8, 8, FigureWidth, FigureHeight] worked perfectly for me on Windows 7. Such a shame there isn’t any way to properly save a figure in “Matlab-provided” functions.

Krishna September 8, 2015 at 03:30 Reply

Hi Yair, I am using your ScreenCapture code and it is very useful for me. The only problem I am facing is resolution of the image. Is there any way to enhance the same?
Thanks and best regards, Krishna

Yair Altman September 8, 2015 at 06:16 Reply

@Krishna – try using the export_fig utility

Klaus December 1, 2015 at 08:58 Reply

Hello Yair,

great tool, thanks! I’m facing one issue: Stand-alone applications using screencapture crash with the following error: “Undefined function or variable ‘ImageSelection’. Do you have any idea on how to fix this?

Thank you and best regards

Klaus

Yair Altman December 1, 2015 at 09:03 Reply

@Klaus – ImageSelection is a Java class that is part of the ScreenCapture utility’s zip file. The ImageSelection.class file should be added to your deployment project so that it would be included in the distributable, otherwise it will not be there when the program attempts to call it, generating the error that you see.

Klaus December 1, 2015 at 09:40

Hello Yair,

I simply copied the ‘ImageSelection.class’ and the ‘ImageSelection.java’ files both to the deployment folder and to the folder containing the deployed m-file. The compiled program still crashes with the same error. I googled some other articles on things like “javaaddpath”. Do you know if need to tell Matlab where to find the files? I’m using deploytool to compile the program.

Thank you and best regards

Klaus
Klaus December 2, 2015 at 01:20
Hello Yair,

I just added
if isdeployed folder=pwd; folder(folder=='')='/'; javaaddpath([folder],'-end') end
if isdeployed folder=pwd; folder(folder=='')='/'; javaaddpath([folder],'-end') end
and now it works. It just gives some warnings, that “testmeas.jar”, “instrument.jar” and “images.jar” are already specified on static java path.

Many thanks

Klaus

Rocco July 14, 2016 at 17:33 Reply

Yair,
I am new to this community and I have been trying to do exactly what your ScreenCapture utility does, with one important twist. Is there a way to capture a non-matlab figure with the mouse and bring it into the matlab workspace automatically without having to save and import the data? Say I am drawing on a pen-tablet and want to select/capture a region of the drawing and bring it into matlab programmatically as rgb pixels. Is this possible? I have attempted to hire someone on freelancer to do this without luck. If it can be done in Matlab, let’s talk $ for a stand-alone executable. Thanks!

Yair Altman July 16, 2016 at 23:35 Reply

@Rocco – the ScreenCapture utility returns an imageData array that you can then use in Matlab, without needing to save and load from file. Read the utility’s documentation carefully for details. Contact me offline (email) if you still need additional consulting.

Kurt July 27, 2016 at 00:14 Reply

When I run this code I get a black box and not the screen capture. Is there something I need or am doing wrong? I have both files in the same directory. Thanks.

Yair Altman July 27, 2016 at 14:57 Reply

@Kurt – perhaps your figure is not visible on-screen (minimized, hidden, or not yet fully-rendered) when you take the screenshot.

Rocco July 27, 2016 at 20:00 Reply

Hi Yair, thanks for the reply. Very useful utility especially when used with frameless (undecorated) figure windows you mention above !
Peter Newton August 9, 2016 at 15:21 Reply
Hi,
Many thanks for a great screenshot utility. I have a question in that I want to use a variable fname2 that will have a difference file name each time I use it. The problem I am having is I can’t get the export_fig script to take the fname2 variable with the size of the screenshot to work together. I have tried putting the fname2 along with the screenshot variables into one variable, it does run it but includes the screenshot variables in the file name and ignores them. Does anyone have any ideal how I can get this to work, below is an example of what I am trying,
export_fig (fname3) -m2.5 -c[05 465 250 65] ;
export_fig (fname3) -m2.5 -c[05 465 250 65] ;
regards
Peter

Peter Newton August 9, 2016 at 15:47 Reply

Hi,
I have found this web page and have resolve my problem, as they say read the manual first.
https://github.com/altmany/export_fig
sorry to have waisted your time.
Regards
Peter

Guillaume October 27, 2016 at 08:56 Reply

Hi Yair,

I successfully use screencapture along with imwrite to save a screenshot of a maximized figure, however I can see some funny results on the bottom and left edges of the saved .png.

See https://cloud.githubusercontent.com/assets/3498686/19756046/b72e277c-9c65-11e6-901a-a862f3ef5fdc.png

Code:

% preserve the color scheme
set(hFig, 'InvertHardcopy', 'off');
 
drawnow;
 
% force figure full screen
try
    warning('off', 'MATLAB:HandleGraphics:ObsoletedProperty:JavaFrame');
    frame_h = get(handle(hFig), 'JavaFrame');
    set(frame_h, 'Maximized', 1);
catch
    disp(['Warning : JavaFrame feature not supported. Figure is going full ' ...
        'screen using normalised units which does not give best results.']);
    oldUnits = get(hFig, 'Units');
    set(hFig, 'Units', 'norm', 'Pos', [0, 0, 1, 1]);
    set(hFig, 'Units', oldUnits);
end
 
% screencapture creates an image of what is really displayed on
% screen.
imwrite(screencapture(hPanel), fileDestination);

Note: export_fig File Exchange Utility – Nguyen Quan Ba Hong June 15, 2018 at 21:13 Reply

[…] interested in high-fidelity export might also consider using Yair’s ScreenCapture utility. Unlike export_fig, which uses MATLAB’s builtin print function to generate (& fix) the […]
Sajad Rahmdel December 26, 2018 at 10:30 Reply

Thank you

HTML tags such as <b> or <i> are accepted.
Wrap code fragments inside <pre lang="matlab"> tags, like this:

<pre lang="matlab">
a = magic(3);
disp(sum(a))
</pre>

I reserve the right to edit/delete comments (read the site policies).
Not all comments will be answered. You can always email me (altmany at gmail) for private consulting.

Click here to cancel reply.

Purely documented

Under the hood

TODO list

Related posts:

35 Responses

Leave a Reply