- Undocumented Matlab - https://undocumentedmatlab.com -

ScreenCapture utility

Posted By Yair Altman On August 15, 2012 | 35 Comments

A few days ago, my ScreenCapture utility [1] was selected [2] as Matlab’s Pick of the Week (POTW). POTW selections are normally extremely useful, well-written and instructive utilities, which are both great to use in their own right, as well as a great source of knowledge about Matlab programming features and good practices. I follow the weekly POTW selections closely, and often learn new stuff from these utilities. I take pride in the fact that some of my utilities have been selected for inclusion in this unique set.
ScreenCapture enables Matlab users to take automated (programmatic) as well as interactive screen-captures of any Matlab GUI component or sub-region. This includes figure windows, axes, images, controls and even the Matlab Desktop. If the target handle for the capture is not specified, then ScreenCapture prompts the user to interactively select the capture region using an rbbox limiting box. ScreenCapture also includes a feature that plants a camera icon in the figure toolbar, such that clicking this icon will immediately trigger the interactive region-selection screen-capture.
Whichever manner the capture was made, the user then has the option of sending the output to Matlab (as a 3D RGB image matrix), or to one of the standard image file formats (e.g., JPG or PNG).
ScreenCapture has extensive help and is well-documented and relatively easy to use. For example:

figure; surf(peaks); imgData=screencapture(gcf); imshow(imgData);

ScreenCapture in action
ScreenCapture in action


Some additional usage examples:

imageData = screencapture;                     % interactively select screen-capture rectangle
imageData = screencapture(hListbox);           % capture image of a uicontrol
imageData = screencapture(0,  [20,30,40,50]);  % capture a small desktop sub-region
imageData = screencapture(gcf,[20,30,40,50]);  % capture a small figure sub-region
% capture a small sub-region of an axes
imageData = screencapture(gca,[10,20,30,40]);
imshow(imageData);  % display the captured image in a matlab figure
imwrite(imageData,'myImage.png');  % save the captured image to file
% capture a sub-region of an image
img = imread('cameraman.tif');
hImg = imshow(img);
screencapture(hImg,[60,35,140,80]);  % in data units, not pixel units
screencapture(gcf,[],'myFigure.jpg');                   % capture the entire figure into file
screencapture('handle',gcf,'filename','myFigure.jpg');  % same as previous
screencapture('toolbar',gcf);                           % adds a screen-capture button to gcf's toolbar
screencapture('toolbar',[],'file','sc.bmp');            % same, using a default output filename

Purely documented

Over the course of the past few years I have submitted 40 utilities [3] to the Matlab File Exchange, several of which have been selected for POTW. Unfortunately, since most of my utilities employ undocumented Matlab features to some extent (I can’t help myself…), they are ineligible for being selected as POTF, useful and deserving as they may be. In fact, my cprintf utility [4], which was selected as POTW, was quickly deselected as POTW because of this very issue. MathWorks fears (and I can certainly understand the concern) that highlighting a utility that relies on some undocumented feature as POTW might be considered as an official endorsement of these features.
ScreenCapture is different in this regard: it uses purely documented Matlab functionality to achieve its aims, and apparently still succeeds in providing useful functionality. This does not mean that ScreenCapture uses pure Matlab. In fact, it relies on the Java Robot class [5]‘s functionality of taking a screen-capture of a specified area of the screen. Using the Java Robot class in such a way is an entirely documented [6] Matlab feature. I have discussed the Java Robot in two past articles on this blog, where guest blogger Kesh Ikuma explained (here [7] and here [8]) how it can be used to simulate mouse and keyboard actions programmatically.

Under the hood

ScreenCapture calculates the requested screen-capture rectangle coordinates and then invokes the Java Robot to take the actual bitmap screen-capture. The output is then converted into a Matlab image matrix for output, or stored in an image file, based on the user’s choice of parameters.
Some difficulties that I overcame when programming ScreenCapture:

  • The screen position of docked windows cannot be computed reliably. Docked windows therefore need to be automatically temporarily undocked for screen-capture.
  • Undocking in Windows 7 with Aero transparency features causes the Robot to take its screen-shot before the window becomes fully opaque. Adding a short delay in undocking solved this issue.
  • Images use reversed Y-axis (Y=0 is at the axes top, not bottom). Also, when specifying a sub-region for capture, many users are used to handling images using data units (e.g., for imcrop) rather than ScreenCapture’s standard pixel units. Taking screen-captures of images proved to be a non-trivial challenge indeed.
  • Different Matlab objects (controls, axes, figures) have different external borders and internal margins. I had to take these into account in order to achieve tight-fitting image captures of these objects.
  • Performance was a problem, and it turned out that the bottleneck was trying to convert from the Java image data format to Matlab’s image data format. A couple of suggestions by Jan Simon and Urs (us) Schwartz significantly improved this performance hotspot. I’ve submitted the relevant code snippets to MathWorks for incorporation in a published technical solution [9], and I was happy to see that they have indeed incorporated them into that solution.
  • Finally, I thought that adding a custom toolbar image would be a nice touch. Since I’m not much of an artist, creating the camera icon programmatically proved to be a bit of a challenge…

Please feel free to download ScreenCapture’s code and check how I chose to program around these issues.

TODO list

Some potentially-useful features have so far eluded me in ScreenCapture’s implementation. Perhaps one day I will find a way to do them:

  1. Enable output of the image data, as an image object, to the system clipboard. It is easy to serialize the data and store it as a string in the clipboard, using the built-in clipboard function. But we would not be able to paste this data as an image into an external editor or image-processing utility. I have not yet found an easy way to store the image data as an object, although it should not be very difficult to do (here’s a starter [10]).
  2. When interactively selecting a screen-capture region, rbbox‘s starting point needs to be somewhere within the boundaries of a Matlab figure. The box can extend beyond the figure’s borders, but it has to start somewhere within the figure. I would like to be able to use rbbox without this limitation.

Addendum Jan 28. 2013: A new version of ScreenCapture was uploaded to the File Exchange [1] today which appears to solve both of the TODO issues above: The copy-to-clipboard feature relies on Jiro Doke’s imclipboard utility [11] as mentioned by Matt below [12] — simply specify the ‘clipboard’ string as the capture target (rather than a filename); the solution of the rbbox-anywhere feature relies on using a temporary transparent window [13] that spans the entire desktop area [14], capturing the user’s rbbox clicks anywhere within the desktop area. Interested readers can easily adapt the code to fit multiple monitors (I didn’t bother).

Categories: GUI, Java, Low risk of breaking in future versions


35 Comments (Open | Close)

35 Comments To "ScreenCapture utility"

#1 Comment By wei On August 17, 2012 @ 08:24

it would be nice if tooltip, active menu/submenu, ect can be captured.

#2 Comment By Yair Altman On August 20, 2012 @ 07:50

@wei – capturing tooltips and open menu items is of course impossible in interactive mode, but I’m pretty sure that it works ok in non-interactive (programmatic mode). Meaning, if a timer callback runs ScreenCapture when a menu-item is open or a tooltip is displayed, then I believe that it will indeed be captured.

#3 Comment By Matt On August 28, 2012 @ 14:18

Yair,

In regards to your ‘ToDo #1’ Jiro Doke from Mathworks seems to have a solution
[21]

Here’s the ‘paste’ code – the real transform seems to occur in when the ImageSelection object is created.

% Import necessary Java classes
import java.awt.Toolkit.*
import java.awt.image.BufferedImage
import java.awt.datatransfer.DataFlavor

% Get System Clipboard object (java.awt.Toolkit)
cb = getDefaultToolkit.getSystemClipboard();

% Get image size
ht = size(data, 1); 
wd = size(data, 2);
        
% Convert to Blue-Green-Red format
data = data(:, :, [3 2 1]);
        
% Convert to 3xWxH format
data = permute(data, [3, 2, 1]);
        
% Append Alpha data (not used)
data = cat(1, data, 255*ones(1, wd, ht, 'uint8'));
        
% Create image buffer
imBuffer = BufferedImage(wd, ht, BufferedImage.TYPE_INT_RGB);
imBuffer.setRGB(0, 0, wd, ht, typecast(data(:), 'int32'), 0, wd);
        
% Create ImageSelection object
%    % custom java class
imSelection = ImageSelection(imBuffer);
        
% Set clipboard content to the image
cb.setContents(imSelection, []);

Thanks so much for all your amazing matlab solutions.

#4 Comment By Yair Altman On August 28, 2012 @ 16:23

@Matt – thanks, I missed that one 🙂

#5 Comment By Matt On August 29, 2012 @ 17:16

Yair – On the topic of matlab clipboard interfaces- is there an easy way in these situations to check if the instance of matlab running the paste code has lost ownership of the system clipboard?

The ClipboardOwner interface has the method lostOwnership, but I’ve had difficulties figuring out how to use that method within the Matlab event handler. It would be sufficient to even be able to check the owner without a callback.
I currently use the FlavorsChangedCallback as a workaround to try to make automated plot generation more robust – but the ideal would be to know if the clipboard owner changed between the ‘copy’ function, and the ‘paste’ function.

#6 Comment By manoj kumar On July 30, 2014 @ 13:20

I ran your screencapture.m im my computer and just included the following code to capture the image in the figure and it ran successfully.

imgData=screencapture(gcf,[790,275,750,600]);

But when I ran my code in another computer where I didn’t run your screencapture.m, i am facing error as :
Undefined function ‘screencapture’ for input arguments of
type ‘double’.

Error in manual_thresh>done (line 550)

imgData=screencapture(gcf,[790,275,750,600]);

Error while evaluating uicontrol Callback

what is the code that i need to include in my GUI code, so that it can run in any computer with running your screencapture.m file.

Thanks…

#7 Comment By Yair Altman On August 6, 2014 @ 03:11

@Manoj – You need to include the screencapture.m file together with your code, otherwise the other computer will not know where this function is located (duh!).

#8 Comment By Hoi Wong On September 18, 2014 @ 15:37

With ScreenCapture utility, is there a way to capture a figure without the borders and toolbar, title, etc?

This tool is awesome and super useful since TMW never managed to do ‘What You See Is What You Print (or export to EPS,PDF for publishing): fonts and text always gets changed, shifted around and multi-grid legend became an uncontrollable mess once I export/print. Thanks a lot for the great utility!

#9 Comment By Yair Altman On September 18, 2014 @ 16:14

@Hoi – thanks. Please remember to add your feedback/vote to the FEX page.

You can run ScreenCapture on the figure’s top-level panel handle, assuming you have such a panel in your figure. ScreenCapture will then only take a screenshot of the panel and its contents, nothing outside it.

#10 Comment By Hoi Wong On September 18, 2014 @ 16:34

Thanks. Just did that.

For simple figures done by plot(), do they have a top level panel handle? All the child objects are of the type ‘axes’.

#11 Comment By Yair Altman On September 18, 2014 @ 16:36

Best thing is to have a top-level panel. If you don’t have it, try pointing ScreenCapture at the axes handle, but it may perhaps not be accurate because of the labels/tick-marks/title.

#12 Comment By Hoi Wong On September 18, 2014 @ 17:00

Thanks for the reply. I have simple figures done by subplot() and plot() commands, so just pointing at one axes handle won’t work. Can you point me to a easy way to retroactively add a top-level panel to natively generated figures?

The reason I’m using the tool is for PowerPoint presentations, where the resolution doesn’t matter more than what I already seen on the screen because I am not going to zoom in the details. Having to add a uipanel then add the axes under it will be too much logistics.

I’m using Greenshot for regular screen capture, which I can use the mouse to select a window to capture (with or without toolbars+title bar). Just wondering if there’s a way to do that programmatically in MATLAB.

#13 Comment By Yair Altman On September 18, 2014 @ 17:31

Of course it’s possible, it’s just that ScreenCapture doesn’t currently have this feature and I have no plans to add it in the near future. You’ll need to add a top-level uipanel. It doesn’t strike me as being such a great hassle.

#14 Comment By Yair Altman On September 18, 2014 @ 17:39

Alternatively, use ScreenCapture’s ability to specify a pixel position vector (second input arg) and use the figure’s size to determine the position vector’s 3rd and 4th values. Something like [2,2,550,400].

#15 Comment By Hoi Wong On September 18, 2014 @ 18:05

Thanks. I’ll try to figure out how to retroactively put the plotted figures into an uipanel. I’m not too used to it other than laying out a GUI. That’s why I thought it’s quite a hassle especially when I am trying to capture the plot windows generated by other people’s code. I’ll let you know if I figure out an adapter for ScreenCapture that does the trick 🙂

#16 Pingback By Another couple of Matlab bugs and workarounds | Undocumented Matlab On November 26, 2014 @ 11:04

[…] a simple printout of a screen-capture would be both much faster and 100% reliable.Which is where my ScreenCapture utility comes in. Unlike Matlab’s print and getframe, ScreenCapture takes an actual screen-capture of […]

#17 Comment By Marc Lalancette On September 4, 2015 @ 14:55

Can you clarify the ‘position’ argument? When I supply get(gcf, ‘Position’), the screen capture is not in the right spot. And the help makes it sound like that’s the default… Like Hoi, I wanted to capture the inside of the figure window (which is a gui with 3 axes and other controls, but no “top-level pannel”). Thanks!

#18 Comment By Yair Altman On September 5, 2015 @ 10:31

The position argument is relative to the bottom-left corner of the specified handle. For example, a value of [10,20,30,40] means a 30×40-pixel rectangle that starts at an offset of x=10, y=20 from the bottom-left corner of the specified handle.

#19 Comment By Marc Lalancette On September 9, 2015 @ 14:50

Thanks so much! Thus using [8, 8, FigureWidth, FigureHeight] worked perfectly for me on Windows 7. Such a shame there isn’t any way to properly save a figure in “Matlab-provided” functions.

#20 Comment By Krishna On September 8, 2015 @ 03:30

Hi Yair, I am using your ScreenCapture code and it is very useful for me. The only problem I am facing is resolution of the image. Is there any way to enhance the same?
Thanks and best regards, Krishna

#21 Comment By Yair Altman On September 8, 2015 @ 06:16

@Krishna – try using the [22]

#22 Comment By Klaus On December 1, 2015 @ 08:58

Hello Yair,

great tool, thanks! I’m facing one issue: Stand-alone applications using screencapture crash with the following error: “Undefined function or variable ‘ImageSelection’. Do you have any idea on how to fix this?

Thank you and best regards

Klaus

#23 Comment By Yair Altman On December 1, 2015 @ 09:03

@Klaus – ImageSelection is a Java class that is part of the ScreenCapture utility’s zip file. The ImageSelection.class file should be added to your deployment project so that it would be included in the distributable, otherwise it will not be there when the program attempts to call it, generating the error that you see.

#24 Comment By Klaus On December 1, 2015 @ 09:40

Hello Yair,

I simply copied the ‘ImageSelection.class’ and the ‘ImageSelection.java’ files both to the deployment folder and to the folder containing the deployed m-file. The compiled program still crashes with the same error. I googled some other articles on things like “javaaddpath”. Do you know if need to tell Matlab where to find the files? I’m using deploytool to compile the program.

Thank you and best regards

Klaus

#25 Comment By Klaus On December 2, 2015 @ 01:20

Hello Yair,

I just added

if isdeployed
    folder=pwd;
    folder(folder=='\')='/';
    javaaddpath([folder],'-end')
end

and now it works. It just gives some warnings, that “testmeas.jar”, “instrument.jar” and “images.jar” are already specified on static java path.

Many thanks

Klaus

#26 Comment By Rocco On July 14, 2016 @ 17:33

Yair,
I am new to this community and I have been trying to do exactly what your ScreenCapture utility does, with one important twist. Is there a way to capture a non-matlab figure with the mouse and bring it into the matlab workspace automatically without having to save and import the data? Say I am drawing on a pen-tablet and want to select/capture a region of the drawing and bring it into matlab programmatically as rgb pixels. Is this possible? I have attempted to hire someone on freelancer to do this without luck. If it can be done in Matlab, let’s talk $ for a stand-alone executable. Thanks!

#27 Comment By Yair Altman On July 16, 2016 @ 23:35

@Rocco – the ScreenCapture utility returns an imageData array that you can then use in Matlab, without needing to save and load from file. Read the utility’s documentation carefully for details. Contact me offline (email) if you still need additional consulting.

#28 Comment By Kurt On July 27, 2016 @ 00:14

When I run this code I get a black box and not the screen capture. Is there something I need or am doing wrong? I have both files in the same directory. Thanks.

#29 Comment By Yair Altman On July 27, 2016 @ 14:57

@Kurt – perhaps your figure is not visible on-screen (minimized, hidden, or not yet fully-rendered) when you take the screenshot.

#30 Comment By Rocco On July 27, 2016 @ 20:00

Hi Yair, thanks for the reply. Very useful utility especially when used with frameless (undecorated) figure windows you mention above !

#31 Comment By Peter Newton On August 9, 2016 @ 15:21

Hi,
Many thanks for a great screenshot utility. I have a question in that I want to use a variable fname2 that will have a difference file name each time I use it. The problem I am having is I can’t get the export_fig script to take the fname2 variable with the size of the screenshot to work together. I have tried putting the fname2 along with the screenshot variables into one variable, it does run it but includes the screenshot variables in the file name and ignores them. Does anyone have any ideal how I can get this to work, below is an example of what I am trying,

export_fig (fname3) -m2.5 -c[05 465 250 65] ;

regards
Peter

#32 Comment By Peter Newton On August 9, 2016 @ 15:47

Hi,
I have found this web page and have resolve my problem, as they say read the manual first.
[23]
sorry to have waisted your time.
Regards
Peter

#33 Comment By Guillaume On October 27, 2016 @ 08:56

Hi Yair,

I successfully use screencapture along with imwrite to save a screenshot of a maximized figure, however I can see some funny results on the bottom and left edges of the saved .png.

See [24]

Code:

% preserve the color scheme
set(hFig, 'InvertHardcopy', 'off');

drawnow;

% force figure full screen
try
    warning('off', 'MATLAB:HandleGraphics:ObsoletedProperty:JavaFrame');
    frame_h = get(handle(hFig), 'JavaFrame');
    set(frame_h, 'Maximized', 1);
catch
    disp(['Warning : JavaFrame feature not supported. Figure is going full ' ...
        'screen using normalised units which does not give best results.']);
    oldUnits = get(hFig, 'Units');
    set(hFig, 'Units', 'norm', 'Pos', [0, 0, 1, 1]);
    set(hFig, 'Units', oldUnits);
end

% screencapture creates an image of what is really displayed on
% screen.
imwrite(screencapture(hPanel), fileDestination);

#34 Pingback By Note: export_fig File Exchange Utility – Nguyen Quan Ba Hong On June 15, 2018 @ 21:13

[…] interested in high-fidelity export might also consider using Yair’s ScreenCapture utility. Unlike export_fig, which uses MATLAB’s builtin print function to generate (& fix) the […]

#35 Comment By Sajad Rahmdel On December 26, 2018 @ 10:30

Thank you


Article printed from Undocumented Matlab: https://undocumentedmatlab.com

URL to article: https://undocumentedmatlab.com/articles/screencapture-utility

URLs in this post:

[1] ScreenCapture utility: http://www.mathworks.com/matlabcentral/fileexchange/24323-ScreenCapture

[2] selected: http://blogs.mathworks.com/pick/2012/08/10/grab-a-screen-shot-from-within-matlab/

[3] 40 utilities: http://www.mathworks.com/matlabcentral/fileexchange/index?term=authorid%3A27420

[4] cprintf utility: http://undocumentedmatlab.com/blog/cprintf-display-formatted-color-text-in-command-window/

[5] Java Robot class: http://download.oracle.com/javase/6/docs/api/java/awt/Robot.html

[6] documented: http://www.mathworks.com/help/techdoc/matlab_external/f44062.html

[7] here: http://undocumentedmatlab.com/blog/gui-automation-robot/

[8] here: http://undocumentedmatlab.com/blog/gui-automation-utilities/

[9] technical solution: http://www.mathworks.com/support/solutions/en/data/1-2WPAYR/

[10] starter: http://www.exampledepot.com/egs/java.awt.datatransfer/ToClipImg.html

[11] imclipboard utility: http://www.mathworks.com/matlabcentral/fileexchange/28708-imclipboard

[12] below: http://undocumentedmatlab.com/blog/screencapture-utility/#comment-106188

[13] transparent window: http://undocumentedmatlab.com/blog/transparent-matlab-figure-window/

[14] spans the entire desktop area: http://undocumentedmatlab.com/blog/minimize-maximize-figure-window/

[15] An interesting uitree utility : https://undocumentedmatlab.com/articles/interesting-uitree-utility

[16] savezip utility : https://undocumentedmatlab.com/articles/savezip-utility

[17] PlotEdit context-menu customization : https://undocumentedmatlab.com/articles/plotedit-context-menu-customization

[18] Frameless (undecorated) figure windows : https://undocumentedmatlab.com/articles/frameless-undecorated-figure-windows

[19] GUI automation using a Robot : https://undocumentedmatlab.com/articles/gui-automation-robot

[20] GUI automation utilities : https://undocumentedmatlab.com/articles/gui-automation-utilities

[21] : http://www.mathworks.com/matlabcentral/fileexchange/28708-imclipboard/content/imclipboard.m

[22] : http://www.mathworks.com/matlabcentral/fileexchange/23629-export-fig

[23] : https://github.com/altmany/export_fig

[24] : https://cloud.githubusercontent.com/assets/3498686/19756046/b72e277c-9c65-11e6-901a-a862f3ef5fdc.png

Copyright © Yair Altman - Undocumented Matlab. All rights reserved.