- Undocumented Matlab - https://undocumentedmatlab.com -

MEX ctrl-c interrupt

Posted By Yair Altman On June 15, 2016 | 5 Comments

I recently became aware of a very nice hack [1] by Wotao Yin [2] (while at Rice in 2010; currently teaching at UCLA). The core problem is that unlike m-files that can be interrupted in mid-run using ctrl-c, MEX functions cannot be interrupted in the same way. Well, not officially, that is.
Interrupts are very important for long-running user-facing operations. They can even benefit performance by avoiding the need to periodically poll some external state. Interrupts are registered asynchronously, and the program can query the interrupt buffer at its convenience, in special locations of its code, and/or at specific times depending on the required responsiveness.
Yin reported that the libut library that ships with Matlab contain a large set of undocumented functions, including utIsInterruptPending() that can be used to detect ctrl-c interrupt events. The original report of this feature seems to be by Matlab old hand Peter Boettcher back in 2002 [3] (with a Fortran wrapper [4] reported in 2013). The importance of Yin’s post is that he clearly explained the use of this feature, with detailed coding and compilation instructions. Except for Peter’s original report, Yin’s post and the Fortran wrapper, precious few mentions can be found online (oddly enough, yours truly mentioned it [5] in the very same CSSM newsletter post in which I outed this blog back in 2009). Apparently, this feature was supposed to have been made documented in R12.1 [6], but for some reason it was not and people just moved on and forgot about it.

The relevant functions seem to be:

// Most important functions (C):
bool utIsInterruptEnabled(void)
bool utIsInterruptPending(void)
bool utWasInterruptHandled(void)
bool utSetInterruptHandled(bool)
bool utSetInterruptEnabled(bool)
bool utSetInterruptPending(bool)
// Related functions (C, signature unknown):
? utHandlePendingInterrupt(?)
? utRestoreInterruptEnabled(?)
? utLongjmpIfInterruptPending(?)
// utInterruptMode class (C++):
utInterruptMode::utInterruptMode(enum utInterruptMode::Mode)  // constructor
utInterruptMode::~utInterruptMode(void)  // destructor
bool utInterruptMode::isInterruptEnabled(void)
enum utInterruptMode::Mode utInterruptMode::CurrentMode
enum utInterruptMode::Mode utInterruptMode::GetCurrentMode(void)
enum utInterruptMode::Mode utInterruptMode::GetOriginalMode(void)
enum utInterruptMode::Mode utInterruptMode::SetMode(enum utInterruptMode::Mode)
// utInterruptState class (C++):
class utInterruptState::AtomicPendingFlags utInterruptState::flags_pending
void utInterruptState::HandlePeekMsgPending(void)
bool utInterruptState::HandlePendingInterrupt(void)
bool utInterruptState::interrupt_handled
bool utInterruptState::IsInterruptPending(void)
bool utInterruptState::IsPauseMsgPending(void)
class utInterruptState & utInterruptState::operator=(class utInterruptState const &)
void utInterruptState::PeekMessageIfPending(void)
bool utInterruptState::SetInterruptHandled(bool)
bool utInterruptState::SetInterruptPending(bool)
bool utInterruptState::SetIqmInterruptPending(bool)
bool utInterruptState::SetPauseMsgPending(bool)
bool utInterruptState::SetPeekMsgPending(bool)
void utInterruptState::ThrowIfInterruptPending(void)
bool utInterruptState::WasInterruptHandled(void)
unsigned int const utInterruptState::FLAG_PENDING_CTRLC
unsigned int const utInterruptState::FLAG_PENDING_INTERRUPT_MASK
unsigned int const utInterruptState::FLAG_PENDING_IQM_INTERRUPT
unsigned int const utInterruptState::FLAG_PENDING_PAUSE
unsigned int const utInterruptState::FLAG_PENDING_PEEKMSG

Of all these functions, we can make do with just utIsInterruptPending, as shown [1] by Yin (complete with compilation instructions):

/* A demo of Ctrl-C detection in mex-file by Wotao Yin. Jan 29, 2010. */
#include "mex.h"
#if defined (_WIN32)
    #include 
#elif defined (__linux__)
    #include 
#endif
#ifdef __cplusplus
    extern "C" bool utIsInterruptPending();
#else
    extern bool utIsInterruptPending();
#endif
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]) {
    int count = 0;
    while(1) {
        #if defined(_WIN32)
            Sleep(1000);        /* Sleep one second */
        #elif defined(__linux__)
            usleep(1000*1000);  /* Sleep one second */
        #endif
        mexPrintf("Count = %d\n", count++);  /* print count and increase it by 1 */
        mexEvalString("drawnow;");           /* flush screen output */
        if (utIsInterruptPending()) {        /* check for a Ctrl-C event */
            mexPrintf("Ctrl-C Detected. END\n\n");
            return;
        }
        if (count == 10) {
            mexPrintf("Count Reached 10. END\n\n");
            return;
        }
    }
}

After returning to Matlab, the Ctrl-C event will stop the execution of the caller function(s). However, sometimes we would like to keep the partial calculation, for example if the calculation can later be resumed from the point of interruption. It’s not possible to save partial result using only the utIsInterruptPending() function. However, it is possible to reset the interrupt state so that Matlab will not stop the execution after returning control from the mex function, and the caller function can get and use the partial calculation. The magic is done using another undocumented libut function named ​utSetInterruptPending(). A short example is included below (provided by Zvi Devir):

// Import libut functions
#pragma comment(lib, "libut.lib")
extern "C" bool utIsInterruptPending();
extern "C" bool utSetInterruptPending(bool);
// Run calculation divided into steps
int n;
for (n = 0; n < count; n++) {
	// some expensive calculation
	a_long_calculation(..., n);
	if (utIsInterruptPending()) {		// check for a Ctrl-C event
		mexPrintf("Ctrl-C detected [%d/%d].\n\n", n, count);
		utSetInterruptPending(false);	// Got it... consume event
		break;
	}
}
// Write back partial or full calculation
...

An elaboration of the idea of Ctrl-C detection was created [7] by Ramon Casero [8] (Oxford) for the Gerardus project [9]. Ramon wrapped Yin's code in C/C++ #define to create an easy-to-use pre-processor function ctrlcCheckPoint(fileName,lineNumber):

...
ctrlcCheckPoint(__FILE__, __LINE__);  // exit if user pressed Ctrl+C
...

Here's the code for the preprocessor header file (GerardusCommon.h [10]) that #defines ctrlcCheckPoint() (naturally, the __FILE__ and __LINE__ parts could also be made part of the #define, for even simpler usage):

 /*
  * Author: Ramon Casero 
  * Copyright © 2011-2013 University of Oxford
  * Version: 0.10.2
  *
  * University of Oxford means the Chancellor, Masters and Scholars of
  * the University of Oxford, having an administrative office at
  * Wellington Square, Oxford OX1 2JD, UK.
  *
  * This file is part of Gerardus.
  *
  * This program is free software: you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
  * the Free Software Foundation, either version 3 of the License, or
  * (at your option) any later version.
  *
  * This program is distributed in the hope that it will be useful,
  * but WITHOUT ANY WARRANTY; without even the implied warranty of
  * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  * GNU General Public License for more details. The offer of this
  * program under the terms of the License is subject to the License
  * being interpreted in accordance with English Law and subject to any
  * action against the University of Oxford being under the jurisdiction
  * of the English Courts.
  *
  * You should have received a copy of the GNU General Public License
  * along with this program.  If not, see
  * .
  */
#ifndef GERARDUSCOMMON_H
#define GERARDUSCOMMON_H
/* mex headers */
#include 
/* C++ headers */
#include 
#include 
#include 
#include 
#include 
/* ITK headers */
#include "itkOffset.h"
/*
 * utIsInterruptPending(): "undocumented MATLAB API implemented in
 * libut.so, libut.dll, and included in the import library
 * libut.lib. To use utIsInterruptPending in a mex-file, one must
 * manually declare bool utIsInterruptPending() because this function
 * is not included in any header files shipped with MATLAB. Since
 * libut.lib, by default, is not linked by mex, one must explicitly
 * tell mex to use libut.lib." -- Wotao Yin,
 * http://www.caam.rice.edu/~wy1/links/mex_ctrl_c_trick/
 *
 */
#ifdef __cplusplus
    extern "C" bool utIsInterruptPending();
#else
    extern bool utIsInterruptPending();
#endif
/*
 * ctrlcCheckPoint(): function to check whether the user has pressed
 * Ctrl+C, and if so, terminate execution returning an error message
 * with a hyperlink to the offending function's help, and a hyperlink
 * to the line in the source code file this function was called from
 *
 * It is implemented as a C++ macro to check for the CTRL+C flag, and
 * a call to function ctrlcErrMsgTxt() inside, to throw the error. The
 * reason is that if ctrlcCheckPoint() were a function instead of a
 * macro, this would introduce a function call at every iteration of
 * the loop, which is very expensive. But then we don't want to put
 * the whole error message part inside a macro, it's bug-prone and bad
 * programming practice. And once the CTRL+C has been detected,
 * whether the error message is generated a bit faster or not is not
 * important.
 *
 * In practice, to use this function put a call like this e.g. inside
 * loops that may take for a very long time:
 *
 *    // exit if user pressed Ctrl+C
 *    ctrlcCheckPoint(__FILE__, __LINE__);
 *
 * sourceFile: full path and name of the C++ file that calls this
 *             function. This should usually be the preprocessor
 *             directive __FILE__
 *
 * lineNumber: line number where this function is called from. This
 *             should usually be the preprocessor directive __LINE__
 *
 */
inline
void ctrlcErrMsgTxt(std::string sourceFile, int lineNumber) {
  // run from here the following code in the Matlab side:
  //
  // >> path = mfilename('fullpath')
  //
  // this provides the full path and function name of the function
  // that called ctrlcCheckPoint()
  int nlhs = 1; // number of output arguments we expect
  mxArray *plhs[1]; // to store the output argument
  int nrhs = 1; // number of input arguments we are going to pass
  mxArray *prhs[1]; // to store the input argument we are going to pass
  prhs[0] = mxCreateString("fullpath"); // input argument to pass
  if (mexCallMATLAB(nlhs, plhs, nrhs, prhs, "mfilename")) { // run mfilename('fullpath')
    mexErrMsgTxt("ctrlcCheckPoint(): mfilename('fullpath') returned error");
  }
  if (plhs == NULL) {
    mexErrMsgTxt("ctrlcCheckPoint(): mfilename('fullpath') returned NULL array of outputs");
  }
  if (plhs[0] == NULL) {
    mexErrMsgTxt("ctrlcCheckPoint(): mfilename('fullpath') returned NULL output instead of valid path");
  }
  // get full path to current function, including function's name
  // (without the file extension)
  char *pathAndName = mxArrayToString(plhs[0]);
  if (pathAndName == NULL) {
    mexErrMsgTxt("ctrlcCheckPoint(): mfilename('fullpath') output cannot be converted to string");
  }
  // for some reason, using mexErrMsgTxt() to give this output
  // doesn't work. Instead, we have to give the output to the
  // standar error, and then call mexErrMsgTxt() to terminate
  // execution of the program
  std::cerr << "Operation terminated by user during "
	    << ""
	    << mexFunctionName()
	    << " (line " << lineNumber
	    << ")"
	    << std::endl;
  mexErrMsgTxt("");
}
#define ctrlcCheckPoint(sourceFile, lineNumber)		\
  if (utIsInterruptPending()) {				\
    ctrlcErrMsgTxt(sourceFile, lineNumber);		\
  }

This feature has remained as-is since at least 2002 (when Peter first reported it), and apparently works to this day. Why then did I categorize this as "High risk for breaking in a future Matlab versions"? The reason is that internal undocumented MEX functions are prone to break in new Matlab releases (example [11]). Hopefully my report today will prompt MathWorks to make this feature documented, rather than to remove it from a future release 🙂
By the way, if anyone knows any use for the other interrupt-related functions in libut that I listed above, and/or the missing signatures, please leave a note below and I will update here accordingly.
Addendum July 2, 2016: Pavel Holoborodko just posted [12] an asynchronous version of this mechanism, which is an improvement of the synchronous code above. Pavel uses a separate Windows thread to check the Ctrl-C interrupt state. Readers can extend this idea to use threads for other asynchronous (multi-threaded) computations or I/O. In chapter 7 of my book "Accelerating MATLAB Performance [13]" I explain how we can use Posix threads (pthreads) or OpenMP threads for similar multithreading in MEX (unlike Windows threads, pthreads and OpenMP are cross platform). Users can also use other multithreading solutions, such as the open-source Boost library (bundled with Matlab) or Intel's commercial TBB.

Categories: High risk of breaking in future versions, Mex


5 Comments (Open | Close)

5 Comments To "MEX ctrl-c interrupt"

#1 Comment By Jon Chappelow On June 16, 2016 @ 01:56

I only knew of 2: utIsInterruptPending and utSetInterruptPending. The two classes, utInterruptMode and utInterruptState, are news to me. Very cool!

Since using them requires linking with libut, I include this library in the link list in my VS property sheets. See here for details: [20]

Thanks for the post, Yair!

#2 Comment By Jeremy Johnson On June 17, 2016 @ 19:49

I know you don’t usually discuss Simulink here, but would it be safe to assume that this applies to S-Functions as well since they are very similar to MEX functions?

#3 Comment By Ander Biguri On June 20, 2016 @ 11:09

Thanks for the post, very useful!
Any insight on how does this function affect the global performance of the mex function?

#4 Comment By Yair Altman On June 20, 2016 @ 11:28

@Ander – I do not believe that you will detect any noticeable performance decrease when using this mechanism.

#5 Comment By Pavel Holoborodko On July 2, 2016 @ 13:23

It depends on where you call it.

If you call it in a loop with small code size – slowdown might be significant (extra conditional & far jump to non-inlined function call).

Check the second section of the article for more details: [21]


Article printed from Undocumented Matlab: https://undocumentedmatlab.com

URL to article: https://undocumentedmatlab.com/articles/mex-ctrl-c-interrupt

URLs in this post:

[1] very nice hack: http://www.caam.rice.edu/~wy1/links/mex_ctrl_c_trick/

[2] Wotao Yin: http://math.ucla.edu/~wotaoyin

[3] Peter Boettcher back in 2002: https://www.mathworks.com/matlabcentral/newsreader/view_thread/37732#95811

[4] Fortran wrapper: https://www.mathworks.com/matlabcentral/newsreader/view_thread/326230

[5] mentioned it: https://www.mathworks.com/matlabcentral/newsreader/view_thread/247651#638201

[6] was supposed to have been made documented in R12.1: https://www.mathworks.com/matlabcentral/newsreader/view_thread/37732

[7] created: https://groups.google.com/d/msg/gerardus-users/m3rzaxHnmMM/eVo6K2WZxvYJ

[8] Ramon Casero: http://www.cs.ox.ac.uk/people/ramon.caserocanas/

[9] Gerardus project: https://github.com/rcasero/gerardus

[10] GerardusCommon.h: https://github.com/rcasero/gerardus/blob/master/matlab/GerardusCommon.h

[11] example: http://undocumentedmatlab.com/blog/serializing-deserializing-matlab-data

[12] just posted: http://www.advanpix.com/2016/07/02/devnotes-3-proper-handling-of-ctrl-c-in-mex-module

[13] Accelerating MATLAB Performance: http://undocumentedmatlab.com/books/matlab-java

[14] Faster csvwrite/dlmwrite : https://undocumentedmatlab.com/articles/faster-csvwrite-dlmwrite

[15] Undocumented feature list : https://undocumentedmatlab.com/articles/undocumented-feature-list

[16] Undocumented Matlab MEX API : https://undocumentedmatlab.com/articles/undocumented-matlab-mex-api

[17] Serializing/deserializing Matlab data : https://undocumentedmatlab.com/articles/serializing-deserializing-matlab-data

[18] Matlab mex in-place editing : https://undocumentedmatlab.com/articles/matlab-mex-in-place-editing

[19] Accessing private object properties : https://undocumentedmatlab.com/articles/accessing-private-object-properties

[20] : http://stackoverflow.com/a/27391300/2778484

[21] : http://www.advanpix.com/2016/07/02/devnotes-3-proper-handling-of-ctrl-c-in-mex-module/

Copyright © Yair Altman - Undocumented Matlab. All rights reserved.