14 Handling errors, problems, and IB messages

14.1 Messages sent from IB

IB constantly sends messages of various severity levels to IB-Matlab. These range from the mundane (e.g., “Market data farm connection is OK: cashfarm {-1, 2104}”) to the problematic (e.g., “No security definition has been found for the request {153745243, 200}”). All these messages arrive as regular events of type error, just like all the other information sent from IB (see §11 above for details).

IB-Matlab automatically infers whether an IB message is an error, warning or informational message. Errors are sent to the standard error stream (stderr) and displayed in red in the Matlab console.152 Other messages are sent to the standard output (stdout) and displayed in regular black text on the Matlab console.

Users can control the display of IB messages in the Matlab console using the MsgDisplayLevel parameter, which accepts the following possible values:

-2 – most verbose output, including all the information contained in all incoming IB events (not just messages)

-1 – display all messages as well as basic events information

0 (default) – display all messages, but not other events

1 – only display error messages, not informational messages

2 – do not display any automated output onscreen (not even errors)

The information contained in the message events varies depending on message type.153 The events have one of the following data sets:

Note: no IB message (regardless of data1, data2) is displayed if MsgDisplayLevel>=2

A typical example of such messages is the following:

>> data = IBMatlab('action','query', 'symbol','EUR');
[API.msg2] No security definition has been found for the request {153745243,200}

Most IB messages are of type msg2 (as in this example) and contain two numeric fields: data1 contains message-specific ID, which typically corresponds to the order ID or request ID that caused the problem; data2 contains the message code (type). In the example above, type='msg2', id (data1) =153745243, and code (data2) =200. This tells us that this error (code=200) occurred for request ID 153745227, so we can correlate between the error and our request. Some messages do not have an associated message-specific ID; in such cases id (data1) = -1. For example, “Market data farm connection is OK:cashfarm” (code 2104) is a general message about IB connectivity that is not related to any specific user request or order, so its id (data1) field is -1.

The full list of message codes (data2) for API.msg2 (which is the most common message type) is listed online.154 It is sub-divided into three groups:

Error messages (data2 codes between 100-999 or >10000)

System messages (data2 codes between 1000-1999)

Warning messages (data2 codes between 2000-2999)

The most recent message from IB can be queried programmatically, as follows: IBMatlab('lastIBError') returns the latest error message, which is a subset of all messages; IBMatlab('lastIBMessage') returns the latest message of any type (error, system or warning). The lastIBMessage will typically have a more recent timestamp than lastIBError, except if the latest IB message was an error (in this case both queries will return the same result):

>> IBMatlab('lastIBError')
ans =
dateNum: 738475.78192559
dateTime: '2021-11-15 18:45:58.371'
msg: 'Historical Market Data Service error message:Trading TWS session is connected from a different IP address'
type: 'msg2'
id: 1222731061
code: 162

>> IBMatlab('lastIBMessage')
ans =
dateNum: 738476.496878067
dateTime: '2021-11-16 11:55:30.265'
msg: 'Market data farm connection is OK:usfarm'
type: 'msg2'
id: -1
code: 2104

We can use this feature to handle IB errors within our program logic, for example:

data = IBMatlab('action','query', ...);
lastError = IBMatlab('lastIBError');
ONE_SECOND = 1 / 24/ 3600;
if now – lastError.dateNum < ONE_SECOND
msgbox(lastError.msg, 'IB error', 'error');
end

In order to ensure that we refer to a recent IB error due to the last request, rather than to a very old message, we can check the time that passed since the error timestamp (as in the code snippet above, which checked for an error message in the past second). Alternatively, add an optional 'clear' parameter to the IBMatlab('lastIBError') or IBMatlab('lastIBMessage') call, to clear the last error/message before sending our main query to IB. This ensures that any error/message that arrives after then can indeed be attributed to our main query:

IBMatlab('lastIBError', 'clear'); %clear the lastError
data = IBMatlab('action','query', ...); %send the main data query
lastError = IBMatlab('lastIBError'); %lastError due to the data query
if ~isempty(lastError)
msgbox(lastError.msg, 'IB error', 'error');
end

We can trap and process the message events just like any other IB events, using Matlab callbacks. Note that the parameter for message callback is CallbackMessage, although for some unknown reason the IB event is called error:

data = IBMatlab('action','query', ..., 'MsgDisplayLevel',-1, ...
'CallbackMessage',@IBMatlab_CallbackMessage);

In this example, all IB messages will be passed through IBMatlab_CallbackMessage, which is a Matlab function that we should create (it is not part of IBMatlab). Here is a sample implementation of such a message processing function that you can adapt:155

% Processing function for IB messages
function IBMatlab_CallbackMessage(ibConnector, eventData)

% Process messages based on their code:
if strcmp(eventData.type, 'msg2') % most common message: API.msg2
switch eventData.data2 % data2 contains the message code
case 100 % Max rate of message has been exceeded
msgbox(eventData.message, 'IB error');
case 200 % Ambiguous/non-existing contract specification
disp('Ambiguous (or invalid) contract specs')
...
otherwise % Any other message type
end
else % msg1 or msg3
...
end

end

IB’s error messages are often cryptic. It is sometimes difficult to understand the problem’s root cause.156 Several mechanisms can help us with this detective work:

We can set IBMatlab’s MsgDisplayLevel parameter to -1 or -2 (see above).

We can set IBMatlab’s Debug parameter to 1 (default=0). This will display in the Matlab Command Window a long list of parameters used by IBMatlab to prepare the request for IB. Check this list for any default values that should actually be set to some non-default values.

We can set the API logging level to “Detailed” in the TWS/Gateway API configuration window.157 By default it is set to “Error”, and can be changed at any time. This affects the amount of information (verbosity) logged in IB’s log files that are located in IB’s installation folder (e.g., C:\Jts).158 The log files are separated by the day of week, and have names such as: ibgateway.Thu.log, log.Wed.txt, api.8981.Tue.log. These refer, respectively, to the main Gateway log, the main TWS log, and a log of requests/responses for a specific ClientID. The api.*.log file reflects the contents of the corresponding tab in the IB Gateway application (see screenshot below159). Note that setting the logging level to “Detail” has a performance overhead and should be avoided except when debugging a specific issue. In other cases, you can set the level to “Information”, “Warning” or back to the default “Error”.

14.2 Ambiguous/invalid security errors

Much of IBMatlab’s functionality relates to a specific security that you choose to query or trade. IB is not very forgiving if you do not provide the exact security specifications (a.k.a. contract) that it expects: in such a situation, data is not returned, and an often-cryptic error message is displayed in Matlab’s Command Window:160

>> data = IBMatlab('action','query', 'symbol','EUR')
[API.msg2] No security definition has been found for the request {153745243,200}
data =
reqId: 153745243
reqTime: '13-Feb-2012 21:25:42'
dataTime: ''
dataTimestamp: -1
ticker: 'EUR'
bidPrice: -1
askPrice: -1
open: -1
close: -1
low: -1
high: -1
lastPrice: -1
volume: -1
tick: 0.01

IB does not report the specific error cause; we need to discover this ourselves. It turns out that in this specific case, some default parameter values (SecType='STK', Exchange='SMART', LocalSymbol=Symbol) are incorrect and must be overridden:

>> data = IBMatlab('action','query', 'symbol','EUR', ...
'localSymbol','EUR.USD', 'secType','cash', ...
'exchange','idealpro')
data =
reqId: 153745244
reqTime: '13-Feb-2012 21:28:51'
dataTime: '13-Feb-2012 21:28:52'
dataTimestamp: 734912.895051898
ticker: 'EUR'
bidPrice: 1.32565
askPrice: 1.32575
open: -1
close: 1.3197
low: 1.32075
high: 1.32835
lastPrice: -1
volume: -1
tick: 5e-005
bidSize: 26000000
askSize: 20521000

Some assets are traded on multiple exchanges (for example, MSFT is traded on both ISLAND and AEB); using the default Exchange='SMART' will thus cause an error.

In other cases, we may also need to specify the Currency (default='USD'). For example, the Russell 2000 index (RUT) is listed on the Toronto Stock Exchange (TSE) and trades in CAD currency. Likewise, the USD.JPY currency pair trades in Yens (JPY currency), not USD.161 Similarly, when Exchange='SMART' and Symbol='IBM', the Currency must be specified since IBM trades in either GBP or USD. Due to such potential ambiguities it is a good idea to always specify Currency.

For options/future we also need to specify the Expiry, Strike and Right parameters. In some cases, specifying the Expiry in YYYYMM format is ambiguous because the underlying contract has several separate futures/options expiring in the same month:

>> data = IBMatlab('action','query','symbol','TNA','secType','opt',...
'expiry','201202', 'strike',47, 'right','CALL')

[API.msg2] The contract description specified for TNA is ambiguous; you must specify the multiplier. {149386474, 200}

The solution is to specify the Expiry date in YYYYMMDD format (i.e., specify the exact full date rather than just the month), or to specify the Multiplier parameter.

If you are unsure of a security’s contract details, try using different parameter values. Alternatively, right-click the ticker in TWS and select “Contract Info / Description”:

Contract descriptions for USD.JPY and an IBM option, as reported in TWS

This specific example shows that the LocalSymbol for the IBM OCT12 PUT option is ‘IBM 121020P00100000’ (Symbol is ‘IBM’). This LocalSymbol has multiple spaces162. For this reason, it is best to copy-paste the value directly from the window.

Alternatively, use your TWS paper-trade (simulated trading) account to buy a virtual unit of the security, then use IB-Matlab to read the portfolio (see §4 below) and check the reported contract data. For example:

>> data = IBMatlab('action','portfolio');
>> data(3)
ans =
symbol: 'EUR'
localSymbol: 'EUR.USD'
exchange: 'IDEALPRO'
secType: 'CASH'
currency: 'USD'
right: '0'
...

As a last resort, contact IB’s API customer support help-desk (see Appendix A.1 below) to request the necessary parameters for a particular security.

Here are some examples of IB symbols:163

To get the full option chain list, see §5.4 and §11.4.

14.3 Programmatic errors

In addition to messages reported by IB, the user’s program must check for and handle cases of exceptions caused by IB-Matlab. In the vast majority of cases, these are due to invalid input parameters being passed to IBMatlab (for example, an invalid Action parameter value). However, an exception could also happen due to network problems, or even an occasional internal bug due to an unhandled edge-case situation.

To trap and handle such programmatic exceptions, wrap your calls to IBMatlab within a try-catch block, as follows:

try
data = IBMatlab('action','query', ... );
catch
% process the exception here
end

Try-catch blocks have negligible performance and memory overheads and are a very effective way to handle programmatic errors. We highly recommend that you use them very liberally within your user program, not just to wrap IBMatlab calls but also for any other processing tasks. I/O sections in particular (reading/writing to files) are prone to errors and are prime candidates for such exception handling. The same applies for processing blocks that handle user inputs (we can never really be too sure what invalid junk a user might enter in there, can we?).

Sometimes it is preferable to silently trap errors, rather than receive a Matlab exception error. This can be done by specifying a 5th output argument in the call to IBMatlab:

[orderId, ibConnector, contract, order, errMsg] = ...
IBMatlab('action','BUY',...);
if ~isempty(errMsg)
% process the error here (errMsg is a char array)
end

In most cases we do not need the ibConnector, contract, order output args of IBMatlab (see §9.6 for their usage), so such cases can be simplified to this:

[orderId, ~, ~, ~, errMsg] = IBMatlab('action','BUY',...);
if ~isempty(errMsg)
% process the error here (errMsg is a char array)
end

Programmatic errors may occur in programs that rely on valid account, portfolio or market-data data returned from IB. Unfortunately, sometimes the IB server data feed (or perhaps only the interface) is not as reliable as it could be. IB sometimes returns empty or invalid data field values, typically -1. This issue, together with some workarounds, is discussed in §14.2 and §4. User programs should implement sanity checks on the returned data, and resend the request until valid data is received. Failing to do so may result in applicative errors or bad trading decisions.

A common cause of program errors is due to specifying numeric values as strings or vice versa. For example, specifying 12 rather than “12”, or “0” rather than 0 or false. Numbers should not be enclosed with quote marks when specifying parameter values. For example, specify IBMatlab(…, 'Strike',5.20), not IBMatlab(…, 'Strike','5.20'), otherwise Matlab might get confused when trying to interpret the string '5.20' as a number. Each parameter has a specific data type, which is listed in the parameter tables in this guide. IB-Matlab is often smart enough to automatically convert to the correct data type, but you should not rely on this: it is better to always use the correct data type.

Another common cause of errors when using IB-Matlab is relying on default parameter values (for example, relying on the default SecType ('STK') for non-equity contracts (Forex, future, option etc.); see §3.2 and §14.2 for additional details.

The final type of error is out-of-memory errors, either directly in Matlab or in Java:

Matlab “out of memory” errors might occur when receiving and storing a huge amount of streaming/historic data. These can be fixed by running IB-Matlab on a computer having more memory, or by reducing the amount of stored data.164

Java memory errors are recognized by the message “java.lang.OutOfMemoryError: Java heap space”. They can be solved by increasing Matlab’s pre-allocated Java heap memory using Matlab’s preferences, or via a java.opts file.165

A possible cause of confusion is Matlab’s default use of the “short” format, which rounds numbers in the Matlab console (Command Window) to 4 digits after the decimal. This is not an error or bug: The data actually has higher precision, so when we use it in a calculation the full precision is used; this is simply not displayed in the console.

IB-Matlab does not truncate/round/modify the IB data in any manner!

To display the full numeric precision in the Matlab console, change your Matlab’s Command Window’s Numeric Format from “short” to “long” (or “long g”) in Matlab’s Preferences window, or use the “format long” Matlab command:

>> data = IBMatlab('action','query','localsymbol','EUR.USD',...);

>> data.askPrice % short format
ans =
1.0727

>> format long g % long format
>> data.askPrice
ans =
1.07265

14.4 Troubleshooting specific problems/errors