12 Messages and logging

12.1 IQML messages

To display detailed information on IQML requests and IQFeed messages, set IQML’s Debug parameter to 1 or true (default=0). IQML will then display in the Matlab console (Command Window) additional information that may help diagnose problems.

For example, setting Debug to 1 (or true) displays the outgoing commands from IQML to IQFeed (“=>”), and incoming messages from IQFeed to IQML (“<=”), along with the message’s local timestamp and port channel:181

>> data = IQML('news' ,'DataType','headlines', 'MaxItems',4, 'debug',1)

=> 20180401 15:14:00.010 (Lookup) NHL,,:,t,5,,
<= 20180401 15:14:01.082 (Lookup) N,CPR,21998204468,,20180401080059, Following Is a Test Release
<= 20180401 15:14:01.086 (Lookup) N,RTI,10134529069,,20180401080029, Quarterly Corporate Earnings (04/01/18)
<= 20180401 15:14:01.092 (Lookup) N,CPR,21998201110,,20180401073059, Following Is a Test Release
<= 20180401 15:14:01.098 (Lookup) N,CPR,21998197500,,20180401070059, April 1 Alert: Introducing, Duty Not Free: Pay-as-you-go toilet time
<= 20180401 15:14:01.107 (Lookup) !ENDMSG!

>> data = IQML('quotes', 'symbol','FB', 'debug',1)

=> 20180401 17:20:29.189 (Level1) wFB
<= 20180401 17:20:29.450 (Level1) F,FB,5,29.1,50158000,195.3200,138.8100,195.3200,149.0200,0.0000,,,,,,,,,,5.49,,2.52,12,,FACEBOOK,FB,47.600,0.63,,48563.0,3760.0,12/31/2017,,2905001,,,,,,14,4,7375,36.25,1,21,02/01/2018,04/11/2017,02/01/2018,03/26/2018,176.4600,,,,,519190,,,
<= 20180401 17:20:29.462 (Level1) P,FB,160.0500,50000,19:59:56.263577,11,0,160.0500,4600,160.0600,200,,,,159.7900,Cbacv,8801
=> 20180401 17:20:29.471 (Level1) rFB

In order to log such messages in a text file, you can use IQFeed’s built-in logging facility, as described below (§12.2).

In certain cases, IQML reports messages as red error messages on the Matlab console. Such messages can be handled by analyzing IQML’s second (optional) output argument, which is typically an empty string, except when an error is reported:

>> [data, errorMsg] = IQML('quotes', 'Symbol','IBM', 'Timeout',0.1)

IQML timeout: either IQFeed has no data for this query, or the Timeout parameter should be set to a value larger than 0.1

data =
[]
errorMsg =
'IQML timeout: either IQFeed has no data for this query, or the Timeout parameter should be set to a value larger than 0.1'

Users can control whether such error messages from IQFeed should raise a Matlab error (exception) in blocking (non-streaming) mode, using the RaiseErrorMsgs parameter (default: true when the errorMsg output arg is not requested; false otherwise)

>> data = IQML('quotes', 'Symbol','IBM', 'RaiseErrorMsgs',false);

When RaiseErrorMsgs is true (the default value) and no second output argument is requested in the IQML query, then run-time errors that occur during IQML’s processing will raise a Matlab error (exception). In such cases, the error message will typically include some context information that may assist you in locating the cause:

Error using IQML
The Symbol parameter is mandatory for 'history' requests.
when calling: IQML('history')
called from: C:\Yair\Utils\Matlab\test.m line 3

Error using IQML
IQFeed error: Symbol 'XXXXX' is invalid/unavailable!
when calling: IQML('quotes','Symbol','XXXXX')
called from: C:\Yair\Utils\Matlab\test.m line 5

In addition to IQFeed messages, your program must handle cases of IQML errors. In most cases, these are due to invalid IQML input parameters (an invalid action or parameter name, or an invalid parameter value). Errors might also happen due to network problems, or even an internal bug due to an unhandled edge-case situation.

To trap and handle such programmatic exceptions, wrap your calls to IQML within a try-catch block, as follows:

try
data = IQML('action','query', ... );
catch err
% process the error exception here, based on the contents of err
end

Try-catch blocks have very small performance or memory overhead and are a very effective way to handle programmatic errors. We recommend that you use them in your program, not just to wrap IQML calls but also for other processing tasks. I/O sections in particular (reading/writing files) are prone to errors, so they are prime candidates for such exception handling. The same applies for code that handles user inputs (we can never really be too sure what invalid junk a user might enter in there, can we?).

Very common causes of errors when using IQML are relying on default parameter values, and specifying numeric parameter values within string quotes (e.g., ‘1’ rather than 1). Users of IQML should take extra precaution in their programs to ensure that these common mistakes do not occur. See the discussion in §3.4.

Matlab “out of memory” errors might occur when receiving and storing a huge amount of streaming or historic data. They can be fixed by running IQML on a computer having more memory, or by reducing the amount of stored data.182

Java memory errors are recognized by the message “java.lang.OutOfMemoryError: Java heap space”. They can be solved by running Matlab with more allocated Java heap memory than the default value of 64MB or 128MB (depending on Matlab release). This value can be increased in Matlab’s preferences, or via a java.opts file.183

12.2 IQFeed logging

IQFeed requests, messages and events (e.g. connections/disconnections), are logged in IQFeed’s log. This is a text file called “IQConnect.txt” in the \DTN\IQFeed subfolder of “My Documents”, e.g. C:\Users\<xyz>\Documents\DTN\IQFeed\IQConnectLog.txt (replace <xyz> with the actual user name in your computer).

Using IQFeed’s Diagnostic Utility, which is installed as part of IQFeed’s client installation, you can control the log file’s folder path (but not its IQConnectLog.txt name, which is fixed), as well as the logged details:

Running IQML('log') without any parameters will return a Matlab struct containing the current log settings (the struct fields are explained below):184

>> data = IQML('log')
data =
struct with fields:
logDetails: [1 0 0 0 0]
logFile: 'C:\Users\Yair\Documents\DTN\IQFeed\IQConnectLog.txt'

In addition to interactively using the Diagnostic Utility to modify the log file path and logging details, you can also control them programmatically, via IQML’s 'log' action:

We can modify the log file’s folder path using the Path parameter, for example:

IQML('log', 'path','C:\My Programs\Logs\')

Note: Path sets the log file’s folder – not its name. As mentioned above, IQFeed’s log file name is fixed (IQConnectLog.txt) and cannot be modified, only its folder can be changed. As a safety measure, IQML will complain if the specified Path does not exist:

>> IQML('log', 'path','C:\No\Such\Folder')

Error using IQML
Bad log path specified: "C:\No\Such\Folder" is not an existing folder

When setting the log Path, the IQML command will return a Matlab struct with the log’s old filename (prior folder name and the fixed name IQConnect.txt). You can use this to restore the original log path after temporarily redirecting logging to a different folder:

data = IQML('log', 'path',newPath);
oldPath = fileparts(data.logFile); % strip out the IQConnectLog.txt file name
... % logging in this section is temporarily redirected to newPath
IQML('log', 'path',oldPath); % restore the original log file's path

Note that changing the log file’s Path does not affect a currently-running IQFeed session, only subsequent sessions (IQFeed’s IQConnect client reads the log Path when it launches). If you need to modify the Path while IQFeed is running, you can temporarily reconnect IQFeed (see §9.1) to ensure that it loads the new Path value.

Be careful when changing log Path. Its default location is set to the user’s MyDocs folder since by default IQConnect.exe launches under the user’s permission group and all users have full access rights to files in their user folder. Ensure that your user account has appropriate permissions in the choosen folder. Also ensure that the folder is not synced (to OneDrive, Dropbox etc.), which could drastically reduce system performance.

Also keep in mind that there is only one log file for all applications that use IQFeed on the computer. If your program might run alongside other software that uses IQFeed, it is recommended that you do not change the logging location from the default.

We can specify the logging details using the Details parameter, which accepts an array of up to 5 numeric or logical values. These values correspond to Administrative,185 Level 1, Level 2, Lookup,186 and Debug.187 By default, IQFeed logs only Admin requests and messages, which corresponds to Details=[1,0,0,0,0] or simply 1 (extra zeros are assumed, so 1 means the same as [1,0,0] or [1,0,0,0,0]). For example, to also log Level 1 and Lookup messages (but not Level 2 or Debug), set Details to [1,1,0,1]:

>> data = IQML('log', 'details',[1 1 0 1])
data =
struct with fields:
logDetails: [1 0 0 0 0]
logFile: 'C:\Users\Yair\Documents\DTN\IQFeed\IQConnectLog.txt'

In this example, note how IQML returned the previous Details setting, prior to its change. A subsequent call to IQML('log') will verify that the Details change was indeed made:

>> data = IQML('log')
data =
struct with fields:
logDetails: [1 1 0 1 0]
logFile: 'C:\Users\Yair\Documents\DTN\IQFeed\IQConnectLog.txt'

Note: IQFeed’s log file can become VERY large VERY fast and potentially reduce system performance if left unattended. It is intended to be used for troubleshooting purposes only and not on a constant basis, and should be used very carefully. This is especially true if you log streaming data, large historic data, and/or Debug data.

Also note that the logging level Details are stored in the computer’s registry and persist across different sessions of Matlab and IQFeed. So after you have set detailed logging and no longer need it, it is good practice to immediately set Details back to [1].

Note that unlike Path (which does not affect a currently-running IQFeed session), setting the logging Details DOES indeed affect the current session; no restart of the IQFeed client is required for the new Details setting to take effect.

In addition to setting log Path and Details, we can also use the 'log' action for several special requests: The CopyTo parameter copies the current log file into a specified folder/path. We can also specify the filename, since it is just a copy of the live log file. Note: if the specified target file already exists, it will be overwritten.

>> IQML('log', 'copyTo','C:\My Programs\Logs\log.txt'); % filename=log.txt

>> IQML('log', 'copyTo','C:\My Programs\Logs\'); % filename=IQConnectLog.txt

We can use the DoThis parameter to reset (empty) the live log file, or to display it in an external editor (the editor displays a log snapshot, it does not automatically refresh):

>> IQML('log', 'doThis','reset'); % reset (empty) the live log file

>> IQML('log', 'doThis','display'); % display live log file in external editor

Note: Both CopyTo and DoThis only affect the log file at the time of the request, not continuously. In other words, CopyTo copies a snapshot of the live log file as of the time of request; DoThis 'reset' (or 'clear' or 'empty') does a one-time reset of the log; and DoThis 'display' (or 'show') displays the current log file snapshot (the editor’s refresh of this file is not automatic).

As with other IQML actions, we can combine different parameters in a single IQML command. For example:

IQML('log', 'details',[1 1 0 1], 'path','C:\Programs\Logs\', 'doThis','reset');

Here is a summary of the IQML parameters that affect IQFeed’s internal logging: