12 Messages and logging

12.1 IQML messages

To display detailed information on IQML requests and IQFeed messages, set IQML’s Debug parameter to 1 or true (default=0). IQML will then display in the Matlab console (Command Window) additional information that may help diagnose problems.

For example, setting Debug to 1 (or true) displays the outgoing commands from IQML to IQFeed (“=>”), and incoming messages from IQFeed to IQML (“<=”), along with the message’s local timestamp and port channel:155

>> data = IQML('news' ,'DataType','headlines', 'MaxItems',4, 'debug',1)

=> 20180401 15:14:00.010 (Lookup) NHL,,:,t,5,,
<= 20180401 15:14:01.082 (Lookup) N,CPR,21998204468,,20180401080059,
Following Is a Test Release
<= 20180401 15:14:01.086 (Lookup) N,RTI,10134529069,,20180401080029,
Quarterly Corporate Earnings (04/01/18)
<= 20180401 15:14:01.092 (Lookup) N,CPR,21998201110,,20180401073059,
Following Is a Test Release
<= 20180401 15:14:01.098 (Lookup) N,CPR,21998197500,,20180401070059,
April 1 Alert: Introducing, Duty Not Free: Pay-as-you-go toilet time
<= 20180401 15:14:01.107 (Lookup) !ENDMSG!

>> data = IQML('quotes', 'symbol','FB', 'debug',1)

=> 20180401 17:20:29.189 (Level1) wFB
<= 20180401 17:20:29.450 (Level1) F,FB,5,29.1,50158000,195.3200,138.8100,195.3200,149.0200,0.0000,,,,,,,,,,5.49,,2.52,12,,FACEBOOK,FB,47.600,0.63,,48563.0,3760.0,12/31/2017,,2905001,,,,,,14,4,7375,36.25,1,21,02/01/2018,04/11/2017,02/01/2018,03/26/2018,176.4600,,,,,519190,,,
<= 20180401 17:20:29.462 (Level1) P,FB,160.0500,50000,19:59:56.263577,11,0,160.0500,4600,160.0600,200,,,,159.7900,Cbacv,8801
=> 20180401 17:20:29.471 (Level1) rFB

In order to log such messages in a text file, you can use IQFeed’s built-in logging facility, as described below (§12.2).

In certain cases, IQML reports messages as red error messages on the Matlab console. Such messages can be handled by analyzing IQML’s second (optional) output argument, which is typically an empty string, except when an error is reported:

>> [data, errorMsg] = IQML('quotes', 'Symbol','IBM', 'Timeout',0.1)

IQML timeout: either IQFeed has no data for this query, or the Timeout parameter should be set to a value larger than 0.1

data =
[]
errorMsg =
'IQML timeout: either IQFeed has no data for this query, or the Timeout parameter should be set to a value larger than 0.1'

Users can control whether such error messages from IQFeed should raise a Matlab error (exception) in blocking (non-streaming) mode, using the RaiseErrorMsgs parameter (default: true).

>> [data, errorMsg] = IQML('quotes', 'Symbol','IBM', 'RaiseErrorMsgs',false);

In addition to IQFeed messages, your program must handle cases of IQML errors. In most cases, these are due to invalid IQML input parameters (an invalid action or parameter name, or an invalid parameter value). Errors might also happen due to network problems, or even an internal bug due to an unhandled edge-case situation.

To trap and handle such programmatic exceptions, wrap your calls to IQML within a try-catch block, as follows:

try
data = IQML(
'action','query', ... );
catch
% process the exception here
end

Try-catch blocks have very small performance or memory overhead and are a very effective way to handle programmatic errors. We recommend that you use them in your program, not just to wrap IQML calls but also for other processing tasks. I/O sections in particular (reading/writing files) are prone to errors, so they are prime candidates for such exception handling. The same applies for code that handles user inputs (we can never really be too sure what invalid junk a user might enter in there, can we?).

Very common causes of errors when using IQML are relying on default parameter values, and specifying numeric parameter values within string quotes (e.g., ‘1’ rather than 1). Users of IQML should take extra precaution in their programs to ensure that these common mistakes do not occur. See discussion in §3.4 above.

Matlab “out of memory” errors might occur when receiving and storing a huge amount of streaming or historic data. They can be fixed by running IQML on a computer having more memory, or by reducing the amount of stored data.156

Java memory errors are recognized by the message “java.lang.OutOfMemoryError: Java heap space”. They can be solved by running Matlab with more allocated Java heap memory than the default value of 64MB or 128MB (depending on Matlab release). This value can be increased in Matlab’s preferences, or via a java.opts file.157

12.2 IQFeed logging

IQFeed requests, messages and events (e.g. connections/disconnections), are logged in IQFeed’s log. This is a text file called IQConnect.txt in the \DTN\IQFeed subfolder of “My Documents”, e.g. C:\Users\<xyz>\Documents\DTN\IQFeed\IQConnectLog.txt (replace <xyz> with the actual user name in your computer).

You can control the log file’s folder path (but not its IQConnectLog.txt name, which is fixed), as well as the logged details, using IQFeed’s Diagnostic Utility. This utility is installed as part of IQFeed’s client installation:

???

Running IQML('log') without any parameters will return a Matlab struct containing the current log settings (the struct fields are explained below):158

>> data = IQML('log')
data =
struct with fields:
logDetails: [1 0 0 0 0]
logFile: 'C:\Users\Yair\Documents\DTN\IQFeed\IQConnectLog.txt'

In addition to interactively using the Diagnostic Utility to modify the log file path and logging details, you can also control them programmatically, via IQML’s 'log' action:

We can modify the log file’s folder path using the Path parameter, for example:

IQML('log', 'path','C:\My Programs\Logs\')

Note: Path sets the log file’s folder not its name. As mentioned above, IQFeed’s log file name is fixed (IQConnectLog.txt) and cannot be modified, only its folder can be changed. As a safety measure, IQML will complain if the specified Path does not exist:

>> IQML('log', 'path','C:\No\Such\Folder')

Error using IQML
Bad log path specified: "C:\No\Such\Folder" is not an existing folder

When setting the log Path, the IQML command will return a Matlab struct with the log’s old filename (prior folder name and the fixed name IQConnect.txt). You can use this to restore the original log path after temporarily redirecting logging to a different folder:

data = IQML('log', 'path',newPath);
oldPath = fileparts(data.logFile);
% strip out the IQConnectLog.txt file name
... % logging in this section is temporarily redirected to newPath
IQML(
'log', 'path',oldPath); % restore the original log file's path

Note that changing the log file’s Path does not affect a currently-running IQFeed session, only subsequent sessions (IQFeed’s IQConnect client reads the log Path when it launches). If you need to modify the Path while IQFeed is running, you can temporarily reconnect IQFeed (see §9.1) to ensure that it loads the new Path value.

warningBe careful when changing log Path. Its default location is set to the user’s MyDocs folder because by default, IQConnect.exe launches under the user’s permission group and all users have full access rights to create, modify, and delete files in their user directory. Make sure your user account has appropriate permissions in the directory you choose.

Also keep in mind that there is only one log file for all applications that use IQFeed on the computer. If your program might run alongside other software that uses IQFeed, it is recommended that you do not change the logging location from the default.

We can specify the logging details using the Details parameter, which accepts an array of up to 5 numeric or logical values. These values correspond to Administrative,159 Level 1, Level 2, Lookup,160 and Debug.161 By default, IQFeed logs only Admin requests and messages, which corresponds to Details=[1,0,0,0,0] or simply 1 (extra zeros are assumed, so 1 means the same as [1,0,0] or [1,0,0,0,0]). For example, to also log Level 1 and Lookup messages (but not Level 2 or Debug), set Details to [1,1,0,1]:

>> data = IQML('log', 'details',[1 1 0 1])
data =
struct with fields:
logDetails: [1 0 0 0 0]
logFile: 'C:\Users\Yair\Documents\DTN\IQFeed\IQConnectLog.txt'

In this example, note how IQML returned the previous Details setting, prior to its change. A subsequent call to IQML('log') will verify that the Details change was indeed made:

>> data = IQML('log')
data =
struct with fields:
logDetails: [1 1 0 1 0]
logFile: 'C:\Users\Yair\Documents\DTN\IQFeed\IQConnectLog.txt'

warningNote: IQFeed’s log file can become VERY large VERY fast and potentially reduce system performance if left unattended. It is intended to be used for troubleshooting purposes only and not on a constant basis, and should be used very carefully. This is especially true if you log streaming data, large historic data, and/or Debug data.

Also note that the logging level Details are stored in the computer’s registry and persist across different sessions of Matlab and IQFeed. So after you have set detailed logging and no longer need it, it is good practice to immediately set Details back to [1].

Note that unlike Path (which does not affect a currently-running IQFeed session), setting the logging Details DOES indeed affect the current session; no restart of the IQFeed client is required for the new Details setting to take effect.

In addition to setting log Path and Details, we can also use the 'log' action for several special requests: The CopyTo parameter copies the current log file into a specified folder/path. We can also specify the filename, since it is just a copy of the live log file. Note: if the specified target file already exists, it will be overwritten.

>> IQML('log', 'copyTo','C:\My Programs\Logs\log.txt'); % filename=log.txt

>> IQML('log', 'copyTo','C:\My Programs\Logs\'); % filename=IQConnectLog.txt

We can use the DoThis parameter to reset (empty) the live log file, or to display it in an external editor (the editor displays a log snapshot, it does not automatically refresh):

>> IQML('log', 'doThis','reset'); % reset (empty) the live log file

>> IQML('log', 'doThis','display'); % display live log file in external editor

Note: Both CopyTo and DoThis only affect the log file at the time of the request, not continuously. In other words, CopyTo copies a snapshot of the live log file as of the time of request; DoThis 'reset' (or 'clear' or 'empty') does a one-time reset of the log; and DoThis 'display' (or 'show') displays the current log file snapshot (the editor’s refresh of this file is not automatic).

As with other IQML actions, we can combine different parameters in a single IQML command. For example:

IQML('log', 'details',[1 1 0 1], 'path','C:\Programs\Logs\', 'doThis','reset');

Here is a summary of the IQML parameters that affect IQFeed’s internal logging:

Parameter

Data type

Default

Description

Path

string

(last-used log folder path; initially set to My Documents\ \DTN\IQFeed)

Path of the folder in which the IQConnect.txt live log file is to be stored/updated.

Details

numeric or logical array

(last-used log details setting; initially set to [1,0,0,0,0] meaning only Admin msgs are logged)

Array of up to 5 numeric/logical values, corresponding to Admin, Level1, Level2, Lookup, and Debug requests/messages.
A value of 1 (or true) indicates that requests/ messages belonging to the corresponding group should be logged; a value of 0 (or false) indicates that they should not be logged.

CopyTo

string

''
(empty string)

Path of folder or file in which a snapshot copy of the live log file is to be placed, overwriting existing file if such a file already exists.

DoThis

string

''
(empty string)

One of the following string values:

'display' or 'show' – display the log file

'reset' or 'clear' – empty the live log file


155 Periodic IQFeed timestamp and client-stats messages (once every second) are not displayed, even Debug is 1 or true

156 Also see: http://www.mathworks.com/help/matlab/matlab_prog/resolving-out-of-memory-errors.html

157 https://www.mathworks.com/matlabcentral/answers/92813-how-do-i-increase-the-heap-space-for-the-java-vm-in-matlab-6-0-r12-and-later-versions

158 Note: the IQML('log') feature uses IQFeed’s Windows registry. In Mac/Linux, IQML('log') will naturally work only when Matlab runs under Parallels/Wine, not in native mode.

159 Admin includes connection/disconnection and other non-data requests and messages

160 Lookup includes history requests/data (§5), news (§7), and symbols/chains lookup (§8)

161 IQFeed’s Debug data provides even more granular logging to aid in troubleshooting issues than the other four log types. This is not related to the Debug parameter that was discussed in §12.1 above.