3 Using IQML

3.1 General usage

IQML uses the IQFeed client7 to connect to the IQFeed server. If an active IQFeed client is not detected, IQML will automatically attempt to start the IQFeed client and to connect to it. Note that this may not work for some IQFeed client installations. You can always start the IQFeed client manually, before running IQML. In any case, if an IQFeed connection is unsuccessful, IQML will error.

IQML’s Matlab wrapper function is called IQML, contained within the IQML.p file. Its accompanying IQML.m file provides basic usage documentation using standard Matlab syntax, e.g.:

>> help('IQML')
>> help
IQML % equivalent alternative

>> doc IQML

The IQML function accepts a variable number of input parameters, and returns data in a single output argument, with an optional errorMsg output. The general syntax is:

>> [data, errorMsg] = IQML(action, parameters);

where:

data is the output value. If this output value is requested, then Matlab processing will block data until the result is available; if the output data is not requested then the Matlab processing will proceed immediately (non-blocking) – the IQFeed data will stream asynchronously (see below).

errorMsg is the latest error message that was reported (if any); see §3.5 below.

action is a string that denotes the requested query type (mandatory input)

parameters can be specified, depending on the requested action. There are several ways to specify parameters, which are described below.

For example:

>> data = IQML('time'); %'time' action (blocking), 0 parameters

>> IQML('quotes', 'Symbol','IBM'); %streaming 'quotes' action, 1 parameter

>> IQML('command', 'String',command, 'PortName','Admin'); %2 parameters

Note that when an output data is requested, IQML treats the request as blocking (synchronous), meaning that Matlab processing will wait for IQFeed’s data (or a timeout) before proceeding with the next Matlab command. For example:

>> t = IQML('time'); % blocking until data is available

When an output data is not requested, IQML treats the request as streaming (non-blocking, a-synchronous) and Matlab processing will proceed immediately. This non-blocking mode is typically useful for sending IQFeed requests (for example, to start streaming trades/ticks), without waiting for a response from IQFeed. The streamed data is accumulated by IQML in the background, and can later be retrieved using the mechanism that is discussed in §6 below. Examples of such non-blocking commands:

>> IQML('quotes', 'Symbol','IBM'); %start non-blocking IBM quotes stream

>> IQML('command', 'String',command); %asynchronous/non-blocking command

Here are the action values recognized by IQML, in the Standard and Professional licenses; trial licenses have the full functionality of a Professional license:

Action

Description

User Guide Section(s)

Standard

Pro & trial

'version'

Display product version information

§2.1

Yes

Yes

'license'

Display the license fingerprint & activation key

§2.2

Yes

Yes

'update'

Update the IQML installation to the latest version

§2.4

Yes

Yes

'revert'

Update the IQML installation to a previous version

§2.4

Yes

Yes

'doc'

Display this User Guide in a separate window

-

Yes

Yes

'quotes'

Fetch quotes/trades information on a ticker

§4.1, §6.1

Yes

Yes

'fundamental'

Fetch fundamental information on a ticker

§4.2

Yes

Yes

'intervalbars'

Fetch custom streaming interval bars on a ticker

§4.3, §6.3

Yes

Yes

'marketdepth'

Fetch level 2 market depth information on a ticker

§4.4, §6.4

-

Yes

'greeks'

Report option Greeks, fair value, implied volatility

§4.5

-

Yes

'history'

Fetch historical or intra-day data bars from IQFeed

§5

Yes

Yes

'summary'

Fetch historical market summary data from IQFeed

§5.6

-

Yes

'regional'

Fetch regional update information on a ticker

§6.2

-

Yes

'news'

Fetch news headlines or stories from IQFeed

§7

-

Yes

'lookup'

Fetch list of symbols/codes matching a set of criteria

§8

Yes

Yes

'chain'

Fetch futures/options chain matching a set of criteria

§8.2

-

Yes

'disconnect'

Disconnect IQML from IQFeed

§9.1

Yes

Yes

'reconnect'

Disconnect and then re-connect IQML to IQFeed

§9.1

Yes

Yes

'time'

Retrieve the latest IQFeed server & message times

§9.2

Yes

Yes

'stats'

Retrieve connection and network traffic statistics

§9.3

Yes

Yes

'command'

Send a custom command to IQFeed

§9.4

Yes

Yes

'registry'

Open Windows Registry Editor at IQFeed’s settings

§9.5

Yes

Yes

'alert'

Alert the users upon IQFeed streaming events

§11

-

Yes

IQML accepts input parameters in several alternative formats, which are equivalent – you can use whichever format that you prefer:

As name-value pairs – for example:

>> IQML('command', 'String',command, 'PortName','Admin'); %2 parameters

As a Matlab struct, with parameters contained in corresponding struct fields e.g.:

>> params = []; % initialize
>> params.String = command;
>> params.PortName =
'Admin';
>> IQML(
'command', params);

As a Matlab class, with parameters contained in corresponding class properties.

As a Matlab table, with parameters contained in corresponding table variables.

As field-separated rows in an Excel input file – for example:

>> IQML('command', 'C:\MyData\inputFile.xlsx');

where:

Each column of the file contains a separate parameter

Row #1 contains the parameter names, and rows 2+ contain their corresponding values, one row per command

All commands must have the same action ('command' in this example)

For example:

???

Each parameter must have an associated value. The value’s data type depends on the specific parameter: it could be numeric, a string, a function handle etc. The definition of all the parameters and their expected data types is listed in the appropriate section in this User Guide that explains the usage for the associated action.

Note: if you specify parameters using a struct/class/table format, and then reuse this object for different IQML commands (by altering a few parameters), the entire set of parameters will be used, possibly including some leftover parameters from previous IQML commands. This may lead to unexpected results. For example:

% 1st IQML command – stop streaming timestamp messages every 1 second
>> params = [];
% initialize
>> params.String =
'S,TIMESTAMPSOFF';
>> params.PortName =
'Level1';
>> IQML(
'command', params);

% 2nd IQML command – stop streaming client stats messages every 1 sec
>> params.String =
'S,CLIENTSTATS OFF'; %reuse existing params struct
>> IQML(
'command', params);

% 3rd IQML command – start streaming quotes messages for IBM
>> params.Symbol =
'IBM'; %reuse existing params struct
>> IQML(
'quotes', params);

In this example, the 2nd IQML command above will have no effect, because the PortName parameter in the params struct from the 1st IQML command will be reused in the 2nd command, sending it to the Level1 port, instead of to the Admin port. Similarly, the 3rd IQML command will result in a warning, because the 'quotes' action does not expect the String and PortName parameters that were carried over (reused) from the 2nd command. To avoid such unexpected results, it is therefore best to reset the object (params=[] for a struct) before preparing each IQML command.

IQML is quite tolerant of user input: parameter names (but generally not their values) are case-insensitive, parameter order does not matter, non-numeric parameter values can be specified as either char arrays ('abc') or strings ("abc"), and some of these can be shortened. For example, the following commands are all equivalent:

>> IQML('quotes', 'Symbol','IBM');
>> IQML(
'quotes', 'symbol','IBM');
>> IQML(
'Quotes', "Symbol","IBM");
>> IQML(
'Quotes', 'Symbol','IBM');
>> IQML(
'QUOTES', 'symbol',"IBM");

The full list of acceptable input parameters, and their expected values, is listed in the following sections, grouped by usage. If you specify an unexpected parameter, it will be ignored and a warning message will be displayed in the Matlab Command Window:

>> IQML('quotes', 'badName',1)

Warning: 'badName' is not a valid parameter for the 'quotes' action in IQML

When using IQML, there is no need to worry about connecting or disconnecting from the IQFeed client – IQML handles these activities automatically, without requiring user intervention. Users only need to ensure that the IQFeed client is active and logged-in when the IQML command is invoked in Matlab.

IQML reads data using the IQFeed account to which the IQFeed client is connected. In other words, the IQFeed account type is transparent to IQML: the only way to control which IQFeed data is available to IQML is to login to the IQFeed client using the appropriate username/password. Refer to §9.1 below for additional details.

3.2 Common request properties

The following properties can be specified in IQML, with most actions:

Parameter

Data type

Default

Description

Symbol or Symbols 8

string

(none)

The asset symbol, as known by IQFeed.9

Timeout

number

5.0

Max number of seconds (0-9000) to wait for data in a blocking request (0 means infinite).

Debug

logical

false or 0

If true or 1, additional information is displayed.

MsgParsingLevel

number

2

One of:

2 – parse all the data in incoming IQFeed messages (default; most verbose, slowest)

1 – do not parse lookup codes (e.g. trade condition, price formats, market id).
The corresponding Description fields will either be missing, or contain empty strings. The codes can be parsed separately (see §8).

0 – do not parse lookup code; also do not convert string data into numeric values (i.e. all data fields will remain strings: ‘3.14’). This is the fastest but least verbose option.

RaiseErrorMsgs

logical

true or 1 10

If true or 1, IQFeed error messages raise a Matlab error in blocking (non-streaming) mode (see §12)

ProcessFunc

function handle

[]

Custom user callback function to process incoming IQFeed data messages (see §10).

NumOfEvents

integer

inf

The maximal number of messages to process.

Additional properties are request-specific and are listed below in the relevant sections. For example, the 'history' action has additional properties that control the parameters of the historic data request (start/end date, data type, etc.).

3.3 Blocking & non-blocking modes

Whenever you specify an output parameter in a call to IQML, the program will block until a response is available (i.e., a synchronous request). If no output parameter is specified, IQML will immediately return (non-blocking, a-synchronous) and additional Matlab commands can immediately be issued. This non-blocking mode is typically useful for sending IQFeed requests to start streaming data (for example, streaming trades/ticks or news headlines), without waiting for any response from IQFeed. The streamed data is accumulated by IQML in the background, and can later be retrieved using the mechanism that is discussed in §6 below. For example:

>> t = IQML('time'); % blocking until data is available

>> IQML('quotes', 'Symbol','IBM'); %start non-blocking IBM quotes stream

>> IQML('command', 'String',command); %asynchronous/non-blocking command

3.4 Common causes of confusion

A common cause of error is specifying symbols incorrectly. IQFeed is very sensitive about this: if the specified symbol is invalid,11 or if your account does not have the corresponding market subscription, IQFeed will report an error:

>> IQML('quotes', 'Symbol','xyz123')

Symbol 'XYZ123' was not found!

If the request is blocking, an error (exception) will be thrown (raised), which can be trapped and handled by the user, using a Matlab try-catch construct:

try
data = IQML(
'fundamental', 'Symbol','xyz123');
catch err
fprintf(2, 'Error: %s\n', err.message);
% do some intelligent processing here...
end

IQFeed’s website includes a detailed symbol-lookup search engine.12 If you are still unsure about a symbol name, please contact IQFeed’s customer support.

If any request parameter is invalid or if the request is not accepted by IQFeed, a run-time error will result, which can be trapped as shown above. For example:

IQML historic data query (EURGBP.FXCM) error: Unauthorized user ID (your IQFeed account is not authorized for this data)

A common confusion source is specifying numeric values as strings or vice versa. For example, IQML(...,'Timeout','10') rather than IQML(...,'Timeout',10). Each IQML parameter expects a value of a specific data type, as listed in the parameter tables in this user guide. IQML is sometimes smart enough to automatically convert to the correct data type, but you should not rely on this: it is better to always use the correct data type. Otherwise, Matlab might get confused when trying to interpret the string '10' as a number, and odd results might happen.

While most of IQML’s functionality is available in all license types, some actions/functionality are only available in the Professional IQML license:

Parallelized queries (§3.6)

Customizable data fields in quotes data (§4.1, §6.1)

Level 2 market depth quotes (§4.4, §6.4, §10.5)

Option Greeks, Fair Value and Implied Volatility (§4.5)

Regional updates (§6.2)

News (§7)

Options/futures chain lookup (§8.2)

Alerts (§11)

If you have a Standard license and try to access Professional-only functionality, a run-time error will result:13

>> data = IQML('news');

The 'news' action is not available in your Standard license of IQML, only in the Professional license. Contact info@iqml.net to upgrade your license.

IQFeed reports dates in different formats, depending on the specific query: either in the standard American mm/dd/yyyy format (for example: '01/29/2018'), or in yyyymmdd format (for example: '2018-01-29' or '20180129 12:29:48'). Dates are usually reported as strings. In some cases, a corresponding Matlab datenum value is also reported, for example (§5.5, §6.1):

Symbol: 'IBM'
Timestamp: '2018-03-07 13:23:02.036440'
Datenum: 737126.557662458
...

Symbol: '@VX#'
LatestEventDatenum: 737128.637260451
LatestEventTimestamp: '20180309 15:17:39'
...

Depending on the data field, the timestamp is either your local computer’s time, or IQFeed servers (New York) time – not the exchange time. To get the exchange time, you would need to do the appropriate time-zone arithmetic.

By default, Matlab displays data in the console (“Command Window”) using “short” format, which displays numbers rounded to 4 digits after the decimal. The data actually has higher precision, so when you use it in a calculation the full precision is used, but this is simply not displayed in the console.

warningIQML does not truncate/round/modify the IQFeed data in any manner!

To display the full numeric precision in the Matlab console, change your Command Window’s Numeric Format from “short” to “long” (or “long g”) in Matlab’s Preferences window, or use the “format long” Matlab command:

>> data = IQML('quotes', 'symbol','ONLIB.X'); %overnight LIBOR rate

>> data.Close % short format (only 4 digits after decimal)
ans =
1.4463

>> format long g % long format (full precision displayed)
>> data.Close
ans =
1.446
25

1

3.5 Returned data format

Many queries in IQML return their data in the form of a struct-array (a Matlab array of structs), for example (see §8.6):

>> data = IQML('lookup', 'DataType','NAICS')
data =
1175×1 struct array with fields:
id
description

>> data(1)
ans =
id: 111110
description: 'Soybean Farming'

>> data(2)
ans =
id: 111120
description: 'Oilseed (except Soybean) Farming'

For various purposes (readability, maintainability, performance, usability), users may wish to modify this data structure. You can easily convert the data using Matlab’s builtin functions struct2cell() (which converts the struct-array into a cell-array), or struct2table() (which converts the struct-array into a Matlab table object):

>> disp(struct2cell(data)')

[111110] 'Soybean Farming'
[111120] 'Oilseed (except Soybean) Farming'
[111130] 'Dry Pea and Bean Farming'
[111140] 'Wheat Farming'
[111150] 'Corn Farming'
[111160] 'Rice Farming'
...

>> disp(struct2table(data))

id description
______ ___________________________________________________
111110 'Soybean Farming'
111120 'Oilseed (except Soybean) Farming'
111130 'Dry Pea and Bean Farming'
111140 'Wheat Farming'
111150 'Corn Farming'
111160 'Rice Farming'
...

Note that empty data cannot be converted using struct2table() or struct2cell():

>> data = IQML('lookup', 'DataType','NAICS', 'Description','xyz')
data =
[]

>> struct2cell(data)

Undefined function 'struct2cell' for input arguments of type 'double'.

>> struct2table(data)

Error using struct2table (line 26)
S must be a scalar structure, or a structure array with one column or one row.

A second, optional, output parameter of IQML returns the latest error message (if any):14

>> [data, errorMsg] = IQML('quotes', 'Symbol','IBM', 'Timeout',0.1)
data =
[]
errorMsg =
'IQML timeout: either IQFeed has no data for this query, or the Timeout parameter should be set to a value larger than 0.1'

3.6 Run-time performance

3.6.1 General considerations

IQML’s standard processing has an overhead of 1-2 milliseconds per IQFeed message, depending on several factors:

Message type/complexity – simple messages such as the periodic timestamp updates are simpler to process than complex messages (e.g. fundamental data).

The Debug parameter (see §3.2) – A value of 1/true is ~1 msec slower per message, compared to the default value of 0/false (depending on message type).

The MsgParsingLevel parameter (§3.2) – A value of 0 is ~1 msec faster per message, compared to the default value of 2 (depending on message type).

The UseParallel parameter (see below) enables query parallelization (faster).

The Fields parameter in quotes queries (§4.1, §6.1) – fewer fields are faster.

User-defined callbacks (§10) add their own processing time per message. See §10.2 for suggested ways to speed-up this callback processing overhead.

Each active alert (§11) uses 1-2 msecs per message (depending on alert type, and only for the alert’s corresponding message type). If the alert action is triggered, then its processing time is added. For example, displaying a popup message might take 1 sec, and sending an email might take a few seconds.

Computer capabilities – faster CPU and memory (RAM) enable faster processing, if your computer has enough physical memory to avoid swapping. Adding memory is typically much more cost-effective than upgrading the CPU.

This means that without any defined alert or user-specified callback, nor any other code running in the background (for example, a Matlab data analysis program), we can expect IQML to process up to 500-1000 IQFeed messages per second by default.

This is a relatively fast throughput, but if you stream real-time quotes for hundreds of liquid securities concurrently then you might reach this limit. When this happens, Matlab may be so bogged-down from the flood of incoming messages that it will become unresponsive, and you may need to restart IQConnect and/or Matlab.

Similarly, if you request a blocking (non-streaming) request with multiple data items (for example, thousands of historical data or news items), the query may take a while to process, requiring us to set a higher-than-default Timeout parameter value. For example, if you issue a blocking request for 20K data bars, IQFeed will send 20K data messages (one message per bar). If each of these messages takes 1-2 msecs to process, the total processing time for the IQML query will be 20-40 secs.

When IQFeed is connected, it continuously sends messages to IQML: periodic “heartbeat” and status messages, and messages for any active streaming quotes or news events that you requested. These messages are automatically processed by IQML in the background, reducing the CPU time that is left available to process other IQML queries (e.g., a blocking historical data query) or Matlab analysis functions. It is therefore advisable to stop streaming IQFeed data when not needed, even if only temporarily.

3.6.2 Paralellization

With the Professional and trial IQML licenses, you can use Matlab’s Parallel Computing Toolbox to parallelize IQFeed queries. This can be done both externally (placing IQML commands in parfor/spmd blocks, so that they will run independently), and internally (for some IQML query types, using the UseParallel parameter). If you have the Standard IQML license, or if you do not have Matlab’s Parallel Computing Toolbox, you can still run concurrent IQML commands in separate Matlab sessions, just not in the same session.

IQML automatically tries to parallelize queries when the UseParallel parameter value (default: false) is set to 1 or true. The list of parallelizable queries includes:

Requests resulting in multiple blocking queries in a single IQML command (for example, historical data for multiple symbols or a date range – see §5)

Requests for full news-story of all returned news headlines in a blocking query, using the GetStory parameter (see §7.2)

Requests for fundamental/quotes data on all symbols in an options-chain or futures-chain, using the WhatToShow parameter (see §8.2)

When setting UseParallel to 1 or true, IQML will use parallel Matlab tasks (so-called ‘headless workers’ or ‘labs’) from the currently-active parallel pool created by the Parallel Computing Toolbox. If no pool is active, the default local pool is automatically started.

IQML parallelization has several performance implications:

Starting the parallel pool can take some time (a few seconds, up to a minute or two, depending on configuration). It is therefore best to start the parallel pool before time-critical operations, to avoid this startup time upon the first parallel query. Starting the pool can be done using Matlab’s parpool function.

The default pool uses the same number of workers as the number of physical cores on your computer. This makes sense for CPU-intensive programs, but IQML queries are limited by I/O, not CPU. Therefore, unless you also use the parallel pool for CPU-intensive computations in your program, it makes sense to start a pool that has more workers than the number of CPU cores. You can configure your local cluster for this.15 Note that the parallel pool size should be set to ≤14, since IQFeed limits the number of concurrent connections.16

In addition to the workers startup time, each worker independently connects to IQFeed upon the first IQML command it encounters, taking an extra few secs.

It is only possible to parallelize workers on the local computer, not on other (distributed) computers in a grid/cluster/cloud. This is due to IQFeed/exchange limitations, which prohibit distribution of data to other computers.

Due to parallelization overheads, inter-task memory transfers, and CPU task-switches (especially in a case of more workers than cores), speedup will always be smaller than the number of workers. The actual speedup will depend on query type and computer/OS configuration. Parallelization may even cause slowdown in some cases (e.g. quote queries, due to waiting for market events, not CPU).

Here is a run-time example showing the effect of using a 4-worker pool to parallelize a news-story query, resulting in a 3.5x speedup (not 4x, due to parallelization overheads):

>> tic, data = IQML('news', 'DataType','headlines', 'MaxItems',100, ...
'GetStory',1); toc

Elapsed time is 56.311768 seconds.

>> parpool('local',4) % start 4 workers in parallel pool (optional)
>> tic, data = IQML('news', 'DataType','headlines', 'MaxItems',100, ...
'GetStory',1, 'UseParallel',1); toc

Elapsed time is 15.799185 seconds.

3.6.3 Quote data-fields

Also in the Professional IQML license, you can customize the fields the IQFeed reports for market data quotes. The Fields parameter can be set to a cell-array of strings ({'Bid','Ask','Last'}), or a comma-separated string ('Bid,Ask,Last'). All subsequent quotes queries, either for the latest snapshot (§4.1) or for streaming quotes (§6.1), will report just the requested fields. For example:

>> data = IQML('quotes', 'Symbol','AAPL', 'Fields',{'Bid','Ask','Last'})
>> data = IQML(
'quotes', 'Symbol','AAPL', 'Fields','Bid,Ask,Last') %equivalent
data =
Symbol: 'AAPL'
Bid: 222.71
Ask: 222.91
Last: 222.11

Note: the fewer fields that you request, the faster the required processing time, by both IQFeed and IQML. By default, IQFeed reports 16 data fields.17 However, ~50 additional fields can be requested (see §4.1 or §6.1 for details). Requesting fewer fields (as in the example above, which only requested 3 fields) will result in faster run-time processing. To improve run-time performance and reduce latency, request only those data fields that your program actually requires.


7 IQConnect.exe on Windows, IQFeed application on MacOS. or ran as a Windows app on Mac/Linux using Parallels/Wine

8 In IQML, the Symbol and Symbols parameters are synonymous – you can use either of them, in any capitalization.

9 https://iqfeed.net/symbolguide

10 Using the 2nd (optional) output parameter of IQML implies a default value of false (0) for RaiseErrorMsgs (see §3.5 below)

11 For example, EURUSD.FXCM is a valid symbol, but EURUSD and USDEUR.FXCM are invalid

12 https://iqfeed.net/symbolguide

13 A Standard license can be converted into a Professional license at any time; contact info@iqml.net for details.

14 Using the 2nd (optional) output parameter of IQML implies a default value of false (0) for the RaiseErrorMsgs parameter.

15 https://www.mathworks.com/help/distcomp/discover-clusters-and-use-cluster-profiles.html#f5-16540

16 IQFeed’s actual limit is 15, but one connection is used by the main (non-parallel) Matlab process, in addition to the workers.

17 Symbol, Most Recent Trade, Most Recent Trade Size, Most Recent Trade Time, Most Recent Trade Market Center, Total Volume, Bid, Bid Size, Ask, Ask Size, Open, High, Low, Close, Message Contents, and Most Recent Trade Conditions