Friday, January 2, 2009

Data Technical News #2009 - 1
NASDAQ OMX Will Modify Historical TotalView-ITCH File Format, Effective January 12, 2009

---

What is the Historical TotalView-ITCH product?

Historical TotalView-ITCH provides a daily record of the order and trade transactions as well as net order imbalance data from the NASDAQ^® Execution System. The Historical TotalView-ITCH product is designed to:

• Perform back-testing of program trading models for quality control purposes by TotalView-ITCH data feed subscribers and market data distributors
• Provide historical tick-by-tick NASDAQ order data needed by researchers and analysts
• Use Soup File specifications to enable firms to easily translate the ITCH files for data storage and query

Why is NASDAQ OMX modifying the Historical TotalView-ITCH data formats?

As outlined in NASDAQ Vendor Alert #2008-77, NASDAQ OMX^SM has experienced a rapid rise in the number of orders being processed by the NASDAQ Execution System in recent months. NASDAQ OMX has a critical field size limitation within its current TotalView-ITCH 3.0 message formats that causes major processing issues when the NASDAQ system exceeds one billion order messages in a single day. To address the situation, NASDAQ OMX released new versions of the TotalView-ITCH message formats with larger Order Reference Number fields for real-time data feed subscribers in November of 2008.

With the new TotalView-ITCH message formats in production, NASDAQ OMX is turning its attention to the Historical TotalView-ITCH product. NASDAQ OMX will begin the migration to the TotalView-ITCH 4.0 message formats for Historical TotalView-ITCH subscribers beginning Monday, January 12, 2009.

Why is the Historical TotalView-ITCH product migrating to the TotalView-ITCH 4.0 message formats?

As outlined in NASDAQ Vendor Alert #2008-77, NASDAQ OMX released two new versions of the TotalView-ITCH product for real-time data feed subscribers in November 2008. While both versions of the TotalView-ITCH messages support the same level of order and trade data, the TotalView-ITCH 4.0 data formats, which use binary code for numeric fields, are designed to support greater time stamp granularity than TotalView-ITCH 3.1 formats.

Due to the large file size, NASDAQ OMX regrets that it is only able accommodate one version of the new message formats on the Historical TotalView-ITCH service. Based on customer input, NASDAQ OMX is opting to migrate to the TotalView-ITCH 4.0 data formats for the Historical TotalView-ITCH product because of the more flexible message structure.

When will Historical TotalView-ITCH migrate to the TotalView-ITCH 4.0 message formats?

Effective Monday, January 12, 2009, NASDAQ OMX will begin to post files in the TotalView-ITCH 4.0 formats on the Historical TotalView-ITCH website and FTP servers. For information on how to access the new files, please refer to the Historical TotalView-ITCH Secured FTP Protocol Specification document.

To facilitate the transition for Historical TotalView-ITCH subscribers, NASDAQ OMX will offer an eleven week parallel period during which it will support both the current TotalView-ITCH 3.0 and the new TotalView-ITCH 4.0 message formats. The parallel period will run from January 12, 2009 to March 31, 2009.

Historical TotalView-ITCH subscribers must complete the transition to the new TotalView-ITCH 4.0 message formats by Tuesday, March 31, 2009. As of Wednesday, April 1, 2009, NASDAQ OMX will only support the TotalView-ITCH 4.0 data formats on the Historical TotalView-ITCH platform.

What technical changes will I need to make to handle the new Historical TotalView-ITCH formats?

NASDAQ OMX will add a new FTP file directory path for the TotalView-ITCH 4.0 files. For details, please refer to the Historical TotalView-ITCH Secured FTP Protocol Specification document.

To better handle the high message volumes, NASDAQ OMX will also be moving to the gzip standard for file compression for the Historical TotalView-ITCH files as part of the file format changes. For more information on this compression standard, please refer to the gzip home page.

To accommodate the TotalView-ITCH 4.0 data formats, NASDAQ OMX will also release a new binary version of the SoupFile protocol. For details, please refer to the new BinaryFile Protocol Specification document.

Firms will also need to adjust their file processing to handle the TotalView-ITCH message formats. For the new message layout, please refer to the TotalView-ITCH 4.0 Data Feed Specification document.

 

Where can I find additional information on the TotalView-ITCH product changes?

• Refer to NASDAQ Vendor Alert #2008-77.
• Refer to NASDAQ Vendor Alert #2008-91.
• Contact NASDAQ OMX Global Data Products at +1 301 978 5307.

---