Market data managers and their teams are beginning to recognise the extraordinary benefits of moving from on-premise ticker plants into the cloud. Costs per gigabyte in storage and compute decrease significantly, while the performance accelerates, and the range of use cases the tick history data can be applied to multiplies. The outcome can be an increased competitive “first to market” advantage, as well as a reduction in overall total cost of ownership.
- Firms are facing twin challenges of an accelerating demand for tick history data, while costs for acquiring, managing, storing, and using this data on-premises in ticker plants are also escalating exponentially.
- As a result, firms are shifting their market data strategy for tick history from an on-premises systems-based design to a data-centric approach in the cloud.
- Some Refinitiv clients have reduced their total cost of ownership (TCO) of tick history data by more than 90 percent by moving to Refinitiv Tick History in Google Cloud Platform.
For more data-driven insights in your Inbox, subscribe to the Refinitiv Perspectives weekly newsletter.
Financial services firms are at a data crossroads.
Their need for tick history data has never been greater, across the front, middle and back offices. Whether tick data is used to do backtesting on trading strategies, to perform transaction cost analysis, or to support a new machine learning or artificial intelligence solution, firms need to be able to engage with this data more than ever before. The volume of data they require is also escalating at a dramatic speed.
High costs compel change in market data strategy
However, the costs associated with acquiring, managing, storing, and using tick history data in an on-premises ticker plant increase as more and more data is required to perform the types of analytics and machine learning algorithms used in today’s market.
According to a recent Refinitiv survey, for every $1 spent on financial market data, a further $8 is spent on processing, storing, and transforming that data before it can be analysed. In fact, the Refinitiv Tick History team have completed detailed analysis of the expenditure associated with an on-premises approach to the management of tick history data. These expenses include:
- Compute Server footprint – This includes the cost of the servers for processing the data, along with the housing of the hardware in a building. Other costs include the cooling, racking, networking connectivity and other supporting infrastructure such as built-in redundancy solutions.
- Storage hardware – The cost of providing and storing database solutions for the data, along with the ability to retrieve the data for the various market-related use cases. This also includes the costs for the cooling, racking, networking connectivity and other supporting infrastructure including built-in redundancy solutions.
- Off-premises storage backup – For business continuity plans, firms will need to have a copy of their tick history database running in another location. It is often a regulatory requirement to have this.
- Relational time series database technology fees – Charges paid to the database software provider (KDB, Microsoft, Oracle etc.).
- Human capital – The cost of the employees who manage databases and network infrastructure, and manage capacity planning for the hardware, database maintenance and storage.
These costs can be substantial and they continue to grow as more and more data is stored on- premises – with tick data, the volume of data stored only ever increases as each day of data is added in.
In total, for firms that want to continue keeping their tick history data on-premises, Refinitiv has assessed the cost to be around $110 per gigabyte of stored data. These expenses are expected to continue to escalate over time as the volume of tick history data continues to grow and grow.
Moving from system-centric to data-centric design
Given the twin pressures of the increasing use of tick history data across the front, middle, and back offices, and the rising costs associated with this data when it is stored in an on-premises ticker plant, it’s little wonder that financial firms are looking for a way to transform this situation.
Many firms are shifting their market data strategy for tick history data from a system-centric to a data-centric approach based in the cloud.
With a systems-based approach, the customer has the responsibility for building a system to house the data themselves. This results in costly on-premises hardware infrastructure, creating differing versions of the data across the organisation as individual internal groups produce subsets of the data for diverse use cases.
Additionally, centralised on-premises tick data creates difficulties for geographically dispersed users, again usually resulting in multiple copies of the data. These issues can create significant costs as well as data integrity challenges.
Leveraging cloud technology presents a significant opportunity. Moving to a data-centric market data strategy for tick history can solve many of these issues at once. The first step is to engage with tick history data and analytics in the cloud. Taking this approach enables:
- Ticker plant rationalisation
- On-premises hardware decommissioning
- Resource and infrastructure reallocation
- Leveraging of new cloud capabilities for data science and analytics
- Migrating workflows to the cloud – ‘query in the cloud – analyse in the cloud’
As a result of these benefits, some Refinitiv clients have reduced their total cost of ownership of tick history data by more than 90 percent by moving to the cloud.
When working with Refinitiv Tick History in Google Cloud Platform, the cost can be as low as $4 per gigabyte because the stored data, the database and the compute engine are moved to the cloud.
It’s clear that adopting a data-centric cloud-based solutions for data, analytics and workflow has significant potential to become an important source of competitive advantage for financial services firms.
Watch: Refinitiv Tick History – 25 years of historical data from 500+ venues across all asset classes
Making tick history in the cloud a reality
Refinitiv Tick History in Google Cloud Platform is a managed service database that contains the full range of Refinitiv Tick History data.
Refinitiv Tick History is a historical archive of real-time pricing data, covering OTC and exchange-traded instruments, from more than 500 trading venues and third-party contributors. The database contains more than 70 million active and retired securities, with coverage that goes back to 1996. Overall, the database holds more than 6PB of historical data.
The Refinitiv Tick History data in Google Cloud Platform is delivered via the Refinitiv Data Platform. The Refinitiv Data Platform offers access to global market data and analytics as a service in the cloud.
Firms can power and deploy their applications with reduced time to market by leveraging the Refinitiv consistent data model, and permissioning and entitlement capabilities, as well as cloud-based tools such as the Google® BigQuery engine.
Firms no longer need to clean, normalise and manage the tick history data – the Refinitiv data is query-ready. Firms can query this database directly in the cloud by using the Google® BigQuery engine, either via the Google provided GUI or via API from the firm’s preferred environment (e.g., Jupyter) and programming language.
After running analytics, firms can leave the results in the cloud, move them to other cloud locations, or to an on-premises system that the firm has – in short ‘query in the cloud – analyse in the cloud’.
Deployment of Refinitiv Tick History in Google Cloud Platform has resulted in significant cost savings for clients.
To help you better understand how moving to a data-centric market data approach could impact your firm, the Refinitiv Tick History team are able to perform a tailored analysis to discover the true size of existing and ongoing costs associated with on-premises tick history data, versus adopting Refinitiv Tick History data in Google Cloud Platform.
The team are also able to talk with stakeholders about the other significant benefits of adopting a data-centric approach to tick history data, and to provide a free trial. To learn more about this analysis, contact us, and specifically mention the Refinitiv Tick History in Google Cloud Platform cost analysis.
