Recent extreme market moves are leading trading teams to rethink how they create their trading strategies and perform backtesting on them. Janelle Veasey, Head of Real-time Customer Proposition at Refinitiv, explores how cloud-based tick data is becoming the foundation of digital transformation for trading teams, generating efficiencies and opportunities for innovation.
- COVID-19’s impact on markets has accelerated existing moves by trading teams to change the way they source and use the data required to build and backtest trading strategies.
- Specifically, teams recognize the need to use historical and real-time tick data, because its enhanced granularity improves accuracy and reduces risk.
- Data scientists are finding that using cloud-based historical and real-time tick data can lower costs and improve agility, opening up the possibility of embracing new approaches.
For more data-driven insights in your Inbox, subscribe to the Refinitiv Perspectives weekly newsletter.
Extreme swings in both volumes and volatility in the markets in February and March — the result of the COVID-19 pandemic crisis — has focused minds in trading teams on the need to create better approaches to building and backtesting trading strategies.
In fact, the extreme conditions greatly exacerbated a range of pre-existing data challenges. As the ‘new normal’ evolves — as news, social media, and alternative data continues to impact securities — trading teams are recognizing the need to rethink the way they source and use the data they require.
Tick History: An archive of historical tick data drawn from our real time content
A new whitepaper from Refinitiv discusses how trading teams are engaging in digital transformation to address challenges in best execution, regulatory change, and innovation. Here, we will look in additional depth at the fourth area the white paper explores — issues specific to building and backtesting trading strategies.
Trusted data is at the foundation of all digital transformation programs, so a strategic approach to data acquisition and data usage is crucial.
Read the new whitepaper from Refinitiv to learn how trusted data can power digital transformation.
Grappling with data challenges
Even before recent events, trading teams were beginning to recognize the importance of using historical tick data for building and backtesting trading strategies.
Data sources with less frequent intervals — such as end-of-day data points — risk producing models with inaccurate results, because a whole day’s price movements are left out, and theoretically trading is restricted to the end of the day.
These problems were brought into sharp relief during the recent volatility surges. So, using historical tick data is increasingly viewed as best practice.
Alongside this, many trading teams are gravitating towards using real-time tick data from the same source as their historical tick data for building and backtesting trading strategies.
If two different data sources are used, they have to be carefully normalized, effectively doubling the effort. Also, if the two data sets are not normalized correctly, the models will produce erroneous results. So, trading teams are recognizing that taking a more holistic approach to data acquisition and use can lead to more agility and reduced risk.
Facing data access issues
Trading teams are considering other issues, too.
Historical tick databases are enormous, for example the Refinitiv database is more than six petabytes. Some financial firms have built their own, on-premises historical tick database.
Data scientists are finding that working with this large volume of data accessed via on-premise servers is becoming unmanageable. They have to wait hours for the data they need to download, and take even more time to clean and normalize the data.
These implicit costs are significant.
According to research by Refinitiv, for every $1 spent on financial market data, firms are spending an additional $8 to process, store, and transform the data before it could be analyzed. The costs in terms of missed trading opportunities can also be significant, particularly in a fast-moving market environment.
Watch: Expert Interview — How Tick History in GCP enables innovation
Transforming backtesting
To tackle these challenges, firms are pivoting towards both historical and real-time tick data in the cloud for building and backtesting trading strategies.
This move to data-as-a-service is part of a more general, digital transformation-oriented shift to the cloud. For example, an autumn 2019 Refinitiv survey revealed that 2020 budget allocations were expected to show that 55 percent of hedge funds are spending more than half of their budget on cloud services.
Data scientists argue there are substantial benefits to working with tick data in the cloud.
David Oliver, Lead Data Scientist — strategy at Refinitiv Labs, said: “Within Refinitiv Labs projects, we’ve found that access to the historical and real-time tick data in the cloud is the best way to generate data sets for machine learning.
“The fact that tick data is a huge data set is no longer a barrier now that it is in the cloud. For example, we can sift through decades of data very quickly on the most granular level; we can do things much more quickly and easily than we could before. We can now start to look at trading analytics in whole new ways because the cloud gives us the time and ability to do that.”
Refinitiv Labs is using tick data in the cloud in several of its trading-oriented projects. For example, Project Mosaic detects and explains extreme price moves in real time, so equity traders understand why prices are changing, and can react in real time.
The project uses pricing and market data, news information, and events data to provide traders with insights they can act upon.
Simulating financial markets
Another Refinitiv Labs project has created a model for simulating financial markets through a combination of agent-based modeling, synthetic data and machine learning.
This enables data scientists to use historical tick data to create fresh data sets to help them generate price paths for various ‘what-if’ scenarios. Certainly, the shock of COVID-19’s market impact has raised awareness of the importance of understanding how trading strategies might perform under unprecedented conditions.
Having historical and real time tick data in the cloud — data-as-a-service — is a real game-changer for trading teams.
Teams can get access to the data they need much quicker, which ultimately means they can more rapidly execute a trading strategy. They can also open themselves up to more innovative ways to create and perform backtesting on trading strategies. The icing on the cake is that trading teams’ overall approach to data will become much more efficient and operationally robust.
