As StarMine SmartEstimate® reaches its 20th anniversary, Tim Gaumer provides an overview of how the financial modelling solution has continued to perform strongly and provide value to investors.
- Refinitiv’s recent study shows that StarMine SmartEstimate is essentially as predictive today as it was 20 years ago.
- StarMine SmartEstimate remains the most accurate version of analyst consensus, and is predictive of the direction of future analyst revisions.
- When compared with the consensus, StarMine SmartEstimate is predictive of earnings surprises and the direction of those surprises.
For more data-driven insights in your Inbox, subscribe to the Refinitiv Perspectives weekly newsletter.
The StarMine SmartEstimate (SE) is a reweighting of I/B/E/S analysts estimates. It places more weight on the more recent estimates and the more accurate analysts.
Placing a greater weight on the estimates from the historically more accurate analysts on a particular company works because it’s a persistent skill. Analysts who have been the most accurate in the past are roughly four times as likely to remain among the most accurate than to drop to the least accurate category.
When compared with the equal weighted I/B/E/S Mean (or “consensus” estimate), the percentage difference between the two is named the StarMine Predicted Surprise percentage (PS%).
When there is a significant difference of greater than ± 2 per cent, the Predicted Surprise percentage gets the direction of earnings surprises directionally correct about 70 per cent of the time. The greater the PS%, the higher the hit rate. Corroborating signals, such as Mean revisions in the same direction also improves its accuracy. For more on that, please ask for a copy of our white paper SmartEstimates and the Predicted Surprise: Construction and Accuracy.
The origins of the StarMine SmartEstimate
So, how did the SmartEstimate come to be?
It goes back to the late ‘90s and the founding of StarMine. Before its acquisition by Reuters in January 2008, StarMine was a privately held, venture capital-backed stand-alone company. It was basically one of the original FinTech firms (before the term had been coined).
Everything started with StarMine’s Founder and CEO, Joe Gatto. I reached out to him about the genesis of the SmartEstimate.
He said: “I remember, back in 1995, I was trying to evaluate investing in DELL quantitatively. I knew that accurate estimates of forward earnings were critical for two reasons. First, forward earnings serve as the baseline level for comparing to price (the ‘E’ in P/E). Second, forward earnings implied a growth rate and higher growth companies command a higher P/E. So getting better forward E is doubly essential for valuation.
“I looked at estimates for Dell (on CompuServe by dial-up modem). The high estimate for the next fiscal year was $4.14, the low estimate was $1.98, and the average analyst estimate was about $3.00 per share.
“I laughed because the average was called the ‘consensus’. The word ‘consensus’ usually means ‘agreement’, yet clearly, there was no agreement among the analysts for Dell. That led me to think about measuring past analyst accuracy and automatically creating smarter estimates by putting more weight on the more accurate analysts.
“It took a few years to research the idea and raise funds to start a company. And so StarMine was born.”
StarMine may still be best known for its method of objectively measuring the performance of sell-side analysts. The best industry analysts receive ‘StarMine Awards’ to this day in many countries around the world.
Gatto recruited a stellar management team (i.e. hiring was one of his great skills). He was joined by David Lichtblau, as VP of Product Management and Marketing; Lyle Tripp, as Chief Technology Officer; and Vinesh Jha, as Director of Quantitative Research. The consultant was Haim Mozes, a professor at Fordham University who had been doing research on revision clusters.
Together they brought the SmartEstimate to life. It, and subsequent quant models, built the company and its reputation for developing profitable stock-ranking analytics. The StarMine management team grew sales to $33 million per year in recurring revenues and StarMine made the Inc 500 List of Fastest Growing U.S. Companies three years in a row before its acquisition by Reuters.
Then and now
Viewed in the original StarMine Professional research application, the SmartEstimate and Predicted Surprise looked like this:
In January 2021, the SmartEstimate for Entain Plc, in Blue, jumped well above the I/B/E/S Mean, in gold.
It remained 2.8 per cent higher, representing the Predicted Surprise percentage. Notice, across the top, StarMine calculates these metrics not just on EPS, but on a host of estimates that analysts submit to I/B/E/S. In fact, the accuracy of the Revenue Predicted Surprise in forecasting the direction of subsequent surprises is even higher than for EPS, with a historical accuracy rate of 78 per cent..
The table below shows the formulation of Entain’s EPS SmartEstimate.
The highest weight is placed on the most accurate analyst with a very recent estimate. Analysts are awarded 1-5 stars, with 5 being the most accurate. Normally, the name of the analyst and the brokerage firm would be shown – they’ve been blurred out here to protect the brokers’ intellectual property.
Below are the same views as shown in Eikon/Workspace with enhanced visuals. This can be found in the ‘Detailed Estimates’ (ESTD) view.
Data is colour-coded in bright green (or red) when a company has a large positive (negative) Predicted Surprise percentage, a large positive (negative) ‘Average Revision %’, or a positive (negative) ‘Bold Estimate’.
There is also an additional column in the analyst section entitled ‘% Difference From Mean’, which is a bar chart showing the deviation between the analyst’s estimate versus consensus. This allows users to easily identify outliers or sort the column to see how many analysts are above or below consensus.
Performance and persistence
We have published SmartEstimate and Predicted Surprise percentage accuracy results three times during its existence.
The first set of data was published in 2001. It examined the earnings surprise prediction success rate against the constituents of the Russell 3000. The success rate was nearly 70 per cent and consistent across both large- and small-cap stocks, growth versus value and by sector. And, the larger the Predicted Surprise, the more accurate the prediction became.
The second time that we published accuracy results was in the February 2009 StarMine white paper cited earlier, SmartEstimates and the Predicted Surprise: Construction and Accuracy. It found similar results:
“1) SmartEstimates are consistently more accurate than the consensus;
2) When the SmartEstimate diverges from the consensus by 2 per cent or more, the signal successfully predicts the direction of the actual surprise with a success rate of about 70 per cent across regions, market caps, sectors and financial measures; and
3) SmartEstimates demonstrate an improved accuracy at predicting earnings surprises when they are accompanied by consensus revisions in the same direction.”
Our most recent examination of its accuracy was published in a 2018 paper, An update on the performance of the StarMine SmartEstimate and Predicted Surprise. It was originally authored by Maria Vieira, Ph.D., Hugh Genin, Ph.D. and Shirley Birman from the StarMine Quantitative Research team, headed by Joe Rothermich.
The paper examined success rates for two different periods: from 1/1/1998 until 30/11/2008 and from 1/12/2008 until 30/11/2017. It verified that “the model performance remained essentially unchanged throughout the years for different sectors, capitalisations and regions.”
It further added: “The SmartEstimate continues to demonstrate better accuracy than the analyst consensus, and the Predicted Surprise continues to accurately predict actual surprises. This enduring performance underlines how StarMine models are robustly formulated based on long-lasting behavioural anomalies, and how they continue to provide value to investors.”
Empowering StarMine quant models
As a more accurate estimate, StarMine uses the SmartEstimate in its quantitative stock-ranking models whenever a forward-looking estimate is needed. It was first applied to what is now named the Analyst Revisions Model (ARM).
Inputs to it are changes in estimates for financial statement items across the income statement and multiple fiscal periods. It uses the Predicted Surprise percentage as a reinforcing signal. It also considers changes in analyst Buy/Sell/Hold recommendations.
Other StarMine models that incorporate the SmartEstimate include the Relative Valuation model, which uses both backward-looking and forward-looking valuation ratios. Another approach to valuation is the Intrinsic Valuation model. It starts with the SmartEstimate and then adjusts for the optimism bias we found in longer-term estimates, especially among faster growing companies.
In addition to the alpha-generating models, we can find the SmartEstimate in the SmartRatios Credit Risk model. It, similar to the Relative Valuation model, uses both historical actual ratios and forward 12-month ratios.
Learn more about the suite of StarMine models and analytics.
Watch: StarMine – The Combined Alpha Model
Newsletter results
For many years, my research team and I have been sticking our necks out and publishing our own predictions of five companies we expect to beat and five to miss during each upcoming earnings season.
Over the last 36 quarters, our U.S. predictions were 75 per cent accurate. We beat the performance of the Predicted Surprise percentage, on which our forecasts are based, by applying a few rules and light human overlay.
In Europe and Asia, where we have a shorter history of publishing forecasts, we were 83 per cent accurate in Europe over the last three years. In Asia ex-Japan, we were 73 per cent accurate. We published our first forecast for Japan about a year ago for FY2019 and got 9 of 10 directionally correct.
Even better during uncertain times?
One might have anticipated that the year 2020 would have challenged the accuracy of these metrics.
The global economy suffered a mighty shock during a hard shut-down and many businesses and industries were severely impacted. How could sell-side analysts have been expected to publish accurate earnings estimates?
In fact, the SmartEstimate and Predicted Surprise excelled during those uncertain times.
The four predictions we published for North America were on average 90 per cent accurate. Our prediction for Europe/UKI was 100 per cent correct – getting 10 for 10. And Asia results also came in above average – at 80 per cent.
Despite the impressive long-run persistence and accuracy of the StarMine SmartEstimate, it has delivered some of its best results during these uncertain times and continues to provide great value to investors.
Joe Gatto came up with the SmartEstimate to solve a problem he had. He was searching for a consensus “agreement” where one didn’t exist. Maybe that’s why it worked so well again in 2020. Perhaps there’s never been a year where there were so few consensus opinions about profitability forecasts, or many other things.
The SmartEstimate and other StarMine quantitative models and analytics are available in the Eikon/Workspace desktop and in QAD/QAC or as a feed for quant investors.
