style="font-size:15px;"> For instance, one study funded by the Division of Research at the Harvard Business School examined the July 1997 to December 2004 period and reached the following conclusions: “Sell-side firm analysts make more optimistic and less accurate earnings forecasts than their buy-side counterparts. In addition, abnormal returns from investing in their Strong Buy/Buy recommendations are negative and under-perform comparable sell-side recommendations.”15
There is a wide range of performance results within the sell-side analyst universe. For example, one study concluded that sell-side securities analysts ranked well by buy-side users of sell-side research out-performed lesser ranked sell-side analysts.16 (Note: This study, which was sponsored by the William E. Simon Graduate School of Business Administration, reviewed performance results from 1991 to 2000.)
How does media exposure affect forecasters?
To see how the working environment can affect the quality of advice, look at Wall Street's emphasis on “instant analysis.” Wall Street economists often devote considerable time and care to preparing economic-indicator forecasts. However, within seconds – literally, seconds – after data are reported at the normal 8:30 a. m. time, economists are called on to determine the implications of an economics report and announce them to clients.
Investment banks and trading firms want their analysts to offer good advice. But they also want publicity. They're happy to offer their analysts to the cameras for the instant analysis prized by the media. The awareness that a huge national television audience is watching and will know if they err can be stressful to the generally studious and usually thorough persons often attracted to the field of economics. Keep this in mind when deciding whether the televised advice of an investment bank analyst is a useful input for decision making. (Note: Securities firms in the current, more regulation-conscious decade generally scrutinize analysts' published reports, which should make the reports more reliable than televised sound bites.)
7. Audiences may condition forecasters' perceptions of professional risks.
John Maynard Keynes famously said: “Practical men, who believe themselves to be quite exempt from any intellectual influences, are usually the slaves of some defunct economist.” Forecasters subconsciously or consciously risk becoming the slaves of their intended audience of colleagues, employers, and clients. In other words, seers often fret about the reaction of their audience, especially if their proffered advice is errant. How the forecaster frames these risks is known as the loss function.
In some situations, such pressures can be constructive. The first trader I met on my first day working as a Wall Street economist had this greeting: “I like bulls and I like bears but I don't like chickens.” The message was clear: No one wants to hear anything from a two-handed economist. That was constructive pressure for a young forecaster embarking on a career.
That said, audience pressures might not be so benign. Yet they are inescapable. The ability to deal with them in a field in which periodic costly errors are inevitable is the key to a long, successful career for anyone giving advice about the future.
8. Statistics courses are not enough. It takes both math and experience to succeed.
To be sure, many dedicated statistics educators are also scholars working to advance the science of statistics. However, teaching and its attendant focus on academic research inevitably leaves less time for building a considerable body of practical experience.
No amount of schooling could have prepared me for what I experienced during my first week as a Wall Street economist in 1980. Neither a PhD in economics from Columbia University nor a half-dozen years as an economist at the Federal Reserve Bank of New York and the Bank for International Settlements in Basel, Switzerland had given me the slightest clue as to how to handle my duties as PaineWebber's Chief Money Market Economist.
At the New York Fed, my ability to digest freshly released labor market statistics, and to write a report about them before the close of business, helped trigger an early promotion for me. But on PaineWebber's New York fixed-income trading floor, I was expected to digest and opine on those same very important monthly data no more than five minutes after they hit the tape at 8:30 a. m.
There were other surprises as well. In graduate school, for example, macroeconomics courses usually skipped national income accounting and measurement. These topics were regarded as simply descriptive and too elementary for a graduate level academic curriculum. Instead, courses focused on the mathematical properties of macroeconomic mechanics and econometrics as the arbiters of economic “truth.” On Wall Street, however, the ability to understand and explain the accounting that underlies any important government or company data report is key to earning credibility with a firm's professional investor clients. In graduate school we did study more advanced statistical techniques. But they were mainly applied to testing hypotheses and studying statistical economic history, not forecasting per se.
In short, when I first peered into my crystal ball, I was behind the eight ball! As in the game of pool, survival would depend on bank shots that combined skill, nerve, and good luck. Fortunately, experience pays: More seasoned forecasters generally do better. (See Figure 1.3. The methodology for calculating the illustrated forecaster scores is discussed in Chapter 2.)
Figure 1.3 More Experienced Forecasters Usually Fare Better
*Number of surveys in which forecaster participated.Source: Andy Bauer, Robert A. Eisenbeis, Daniel F. Waggoner, and Tao Zha, “Forecast Evaluation with Cross-Sectional Data: The Blue Chip Survey,” Federal Reserve Bank of Atlanta, Second Quarter, 2003.
In summation, then, it is difficult to be prescient because:
• Behavioral sciences are inevitably limited.
• Interpreting current events and history is challenging.
• Important causal factors may not be quantifiable.
• Work environments and audiences can bias forecasts.
• Experience counts more than statistical courses.
Bad Forecasters: One-Hit Wonders, Perennial Outliers, and Copycats
Some seers do much better than others in addressing the difficulties cited earlier. But what makes these individuals more accurate? The answer is critical for learning how to make better predictions and for selecting needed inputs from other forecasters. We first review some studies identifying characteristics of both successful and unsuccessful forecasters. That is followed in Chapter 2 by a discussion of my experience in striving for better forecasting accuracy throughout my career.
What Is “Success” in Forecasting?
A forecast is any statement regarding the future. With this broad definition in mind, there are several ways to evaluate success or failure. Statistics texts offer a number of conventional gauges for judging how close a forecaster comes to being right over a number of forecast periods. (See an explanation and examples of these measures in Chapter 2.) Sometimes, as in investing, where the direction of change is more important than the magnitude of change, success can be defined as being right more often than being wrong. Another criteria can be whether a forecaster is correct about outcomes that are especially important in terms of costs of being wrong and benefits of being right (i.e., forecasting the big one.)
Over a forecaster's career, success will be judged by all three criteria – accuracy, frequency of being correct,
Boris Groysberg, Paul Healy, Greg Chapman, and Yang Gui, “Do Buy-Side Analysts Out-Perform the Sell-Side?” Division of Research at Harvard Business School, July 13, 2005.
Andrew Leone and Joanna Shuang Wu, “What Does It Take to Become a Superstar? Evidence from Institutional Investor Rankings of Financial Analysts,” William E. Simon Graduate School of Business Administration – University of Rochester, May 23, 2007.
