Presentation is loading. Please wait.

Presentation is loading. Please wait.

Why Forecasts Differ and Why are they so Bad? Roy Batchelor Professor of Banking and Finance, Cass Business School.

Similar presentations

Presentation on theme: "Why Forecasts Differ and Why are they so Bad? Roy Batchelor Professor of Banking and Finance, Cass Business School."— Presentation transcript:

1 Why Forecasts Differ and Why are they so Bad? Roy Batchelor Professor of Banking and Finance, Cass Business School


3 But back in the real world… Forecasters often look (and feel) stupid This is because  Sometimes they can’t help it.  Sometimes they deliberately make biased forecasts. Aim is to separate these causes of error, using evidence from panels of forecasters  Consensus Economics  Blue Chip Financial Forecasts Payoff – which forecasters are worth listening to, and when?

4 Plan of lecture Insights from recession forecasts Reasons for biased forecasts Who is most biased?


6 Forecasting the 1991 recession

7 Forecasting the 2009 recession

8 Should we privatise GDP forecasts? “Ageing model leads Treasury astray” (Sunday Times 9/2/92) “Time to take forecasting away from the Treasury” (Sunday Times, 12/4/92). What do you think?

9 Forecasts for 1990

10 Forecasts for 2008

11 Forecasts for 2010: … it’s too soon to know

12 Recession forecasts in general Loungani, P, 2001, How accurate are private sector forecasts? Cross-country evidence from consensus forecasts of output growth, International Journal of Forecasting, 17, 419-432

13 Stylised facts about Forecasts It’s hard to forecast recession. These are rare events, always with different causes. Consensus forecast usually starts close to the average growth rate, and then adjusts Accuracy-improving information arrives only about 12-15 months in advance of the end of the target year. Individual forecasts are distributed above and below the consensus in a consistent way. There are persistent optimists and pessimists.

14 Plan of lecture Insights from recession forecasts Reasons for biased forecasts Who is most biased?

15 Reasons For Bias Incompetence (unlikely, really…) Bias in the Consensus:  Learning about Structural Breaks  Market Incentives for Bias Bias in Individual Forecasts  Market Incentives for Product Differentiation

16 Learning about Structural Change Important Many countries (Japan, Germany, Italy, France) experienced slowdown in trend growth Optimal forecast will be biased to optimism as forecasters learn about the new trend Evidence supports this - forecasts in US, UK less biased

17 US – Log(GDP) and Forecast (Non-) Bias

18 Italy – Log(GDP) and Forecast Bias

19 Market incentives for bias Important for Investment Analysts Bias to optimism in earnings forecasts can arise from  Selection of firms/ sectors you believe in  Relationship building  Trade generation However, regulatory changes (Sarbanes Oxley) can change incentives… May also apply to some government forecasts

20 Financial analysts: Bias to optimism 1990-2003

21 Financial analysts: Bias to pessimism 2003-

22 Dispersion v Herding in Individual Forecasts There is evidence of herding – excessive convergence on the consensus – for financial analysts However, there is evidence of the excessive dispersion of forecasts by economic forecasters, who  underweight information in the consensus forecast (Batchelor and Dua, 1992, J Forecasting )  Maintain consistently optimistic or pessimistic priors from year to year (Batchelor and Dua, 1990, Int J Forecasting; Batchelor, 2007, Int J Forecasting)

23 Herding Financial forecasters have incentives to overweight consensus forecasts:  “Information cascade”: forecasts are made sequentially, so each forecast becomes part of the next forecasters prediction set. In aggregate published forecasts are biased towards the early forecasts.  “Incentive concavity”: rewards for an accurate “bold” forecast are smaller than penalties for an inaccurate bold forecast. Less experienced forecasters herd more since career prospects are at stake.

24 Dispersion and Product Differentiation Hotelling “location model” – don’t set up your store right next to everyone else, but don’t be too far out of town. So profit maximising strategy is to shade forecasts consistently away from the Consensus (even if you believe it is the best forecast) Batchelor and Dua (1990), Batchelor (2007) find individual forecasters persistently make optimistic or pessimistic forecasts relative to the consensus. Interpreted as an attempt to differentiate their product, increase press coverage, book sales, speaking fees etc. Does not harm accuracy, except in extreme cases

25 Evolution of Forecast Disagreement Lahiri, K., and X Shen, 2008, Evolution of Forecast Disagreement in a Bayesian learning Model, Journal of Econometrics, 144, 325-340.

26 News and Dispersion of Forecast Revisions Forecasts converge, but quarterly GDP releases give forecasters an opportunity to put different spins on the figures

27 Plan of lecture Insights from recession forecasts Reasons for biased forecasts Who is most biased?

28 Blue Chip Financial Forecasts

29 Method Forecasts from US Blue Chip Financial Forecasters  TB3, TB30, RGDP, CPI,  1983-1997 (+ updating), Horizons 15 mths – 1 mth Questionnaire on Forecaster Characteristics  Sent to 80 BCFF participants, Nov 1993 - Jan 1994  43 useable responses  25 with full track record.

30 Ranking by Forecast Quality For the 25 forecasters, we compute average (over horizon) ranks for each target variable by Bias (actual-forecast, low rank = overprediction) Extremism (absolute deviation from consensus forecast, high rank = far from consensus) Accuracy (RMSE, low rank = high accuracy) Calculate Rank Correlations with Forecaster Characteristics

31 Individual Characteristics Please provide the following information about you and your organisation: Type of Organisation: Location: Highest College Degree (tick): Bachelors Masters PhD Number of years experience in forecasting Percentage of work time spent in forecasting Number of staff involved in forecasting at your organisation

32 Individual effects: Rank Correlations Banks made higher, less extreme, more accurate GDP forecasts Experienced forecasters made slightly more extreme forecasts of TB3, RGDP, but no convincing evidence of extremism Location, education, attention, size of team not significant

33 Clientele Please indicate the relative importance of the following groups as users of your forecasts (weights should add to 100): Traders inside your organisation Other colleagues inside your organisation Clients of your organisation General public Other

34 Concentration measures We have constructed measures of concentration of Clientele, Technique, Theory and Information weights  A forecaster who only served external clients would have a high concentration measure (UCONC)  A forecaster who put equal weight on all types of user would have a low concentration measure Hypothesis is that low concentration may reduce extremism and improve accuracy

35 Strong Clientele Effects Forecasters giving weight to traders, internal users, or public, made less extreme and more accurate forecasts Forecasters with external clients made more extreme and less accurate forecasts Concentration on one type of client also increases extremism

36 Forecast Techniques In making forecasts of US interest rates 3 to 6 months ahead, what weight do you assign to the following forecast techniques: (weights should add to 100) Econometric Models (structural, regression) Time Series Models (Box Jenkins, ARIMA, VAR) Exponential Smoothing methods Technical Analysis (Chart Analysis) Judgment Other Also give weights for forecasting 1 year ahead and beyond, if different.

37 Weak Technique effects Smoothing methods are associated with higher accuracy, but little used Use of Judgement, and Concentration on one technique increased extremism and reduced accuracy of real GDP forecasts

38 Theory If you use Econometric Models, what weight do you put on the following types of economic theory: (weights should add to 100) Keynesian Monetarist Rational Expectations Supply Side Business Cycle Other (specify)

39 Some Theory effects Keynesians made low forecasts of interest rates Monetarists made low forecasts of interest rates, inflation, high forecasts of real growth. Business Cycle theorists had the opposite biases RE theorists made consistently high forecasts of real growth Few differences in forecast accuracy, however

40 Judgment If you use Judgment, what weight so you place on the following processes? (weights should add to 100) Own analysis of current event Group analysis within your organisation (meetings, surveys) Other (specify)

41 Some Judgment Process effects Some evidence that group processes improve accuracy of interest rate forecasts, reliance on own judgment harms accuracy

42 Information If you use Judgment, what weight so you place on the following pieces of information? (weights should add to 100) Current Official Economic Statistics (GDP, Inflation, …) Forecasts made by other organisations (e.g. Blue Chip Financial Forecasts) Surveys of Consumer and Business Confidence Other (specify)

43 Strong Information Source effects Forecasters who place a lot of weight on the forecasts of other forecasters are less extreme, and significantly more accurate on interest rate and real GDP forecasts Forecasters who rely heavily on one source of information tend to be less accurate

44 Who should you listen to? Less extreme  work in bank  serve general public More accurate  serve general public  use any theory  … except RE  pay attention to other forecasters More extreme  rational expectations theory  listen to friends, boss  overweight news Less accurate  limited range of users (e.g. only external clients)  doctrinaire use of theory (e.g. only Monetarism)

Download ppt "Why Forecasts Differ and Why are they so Bad? Roy Batchelor Professor of Banking and Finance, Cass Business School."

Similar presentations

Ads by Google