1. Air Travel Forecast Problem 1 Objectives Introduction to forecasting methods Experience with Delphi Experience with consensus-seeking techniques Strength/weaknesses of various methods

2. Air Travel Forecast Problem 2 Methodology Tree for Forecasting Causal models Data mining Statistical Univariate Theory- based Data- based Extrapolation models Multivariate Rule-based forecasting Unaided judgment Judgmental SelfOthers Role playing (Simulated interaction) RoleNo role Conjoint analysis Knowledge source Quantitative analogies UnstructuredStructured FeedbackNo feedback Prediction markets Delphi Decom- position Structured analogies Methodology Tree for Forecasting forecastingpriciples.com JSA-KCG September 2005 Neural nets Expert systems Intentions/ expectations Judgmental bootstrapping Segmentation LinearClassification Game theory

3. Air Travel Forecast Problem 3 Techniques for Forecasting Form groups of about 5 to 7 people, then use the: Delphi procedure First estimate – individual and anonymous Statistical summary – group Group discussion (use consensus technique) Second estimate – individual and anonymous Statistical summary - group Minutes 12 3 20 2 3 40

4. Air Travel Forecast Problem 4 Group Results Accuracy Rankings: (Round 2) Group 12345Average ranks Judgment Bootstrapping Segmentation Causal model Extrapolations

5. Air Travel Forecast Problem 5 Discussion Discuss Delphi Expected results When to use Actual Results Initial hypotheses Results in Air Travel study Calculation of your error score Conclusions

6. Air Travel Forecast Problem 6 Delphi Agreement among experts Your results More agreement among panelists on Round 1_____ No differences (Round 1 vs. 2)_____ More agreement on Round 2_____ Findings from literature: Typically more agreement on later rounds Expected accuracy: Which do you expect to be closest to actual ranks? Your opinions Round 1 more accurate_____ Round 2 more accurate_____ No difference_____ Delphi improves accuracy vs. traditional meetings given some expertise among panelists

7. Air Travel Forecast Problem 7 Round 2: Previous Rankings vs. Your Rankings Method Average Ranking MBA (21 groups)* Adv. Mgmt. (28 groups)* You Judgment2.22.4 Bootstrapping3.22.9 Segmentation2.22.0 Causal2.62.9 Extrapolation4.74.8 *Groups from U.S., Sweden, Norway, and Netherlands

8. Air Travel Forecast Problem 8 Evidence-based Findings (“>” means “more accurate than”) 1.Objective methods > subjective: especially for large changes 2.Causal methods > naïve: especially for large changes 3.Bootstrapping > Judgment 4.Structured meetings > unstructured

9. NoYes Sufficient objective data YesNo YesNo Large changes expected Policy analysis YesNo Conflict among a few decision makers Type of knowledge Policy analysis No Yes DomainSelf YesNo Time seriesCross-section Type of data Good knowledge of relationships Policy analysis NoYes Good domain knowledge YesNo YesNo Large changes likely Similar cases exist Yes No Judgmental methodsQuantitative methods YesNo Delphi/ Prediction markets Judgmental bootstrapping/ Decomposition Conjoint analysis Intentions/ expectations Role playing (Simulated interaction/ Game theory) Structured analogies Expert systems Rule-based forecasting Extrapolation/ Neural nets/ Data mining Causal models/ Segmentation Quantitative analogies Accuracy feedback Unaided judgment No Yes Selection Tree for Forecasting Methods forecastingprinciples.com JSA-KCG January 2006 YesNo Use adjusted forecast Several methods provide useful forecasts Single method Omitted information ? Combine forecasts Use unadjusted forecast Using the Selection Tree ? 9

10. Air Travel Forecast Problem 10 Rankings based on Evidence-based Findings MethodRankWhy? Causal model1.5 Objective and causal Segmentation1.5 Extrapolation3Objective and naïve Bootstrapping4Objective/subjective and causal Judgment5Subjective and causal Evidence summarized in Armstrong (1985), Long-Range Forecasting, and Armstrong (2001), Principles of Forecasting – see forecastingprinciples.com

11. Air Travel Forecast Problem 11 Accuracy of the Different Methods of Forecasting U.S. Air Travel, 1963-1968 (Successive updating used) Source: Armstrong & Grohman (1972) in full text at forecastingprinciples.com Forecast Horizon Years (Number Ahead of Forecasts) Mean Absolute Percentage Error* ExtrapolationJudgmentEconometric 1 (6) 2 (5) 3 (4) 4 (3) 5 (2) 6 (1) 5.7 12.7 17.4 22.5 27.5 29.9 6.8 15.6 25.1 34.1 42.1 45.0** 4.2 6.8 7.3 9.8 6.2 0.7 Averages (21)19.328.15.8 * The forecasts were lower than actual in nearly all cases. ** Estimated

12. Air Travel Forecast Problem 12 Average Error Scores* Round 2 MBAs7.4 Advanced Mgt.7.5 Forecasting Experts8.4 You *Key:Best possible= 0 No information (all ties)= 6 Worst possible= 12

13. Air Travel Forecast Problem 13 General Advice Beware of unaided judgment Be conservative when uncertain – thus, use equal ranks given uncertainty about most accurate method