Air Travel Forecast Problem 1 Objectives Introduction to forecasting methods Experience with Delphi Experience with consensus-seeking techniques Strength/weaknesses of various methods
Air Travel Forecast Problem 2 Methodology Tree for Forecasting Causal models Data mining Statistical Univariate Theory- based Data- based Extrapolation models Multivariate Rule-based forecasting Unaided judgment Judgmental SelfOthers Role playing (Simulated interaction) RoleNo role Conjoint analysis Knowledge source Quantitative analogies UnstructuredStructured FeedbackNo feedback Prediction markets Delphi Decom- position Structured analogies Methodology Tree for Forecasting forecastingpriciples.com JSA-KCG September 2005 Neural nets Expert systems Intentions/ expectations Judgmental bootstrapping Segmentation LinearClassification Game theory
Air Travel Forecast Problem 3 Techniques for Forecasting Form groups of about 5 to 7 people, then use the: Delphi procedure First estimate – individual and anonymous Statistical summary – group Group discussion (use consensus technique) Second estimate – individual and anonymous Statistical summary - group Minutes
Air Travel Forecast Problem 4 Group Results Accuracy Rankings: (Round 2) Group 12345Average ranks Judgment Bootstrapping Segmentation Causal model Extrapolations
Air Travel Forecast Problem 5 Discussion Discuss Delphi Expected results When to use Actual Results Initial hypotheses Results in Air Travel study Calculation of your error score Conclusions
Air Travel Forecast Problem 6 Delphi Agreement among experts Your results More agreement among panelists on Round 1_____ No differences (Round 1 vs. 2)_____ More agreement on Round 2_____ Findings from literature: Typically more agreement on later rounds Expected accuracy: Which do you expect to be closest to actual ranks? Your opinions Round 1 more accurate_____ Round 2 more accurate_____ No difference_____ Delphi improves accuracy vs. traditional meetings given some expertise among panelists
Air Travel Forecast Problem 7 Round 2: Previous Rankings vs. Your Rankings Method Average Ranking MBA (21 groups)* Adv. Mgmt. (28 groups)* You Judgment Bootstrapping Segmentation Causal Extrapolation *Groups from U.S., Sweden, Norway, and Netherlands
Air Travel Forecast Problem 8 Evidence-based Findings (“>” means “more accurate than”) 1.Objective methods > subjective: especially for large changes 2.Causal methods > naïve: especially for large changes 3.Bootstrapping > Judgment 4.Structured meetings > unstructured
NoYes Sufficient objective data YesNo YesNo Large changes expected Policy analysis YesNo Conflict among a few decision makers Type of knowledge Policy analysis No Yes DomainSelf YesNo Time seriesCross-section Type of data Good knowledge of relationships Policy analysis NoYes Good domain knowledge YesNo YesNo Large changes likely Similar cases exist Yes No Judgmental methodsQuantitative methods YesNo Delphi/ Prediction markets Judgmental bootstrapping/ Decomposition Conjoint analysis Intentions/ expectations Role playing (Simulated interaction/ Game theory) Structured analogies Expert systems Rule-based forecasting Extrapolation/ Neural nets/ Data mining Causal models/ Segmentation Quantitative analogies Accuracy feedback Unaided judgment No Yes Selection Tree for Forecasting Methods forecastingprinciples.com JSA-KCG January 2006 YesNo Use adjusted forecast Several methods provide useful forecasts Single method Omitted information ? Combine forecasts Use unadjusted forecast Using the Selection Tree ? 9
Air Travel Forecast Problem 10 Rankings based on Evidence-based Findings MethodRankWhy? Causal model1.5 Objective and causal Segmentation1.5 Extrapolation3Objective and naïve Bootstrapping4Objective/subjective and causal Judgment5Subjective and causal Evidence summarized in Armstrong (1985), Long-Range Forecasting, and Armstrong (2001), Principles of Forecasting – see forecastingprinciples.com
Air Travel Forecast Problem 11 Accuracy of the Different Methods of Forecasting U.S. Air Travel, (Successive updating used) Source: Armstrong & Grohman (1972) in full text at forecastingprinciples.com Forecast Horizon Years (Number Ahead of Forecasts) Mean Absolute Percentage Error* ExtrapolationJudgmentEconometric 1 (6) 2 (5) 3 (4) 4 (3) 5 (2) 6 (1) ** Averages (21) * The forecasts were lower than actual in nearly all cases. ** Estimated
Air Travel Forecast Problem 12 Average Error Scores* Round 2 MBAs7.4 Advanced Mgt.7.5 Forecasting Experts8.4 You *Key:Best possible= 0 No information (all ties)= 6 Worst possible= 12
Air Travel Forecast Problem 13 General Advice Beware of unaided judgment Be conservative when uncertain – thus, use equal ranks given uncertainty about most accurate method