Professor of Engineering Systems Civil and Environmental Engineering Massive Uncertainty Richard de Neufville Professor of Engineering Systems and of Civil and Environmental Engineering Massive Uncertainty: Slide 1 of 22 / RdN ©
Massive Uncertainty: Slide 2 of 22 / RdN © Objective: To present realistic context of forecasting exercise Topics Evidence Consequences Massive Uncertainty: Slide 2 of 22 / RdN ©
Massive Uncertainty: Slide 3 of 22 / RdN © Causes of Uncertainty Underlying variability of phenomenon Difficulties in measurement or estimation Unforeseen or “unpredictable” circumstances Limits to valid measurement for example: behavioral patterns Massive Uncertainty: Slide 3 of 22 / RdN ©
Massive Uncertainty: Slide 4 of 22 / RdN © Evidence 1. Simple Physical Systems 2. Overall Traffic 3. Local Traffic (Worse) 4. Other Operations Massive Uncertainty: Slide 4 of 22 / RdN ©
Ratio of Real Costs to Estimated Costs for Airport Projects Costs expressed in constant dollars Median ~= 1.25 Percent of Occurrences Real/Estimated Cost Ratio Massive Uncertainty: Slide 5 of 22 / RdN ©
Cost Growth for Various Projects 3.0 2.5 2.0 1.5 1.0 0.5 0.0 Ratio of Actual to Estimated Cost DOD60 HWAY WATER BLDNG DOD50 ADHOC MAJOR ENRGY NASA NASA AVG St.Dev. CONST SAT m g Massive Uncertainty: Slide 6 of 22 / RdN ©
NASA Projects Cost Growth 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0 Ratio of Actual to Estimated Cost HST GLL UARS GRO COBE MGL MOBS LSAT EUVE ERBE AVG St.Dev. Massive Uncertainty: Slide 7 of 22 / RdN ©
Massive Uncertainty: Slide 8 of 22 / RdN © Results of a 2004 study Adapted from: Terminal Area Forecast (TAF) Accuracy Assessment Results Jerome Friedman, MITRE CAASD, Sept. 30, 2004 Massive Uncertainty: Slide 8 of 22 / RdN ©
Massive Uncertainty: Slide 9 of 22 / RdN © Results of a 2004 study Note: Average error ~ 11% Adapted from: Terminal Area Forecast (TAF) Accuracy Assessment Results Jerome Friedman, MITRE CAASD. Study dated Sept. 30, 2004, but data until 2000. Deliberate omission of 2001, 2002 – when traffic dropped enormously Massive Uncertainty: Slide 9 of 22 / RdN ©
Error Data from U.S Source: MITRE CAASD and FAA Massive Uncertainty: Slide 10 of 22 / RdN ©
Error Data from U.S Source: MITRE CAASD and FAA Massive Uncertainty: Slide 11 of 22 / RdN ©
Actual 2008 traffic compared to that forecast in 2004 TAF Source: US FAA
Massive Uncertainty: Slide 13 of 22 / RdN © Actual vs. Forecast 10 years earlier Source: FAA Aerospace Forecast FY 2006-2017 Note: These are aggregate data, in which greater local variations tend to cancel each other out Massive Uncertainty: Slide 13 of 22 / RdN ©
Older FAA Forecasts vs. Actual Data (% Difference) Review of the FAA 1982 National Airspace System Plan Massive Uncertainty: Slide 14 of 22 / RdN ©
Forecast vs. Actual International Pax in Japan Massive Uncertainty: Slide 15 of 22 / RdN ©
Forecast vs. Actual International Pax to Japan Massive Uncertainty: Slide 16 of 22 / RdN ©
Massive Uncertainty: Slide 17 of 22 / RdN © Notice the Pattern! Forecasting is an exercise in projecting past into future – …like steering car by looking into rear view mirror! Past low growth => under estimation Past high growth => over estimation Almost never right! Massive Uncertainty: Slide 17 of 22 / RdN ©
Forecast vs Actual International Pax in Sydney Massive Uncertainty: Slide 18 of 22 / RdN ©
ACTIVE GENERAL AVIATION AIRCRAFT 1980 FORECAST Massive Uncertainty: Slide 19 of 22 / RdN ©
ACTIVE GENERAL AVIATION AIRCRAFT 1985 FORECAST Massive Uncertainty: Slide 20 of 22 / RdN ©
1990 FORECAST ACTIVE GENERAL AVIATION AIRCRAFT Massive Uncertainty: Slide 21 of 22 / RdN ©
1998 FORECAST ACTIVE GENERAL AVIATION AIRCRAFT Massive Uncertainty: Slide 22 of 22 / RdN ©
Summary and Recommendations Forecast Errors have been large Likely to continue Recommendations: Expensive Forecasting is cost-ineffective Use general trends ...With large ranges Flexible Approach to Planning!!! Massive Uncertainty: Slide 23 of 22 / RdN ©