1. Professor of Engineering Systems Civil and Environmental Engineering
Massive Uncertainty
Richard de Neufville
Professor of Engineering Systems
and of
Civil and Environmental Engineering
Massive Uncertainty: Slide 1 of 22 / RdN ©

2. Massive Uncertainty: Slide 2 of 22 / RdN ©
Objective: To present realistic context of forecasting exercise
Topics
Evidence
Consequences
Massive Uncertainty: Slide 2 of 22 / RdN ©

3. Massive Uncertainty: Slide 3 of 22 / RdN ©
Causes of Uncertainty
Underlying variability of phenomenon
Difficulties in measurement or estimation
Unforeseen or “unpredictable” circumstances
Limits to valid measurement
for example: behavioral patterns
Massive Uncertainty: Slide 3 of 22 / RdN ©

4. Massive Uncertainty: Slide 4 of 22 / RdN ©
Evidence
1. Simple Physical Systems
2. Overall Traffic
3. Local Traffic (Worse)
4. Other Operations
Massive Uncertainty: Slide 4 of 22 / RdN ©

5. Ratio of Real Costs to Estimated Costs for Airport Projects
Costs expressed in constant dollars
Median ~= 1.25
Percent of Occurrences
Real/Estimated Cost Ratio
Massive Uncertainty: Slide 5 of 22 / RdN ©

6. Cost Growth for Various Projects
3.0
2.5
2.0
1.5
1.0
0.5
0.0
Ratio of Actual to Estimated Cost
DOD60 HWAY WATER BLDNG DOD50 ADHOC MAJOR ENRGY NASA NASA AVG St.Dev.
CONST SAT m g
Massive Uncertainty: Slide 6 of 22 / RdN ©

7. NASA Projects Cost Growth
3.5
3.0
2.5
2.0
1.5
1.0
0.5
0.0
Ratio of Actual to Estimated Cost
HST GLL UARS GRO COBE MGL MOBS LSAT EUVE ERBE AVG St.Dev.
Massive Uncertainty: Slide 7 of 22 / RdN ©

8. Massive Uncertainty: Slide 8 of 22 / RdN ©
Results of a 2004 study
Adapted from: Terminal Area Forecast (TAF) Accuracy Assessment Results
Jerome Friedman, MITRE CAASD, Sept. 30, 2004
Massive Uncertainty: Slide 8 of 22 / RdN ©

9. Massive Uncertainty: Slide 9 of 22 / RdN ©
Results of a 2004 study
Note: Average error ~ 11%
Adapted from: Terminal Area Forecast (TAF) Accuracy Assessment Results
Jerome Friedman, MITRE CAASD. Study dated Sept. 30, 2004, but data until Deliberate omission of 2001, 2002 – when traffic dropped enormously
Massive Uncertainty: Slide 9 of 22 / RdN ©

10. Error Data from U.S Source: MITRE CAASD and FAA
Massive Uncertainty: Slide 10 of 22 / RdN ©

11. Error Data from U.S Source: MITRE CAASD and FAA
Massive Uncertainty: Slide 11 of 22 / RdN ©

12. Actual 2008 traffic compared to that forecast
in 2004 TAF Source: US FAA

13. Massive Uncertainty: Slide 13 of 22 / RdN ©
Actual vs. Forecast 10 years earlier Source: FAA Aerospace Forecast FY
Note: These are aggregate data, in which greater local variations tend to cancel each other out
Massive Uncertainty: Slide 13 of 22 / RdN ©

14. Older FAA Forecasts vs. Actual Data (% Difference)
Review of the FAA 1982 National Airspace System Plan
Massive Uncertainty: Slide 14 of 22 / RdN ©

15. Forecast vs. Actual International Pax in Japan
Massive Uncertainty: Slide 15 of 22 / RdN ©

16. Forecast vs. Actual International Pax to Japan
Massive Uncertainty: Slide 16 of 22 / RdN ©

17. Massive Uncertainty: Slide 17 of 22 / RdN ©
Notice the Pattern!
Forecasting is an exercise in projecting past into future – …like steering car by looking into rear view mirror!
Past low growth => under estimation
Past high growth => over estimation
Almost never right!
Massive Uncertainty: Slide 17 of 22 / RdN ©

18. Forecast vs Actual International Pax in Sydney
Massive Uncertainty: Slide 18 of 22 / RdN ©

19. ACTIVE GENERAL AVIATION AIRCRAFT
1980 FORECAST
Massive Uncertainty: Slide 19 of 22 / RdN ©

20. ACTIVE GENERAL AVIATION AIRCRAFT
1985 FORECAST
Massive Uncertainty: Slide 20 of 22 / RdN ©

21. 1990 FORECAST ACTIVE GENERAL AVIATION AIRCRAFT
Massive Uncertainty: Slide 21 of 22 / RdN ©

22. 1998 FORECAST ACTIVE GENERAL AVIATION AIRCRAFT
Massive Uncertainty: Slide 22 of 22 / RdN ©

23. Summary and Recommendations
Forecast Errors have been large
Likely to continue
Recommendations:
Expensive Forecasting is cost-ineffective
Use general trends
...With large ranges
Flexible Approach to Planning!!!
Massive Uncertainty: Slide 23 of 22 / RdN ©