1. Hugh Cronin Jim Scheer

2. How to determine the accuracy of CBA Sources of error Distribution of net benefits over time The Coquihalla Highway CBAs Conclusion Tips for dealing with error in CBA

3. ex ante (before) in medias res (during) ex post (after) Project inception Project implementation Project completion Study 1 – ex anteStudy 2 – in media resStudy 3 – ex post Benefits$394.6$639.4$1,307.8 Costs$354.4$767.6$913.5 Net benefits$40.2-$128.2$394.3

4. ErrorFactors contributing to increasing levels of error Note! Omissions (Double counting) Impacts excluded(e.g. environmental) Counting benefits in primary and secondary markets Reduces asprojects evolve (knowledge increase) Forecasting Complex/ changing project Human response unpredictable Cognitive biases (perception, emotion) Always wrong! Potentialfor ex post error remains (counterfactual) Also reduce over time Valuation Shadow price determination Assumptions usually debatable Reduceover time Measurement Inaccurate observation or interpretation Perceivedas relatively less important than other errors

5. Distribution of ex ante and ex post estimates of net benefits Net benefits Time

6. ErrorFactors contributing to increasing levels of error Note Omissions(Double counting) 1.Opportunity cost of land 2.Cost of environmental protection 1.Low impact on result 2.Mitigation construction cost included but no pollution costs etc. Incl. r Forecasting1.Future Traffic volumes 2.Time and distance savings ( educed congestion on alt. routes) 3.Accident rates 1.Refined over time (toll receipts) 2.Range of estimates and methods 3.Range ofestimates Value % Valuation1.of time savings 2.Valuing safety benefits (accident, injury, fatality) 3.Terminal value of asset 1.Average wages (% of wage for leisure) 2.New research led to vast change in assumptions 3.of construction cost accepted method Measurement1.Maintenance 2.Construction costs 1.Misinterpretation ofmaint. quote (km v lane km) 2.Clearer expost (reduced strategic bias)

7. Disconcertingly large errors in estimation for a relatively straightforward project. ̵ Many potential sources of error ̵ Categorisation helps Importance of generating independent values for costs (strategic bias minimisation) Should one in ten ex ante CBAs be subject to in medias res or ex post follow-up and comparison to ex ante analysis? “Prediction is very difficult, especially about the future.” Niels Bohr (attributed)

8. Time spent thinking about possible sources of error is time well spent – framework available At pre-project stage think not only about which costs and benefits but how they will be measured..and ensure data stream exists − Programme design to enable measurement Understanding what factors a CBA is sensitive to (sensitivity analysis) – presenting scenarios seems sensible!