1 Impact of methodological choices on road safety ranking SAMO conference: 20/06/07 Elke Hermans: Transportation Research Institute - Hasselt University (Belgium)

2 Overview 1. Introduction to indicators 2. Introduction to road safety 3. The road safety index process 4. Study design 5. Method: UA & SA 6. Results 7. Conclusions

3 1. Introduction to indicators Popular concept Represent large amount of information Communicate simplified, clear message Use: trends, bottlenecks, policy targets and priorities, communication, … # indicators aggregation in 1 index (e.g. TAI, IMI) Road safety index = new, challenging and necessary matter!

4 2. Introduction to road safety Causes of accidents and casualties: human-vehicle-environment International literature defines risk domains: –Alcohol and drugs –Speed –Protective systems –Vehicle park –Infrastructure –Trauma management –Others: youth, VRUs, DRL, tiredness, … Country-specific factors (low policy impact)

5 General framework Social cost Final outcomes Safety Performance Indicators Billions of EUROs Safety measures and safety programs injury acc fatalities seriously inj slightly inj. Belgium,

6 3. The road safety index process A methodologically sound road safety index (RSI) will be developed Objective: –Comprehensive presentation of information –Better understanding of accident process –Comparing RS performance of regions –Measuring progress to objectives –Supporting policy by means of specific actions

7 Sweden United Kingdom The Netherlands Switzerland Denmark Germany Ireland France Austria Belgium Estonia Slovenia Spain Hungary Czech Republic Greece Portugal Poland Alcohol & drugs  % road users < BAC limit Speed  % road users < speed limit Protective systems  seatbelt wearing % in front Visibility  daytime running lights law Vehicle  % cars < 6 years Trauma management  health expenditure as GDP% Infrastructure  network density CASUALTIESCASUALTIES ACCIDENTSACCIDENTS Imputation Indicator selection Road Safety Index Road Fatality Ranking Indicators Normalisation Weighting Aggregation Uncertainty and sensitivity analysis Index process The RSI framework

8 4. Study design 3 methodological aspects –Weighting method: AHP or BA –Expert: 9 RS experts assigned weights –Indicator set: 7 or 6 RS indicators RSI = ∑ stand. ind.values x weights Output = avg. Δ in country ranking based on RSI compared to RFR

9 Dataset 7 road safety indicators Data available for 18 European countries (≠ sources) Zwitserl.

10 5. Method: UA & SA UA & SA are essential for indexes –Several subjective choices are made –Focus on ranking and 1 position –Offer correct and robust results UA estimates uncertainty in output taking into account uncertainty in input SA studies how uncertainty in output can be apportioned to different sources of uncertainty Global variance based sensitivity method Factors prioritisation setting SIMLAB

11 Step-by-step analysis (Saltelli et al., 2004) Output = average shift in rank 3 input factors: weighting, exp., ind. Uniform distributions Extended FAST method Generation of 10,000 x 3 sample Calculation of 10,000 output values Analysis of the output Conclusions

12 M = F 11 F 12 F 13 F 21 F 22 F 23 ……… F N1 F N2 F N3 e.g. F 21 = BA F 22 = expert 2 F 23 = all 7 indicators W = [0.286; 0.429; 0.071; 0.000; 0.071; 0.071; 0.071] Z AT = [0.14; 0.62; -0.17; 1.30; -0.21; -0.32; 1.35] … Z UK = [0.99; -0.05; 0.97; -1.46; 0.58; -0.96; -1.31] CountryOrder of rank RSI Road fatality ranking AT79 ……… UK42 for row 2 Determining the output

13 6. Results UA: output distribution μ = 5.64 σ = 0.75 Large ≠: –More and better ind. –Small EU data set BA; exp. 6; 6 indic. (no infrastr.)

14 Results (2) SA: first order and total effect index for each input factor FactorSiSi S ti Weighting method Expert selection Indicator selection Sum

15 7. Conclusions Importance and usefulness of UA & SA has been shown  essential part in the RSI development process Set of indicators is most influencing input factor  focus on theoretical framework and indicator selection Expert selection and weighting method had an impact mostly by interaction effects Weighting method BA or AHP had the least impact but they have some similarities These three aspects proved important in other studies as well

16 Further research Incorporate more methodological aspects: normalisation, imputation, aggregation and more possible weighting techniques in UA & SA Other output of interest: country level Methodological adaptations